Le blog de Jean David TECHER, un Réunionnais à Saint-Priest/Lyon

Here you will find required patches and notes about NVIDIA and VGA Pass Through.

Please be informed that

ABOUT PATCHES I am not the author who wrote the original patches. I've just tried to maintain the patches I found from Xen mailing-list. Since Summer 2011, I've begun to test Xen VGA PassThrough I try to test patches once a week (week-end) LAST UPDATE: 03/25/2012 / LAST XEN REVISION: 25099 BEFORE YOUR TEST You have to read the required documentation taken from Xen Wiki: Xen VGA PassThrough You must have the specific hardware before doing any test: NVIDIA graphic card, specific processor Intel (Vt-d) or AMD (IOMMU), a specific motherboard VT-d capable

1. Examples: Video for playing Crysis 2 single player and screenshots for Windows XP

1. Here is a (poor) video for Crysis 2 http://www.youtube.com/watch?v=3SaYO0ERW44 I did using my old GT 440 (before I bought my GTX 460).
2. Here are a few screenshots using CPUZ64
   
   
   

2. Tested Hardware

Item	Reference	Comment
Processor	Intel(R) Core(TM) i5-2400 CPU @ 3.10GHz
Motherboard	Manufacturer: MSI / Product Name: H61MU-E35 (MS-7680)	Bios Version V10.11
Graphic Card (*)	GTX 460	GTX 460 SE EVGA 1024 MB 01G-P3-1366-ET
Graphic Card	GTX 560 TI	GTX 560 TI Point Of View 2048 MB GDDR5

(*) : It was my previous card ( before 01/09/2012)

3. Xen Patches

All patches are applied from xen-unstable staging (http://xenbits.xensource.com/staging/xen-unstable.hg/)

From Revision	To Revision	Kernel / Linux Distrib	Download	Comments
25099	24888	3.1.10 / Debian 6.0.4 64 bits	hhttp://www.davidgis.fr/download/xen-4.2_rev24798_gfx-passthrough-patchs.tar.bz2	A new file tools/firmware/hvmloader/rombios.c has been pushed by Julian (maintainer). A lot of things have been moved to this new file (revisions from 24782 to 24785)
24563	24779	3.1.10 / Debian 6.0.2 64 bits	http://www.davidgis.fr/download/xen-4.2_rev24463_gfx-passthrough-patchs.tar.bz2	Path for patch 'pass-through.c' has changed from 'tools/ioemu-remote/hw/pass-through.c' to 'tools/qemu-xen-traditional-dir-remote/hw/pass-through.c'
24407	24547	3.1.4, 3.1.6, 3.1.8 / Debian 6.0.2 64 bits	http://www.davidgis.fr/download/xen-4.2_rev24232_gfx-passthrough-patchs.tar.bz2	May not work from 24492 to 24519 due to revision 24492. Fixed in revision 24520
24232	24407	3.1.4 / Debian 6.0.2 64 bits	http://www.davidgis.fr/download/xen-4.2_rev24232_gfx-passthrough-patchs.tar.bz2
24014	24228	3.1.0 / Debian 6.0.2 64 bits	http://www.davidgis.fr/download/gfx_patchs_gtx-460-se-evga.tar.bz2
23799	23799	2.6.39.3 / Debian 6.0.2 64 bits	http://www.davidgis.fr/download/gfx_patch.tar.bz2

4. Linux Kernel .config

My tests are based on a simple Debian 6.0.2 64 bits.

Notice: since 3.1.0, I used to compile Xen including all "Xen capabilitites" in the kernel (=Y) not as modules (=M).

root@mercury:~# grep -i xen /boot/config-3.1.6
CONFIG_XEN=y
CONFIG_XEN_DOM0=y
CONFIG_XEN_PRIVILEGED_GUEST=y
CONFIG_XEN_PVHVM=y
CONFIG_XEN_MAX_DOMAIN_MEMORY=128
CONFIG_XEN_SAVE_RESTORE=y
# CONFIG_XEN_DEBUG_FS is not set
# CONFIG_XEN_DEBUG is not set
CONFIG_PCI_XEN=y
CONFIG_XEN_PCIDEV_FRONTEND=y
CONFIG_XEN_PCIDEV_FE_DEBUG=y
CONFIG_XEN_BLKDEV_FRONTEND=y
CONFIG_XEN_BLKDEV_BACKEND=y
CONFIG_NETXEN_NIC=y
CONFIG_XEN_NETDEV_FRONTEND=y
CONFIG_XEN_NETDEV_BACKEND=y
CONFIG_INPUT_XEN_KBDDEV_FRONTEND=y
CONFIG_HVC_XEN=y
CONFIG_XEN_WDT=y
CONFIG_XEN_FBDEV_FRONTEND=y
# Xen driver support
CONFIG_XEN_BALLOON=y
CONFIG_XEN_SELFBALLOONING=y
CONFIG_XEN_BALLOON_MEMORY_HOTPLUG=y
CONFIG_XEN_SCRUB_PAGES=y
CONFIG_XEN_DEV_EVTCHN=y
CONFIG_XEN_BACKEND=y
CONFIG_XENFS=y
CONFIG_XEN_COMPAT_XENFS=y
CONFIG_XEN_SYS_HYPERVISOR=y
CONFIG_XEN_XENBUS_FRONTEND=y
CONFIG_XEN_GNTDEV=y
CONFIG_XEN_GRANT_DEV_ALLOC=y
CONFIG_XEN_PLATFORM_PCI=y
CONFIG_SWIOTLB_XEN=y
CONFIG_XEN_TMEM=y
CONFIG_XEN_PCIDEV_BACKEND=y

Here are my configs

Kernel	Download	Comments
3.1.10	http://www.davidgis.fr/download/config-3.1.10.tar.bz2	Tested 01/28/2012
3.1.8	http://www.davidgis.fr/download/config-3.1.8.tar.bz2	Tested 01/10/2012
3.1.6	http://www.davidgis.fr/download/config-3.1.6.tar.bz2	Tested 12/25/2011
3.1.4	http://www.davidgis.fr/download/config-3.1.4.tar.bz2	Tested date < 12/25/2011
3.1.0	http://www.davidgis.fr/download/config-3.1.0.tar.bz2
2.6.39.3	http://www.davidgis.fr/download/config-2.6.39.3.tar.bz2

5. Tested domU

domU	Graphic card	Nvidia driver (*)	Comments
Windows XP 32 / 64 Bits	GTX 460 SE EVGA 1024 MB GTX 560 Ti Point Of View 2048 MB GDDR5	275.33 275.50	[+] Crysis 2 works very well, single player mode, resolution 1920x1080 [-] It it not possible to restart domU without restarting dom0
Ubuntu Lucid 10.04.4 64 Bits	GTX 460 SE EVGA 1024 MB GTX 560 Ti Point Of View 2048 MB GDDR5	275.28 aka NVIDIA-Linux-x86_64-275.28.run 275.43 aka NVIDIA-Linux-x86_64-275.43.run	[+] Quake 3 works very well using ioquake3 but you need to build ioquake3 from sources svn co svn://svn.icculus.org/quake3/trunk ioquake3 cd ioquake3 make If you've got the following error LD build/release-linux-x86_64/renderer_opengl1_x86_64.so /usr/bin/ld: cannot find -lGL collect2: ld returned 1 exit status just do rm -vf /usr/lib/libGL.so ln -s /usr/lib/libGL.so.275.28 /usr/lib/libGL.so make cd build/release-linux-x86_64/ cp -rvf . /home/david/ioquake3/ [+] It is possible to restart domU without restarting dom 0 but Linux domU needs to be the first domU started - for HVM domU list - since dom0 booted

(*): This is the required version, may not work with version >= 280.XX

6. Quick instructions to install Xen with patches

1. Extract your EEPROM from your NVIDIA graphic card

   Use nvtools or your favorite tool (Please Google for that).

   Optional way If you've got a USB Key (small size), you can use the following way. I made a ISO for nvflash 5.100 so you can boot from a USB Key.
   • Plug your USB Key
   • Download iso

     wget http://www.davidgis.fr/download/nvflash_5.100.1_usb.iso.tar.bz2
     tar xvzf nvflash_5.100.1_usb.iso.tar.bz2

   • Now copy iso to the USB Key

     Let's suppose that we have the USB key mounted on /dev/sdg1

     david@gemini:~$ mount |grep media
     /dev/sdf1 on /media/lmmmmnmkddc type vfat (rw,nosuid,nodev,uhelper=udisks,uid=1000,gid=1000,shortname=mixed,dmask=0077,utf8=1,flush)
     /dev/sdg1 on /media/LEXAR type vfat (rw,nosuid,nodev,uhelper=udisks,uid=1000,gid=1000,shortname=mixed,dmask=0077,utf8=1,flush)
     

     Umount

     umount /media/LEXAR

     The magic command

     cat nvflash_5.100.1_usb.iso > /dev/sdg

     Notice: There is no error. It is /dev/sdg and not /dev/sdg1 in the command above

     Unplug the USB Key. Boot on the USB Key using BIOS and use

     • nvflash.exe --list

       to find your card

     • nvflash.exe --save vgabios.rom

       to get your EEPROM
     • Use [CTRL]+[ALT]+[DEL] to quit.

   Once it is done,copy file vgabios.rom somewhere on your computer and rename it to vgabios-pt.bin

2. Download Xen form staging

   rev=24407;hg clone -r $rev http://xenbits.xensource.com/staging/xen-unstable.hg/ xen-unstable.hg-rev-${rev}
   

3. Make a init compil in tools

   cd xen-unstable.hg-rev-24407/
   cd tools/
   make 
   make clean
   cd ..
   

4. Download patches and apply (See section 3 "Xen Patches")

   wget -q http://www.davidgis.fr/download/xen-4.2_rev24232_gfx-passthrough-patchs.tar.bz2
   tar xvjf xen-4.2_rev24232_gfx-passthrough-patchs.tar.bz2 
   for file in $(ls xen-4.2_rev24232_gfx-passthrough-patchs/*);do patch -p1 < $file;done
   

5. Modify the values assigned to your graphic card to tools/firmware/hvmloader/acpi/dsdt.asl. So here is an example for my own video graphic card
   • In order to getting the required values, first of all find the identifier for your card

     root@mercury:~# lspci |grep VGA
     01:00.0 VGA compatible controller: nVidia Corporation Device 0e23 (rev a1)
     

   • Use dmesg to get the required range

     root@mercury:~# dmesg | grep 01:00.0 | grep BAR
     [    2.883158] pci 0000:01:00.0: BAR 0: reserving [mem 0xf8000000-0xf9ffffff flags 0x40200] (d=0, p=0)
     [    2.883161] pci 0000:01:00.0: BAR 1: reserving [mem 0xd0000000-0xd7ffffff flags 0x14220c] (d=0, p=0)
     [    2.883163] pci 0000:01:00.0: BAR 3: reserving [mem 0xd8000000-0xdbffffff flags 0x14220c] (d=0, p=0)
     [    2.883166] pci 0000:01:00.0: BAR 5: reserving [io  0xe000-0xe07f flags 0x40101] (d=0, p=0)
     

     The 3 ranges [0xf8000000-0xf9fffff], [0xd0000000-0xd7ffffff], [0xd8000000-0xdbfffff] need to be specify in dsdt.asl.patch.

     You need to use the formula

     Maximum - Minimum + 1 for the difference

   • Let' have a try for the first range [0xf8000000-0xf9fffff]

     Maximum = (0xf9fffff) / Minimum = 0xf8000000 => Difference = 4194303999 - 4160749568 + 1 = 33554432 => hexadecimal = 0x02000000

     so for the first range

     Maximum = 0xf9fffff
     Minimum = 0xf8000000
     Difference = 0x02000000
     

     Do the same for the two other ranges.
   • Then open tools/firmware/hvmloader/acpi/dsdt.asl. Using your favorite editor (nano, vim...) ,try to find the text reserve MMIO BARs of gfx and put the required values

     ...
                         /* reserve MMIO BARs of gfx for 1:1 mapping */
                         DWordMemory(
                             ResourceProducer, PosDecode, MinFixed, MaxFixed,
                             Cacheable, ReadWrite,
                             0x00000000,
                             0xF8000000,
                             0xF9FFFFFF,
                             0x00000000,
                             0x02000000)
     
                         DWordMemory(
                             ResourceProducer, PosDecode, MinFixed, MaxFixed,
                             NonCacheable, ReadWrite,
                             0x00000000,
                             0xD0000000,
                             0xD7FFFFFF,
                             0x00000000,
                             0x08000000)
     
                         DWordMemory(
                             ResourceProducer, PosDecode, MinFixed, MaxFixed,
                             Cacheable, ReadWrite,
                             0x00000000,
                             0xD8000000,
                             0xDBFFFFFF,
                             0x00000000,
                             0x4000000)
     ...
     

6. Copy your vgabios-pt.bin (see point 1 - extracting EEPROM) in tools/firmware/vgabios/

cp -vf /root/xen-unstable.hg-rev-24109/tools/firmware/vgabios/vgabios-pt.bin tools/firmware/vgabios/vgabios-pt.bin

7. Compil and install xen

   make xen && make tools && make stubdom
   make install-xen && make install-tools PYTHON_PREFIX_ARG= && make install-stubdom && reboot
   

7. Notes for domU configuration

Here is the configuration of my domU

root@mercury:~# grep -vE '^(#|$)|#' /etc/xen/machines/mercury-xen03.cfg
builder='hvm'
memory = 3072
shadow_memory = 1024
name = "mercury-xen03"
vcpus='4'
vif = [ 'bridge=eth0,mac=00:14:3e:00:8f:c2' ]
disk = [
		'phy:/dev/xenvg/mercury-xen03-disk-6,xvda,w',		
		'phy:/dev/sr0,xvdc:cdrom,r',
         ]
boot="cd"
sdl=0
vnc=1
vnclisten="0.0.0.0"
vncconsole=1
vncpasswd=''
acpi=1
apic=1
xen_extended_power_mgmt=1
pae=1
arch='x86_64'
hpet = 1
hap = 1
viridian = 1
monitor=1
serial='pty'
keymap='fr'
soundhw = "all"
audio="on"
ne2000=0
on_poweroff = 'destroy'
on_reboot   = 'restart'
on_crash    = 'restart'
xen_platform_pci=1 
gfx_passthru=1
pci  = [ '01:00.0','00:1a.0','00:1d.0' ]
pci_msitranslate = 1
pci_power_mgmt = 1
acpi_s3 = 1
acpi_s4 = 1

Now we will use PCI-STUB in order to being able to boot the domU. Please refer to Xen VT-d Wiki Section "Binding Devices to pci-stub"

1. root@mercury:~# lspci | grep VGA
   01:00.0 VGA compatible controller: nVidia Corporation Device 0e23 (rev a1)

2. root@mercury:~# lspci -n | grep "01:00.0"
   01:00.0 0300: 10de:0e23 (rev a1)

3. Now we can boot the domU

   root@mercury:~# grep -vE '^(#|$)' start_windows.sh 
   echo "10de 0e23" > /sys/bus/pci/drivers/pci-stub/new_id
   echo "0000:01:00.0" > /sys/bus/pci/devices/0000:01:00.0/driver/unbind
   echo "0000:01:00.0" > /sys/bus/pci/drivers/pci-stub/bind
   xl   create /etc/xen/machines/mercury-xen03.cfg

Here are the configration files I use

1. Slon configuration: master

vac_frequency=3
log_level=1
sync_interval=1000
sync_interval_timeout=10000
sync_group_maxsize=900
sync_max_rowsize=8192
sync_max_largemem=5242880
syslog=0
log_pid=true
log_timestamp=true
log_timestamp_format='%Y-%m-%d %H:%M:%S %Z'
sql_on_connection="SET log_min_duration_statement TO '500';SET work_mem='1024MB';SET log_statement to 'mod';"
#cleanup_interval="240 minutes"
#cleanup_deletelogs = 'false'
cluster_name = 'XXXXXXXX'
conn_info = 'service=XXXX-YYYYYY-ZZZZZZZ'

2. Slon configuration: slave

vac_frequency=3
log_level=1
sync_interval=4000
sync_interval_timeout=10000
sync_group_maxsize=600
desired_sync_time=0
syslog=0
log_pid=true
log_timestamp=true
log_timestamp_format='%Y-%m-%d %H:%M:%S %Z'
sql_on_connection="SET log_min_duration_statement TO '500';SET work_mem='1024MB';"
#cleanup_interval="240 minutes"
#cleanup_deletelogs = 'false'
cluster_name = 'XXXXXXXX'
conn_info = 'service=XXXX-YYYYYY-ZZZZZZZ'

Mise à  jour du lundi 2011/12/05: J'ai parcouru ce lien et j'ai trouvé quelques réponses à  mes interrogations suite à  la lecture de

• http://forums.postgresql.fr/viewtopic.php?id=1688&p=1
• http://forums.postgresql.fr/viewtopic.php?pid=11447

Je ne vais pas rentrer dans le débat mais on sent la fracture annoncée.

L'Article en question

Ce n'est que ce soir en surfant sur le Web que j'ai pu découvrir cet article "Migration Oracle PostGreSQL : Les 13 grandes lacunes qui peuvent s'avérer cauchemardesque !" dont je recommande vivement la lecture Avant de commencer à  taper dessus, je me suis attardé à  lire les différents points dit critiques....Les débuts de l'article sont louables mais arrivé à  un certain point, celà  dérive complètement.

Du calme, du calme...restons professionnel

En tant que DBA, on apprend à  prendre les choses avec du recul et à  accepter les critiques. Débattre ici des différents points critiques abordés n'est pas le sujet de ce billet, car l'article ne m'apprend rien de nouveau.

1. De manière objective: L'auteur livre un tour de table intéressant et plus ou moins complet. A part les différences entre SQL Server et PostgreSQL, je n'ai vraiment rien appris de nouveau.(...)Cédric Villemain a répondu à  cet article: http://expert-postgresql.fr/blog/news/migrer-a-postgresql/ de manière correcte. Je me cantonnerais à  ses références, celà  m'évitera de dupliquer un billet
2. De manière subjective: A la lecture de cet article, vu qu'il est publié sur developpez/com dont le sérieux fait référence dans la communauté informatique et étant moi même DBA PostgreSQL depuis bien des années, j'ai vraiment du mal à  me poser la question de ce qui a pu pousser l'auteur à  dresser un article aussi noir de PostgreSQL. J'ai toujours apprécié le sérieux des articles de developpez.com mais là  on sent quelques pics assassines.Querelles personnelles avec des aficionnados PostgreSQL? Clients voulant migrer sous PostgreSQL à  cause de la crise? Motivations personnelles/professionnelles?

Si le but est d'être polémique, je dirais qu'avec moi, c'est raté - je n'y apprends rien de nouveau -. Je m'interroge juste sur de telles motivations car pour flinguer la base, faut y aller! Celà  démotive plus qu'autre chose que d'envisager une migration vers PostgreSQL. On a l'impression qu'Oracle et SQL Server sont les seuls SGDBR. Point à  la ligne. Voir même que la profession d'expert Oracle se sentirai menacer par une montée en puissance de PostgreSQL?

Défendre son gagne-pain est quelque chose de tout à  fait honorable. Me dire que la motivation de la rédaction d'un tel article fait suite à  des échanges houleux (ou je ne sais quoi) avec des extrémistes de PostgreSQL me semble pour le moins improbable. J'émets ici un doute sur une telle supposition car un expert aussi bon soit-il ne rédigerait pas un article suite à  une telle motivation.

Mise à  jour du lundi 2011/12/05: J'ai parcouru les autres articles rédigés par l'auteur et celà  me fait au moins plaisir de voir que oui finalement il ya des articles démagogiques.

Si le but est de réveiller les vieux démons de la sacro sainte guéguerre Oracle/PostgreSQL, Bof!!! Sans intérêt. J'émets de sérieux doutes quand à  la réelle motivation louable de cet article car en 8 ans je n'ai jamais vu un article aussi négatif sur PostgreSQL. J'espère que je me trompe! En tout cas, celà  ne me démotive pas de continuer à  travailler avec PostgreSQL. Le cà´té trop négatif de l'article m'a vraiment coupé l'envie de répondre techniquement à  tous les points abordés. Comme l'auteur, j'ai moi aussi un bagage bien costaud sous PostgreSQL mais là  faut pas pousser!

Since Slony 2.0.4 is the version bundled/supported by EnterpriseDB Advanced Server 8.4, here are the required steps to build againts Slony >= 2.1.0, without deleting Slony 2.0.4 binaries.

• Install the prerequisites

  sudo apt-get install perl build-essential bison flex autoconf

• Create a temporary symbolic link to libdir.

  sudo mkdir -pv /opt/slony/2.1.0
  sudo ln -s /opt/PostgresPlus/8.4AS/lib /opt/slony/2.1.0/lib
  

• Download Slony's sources

  cd /tmp/
  wget -q "http://git.postgresql.org/gitweb/?p=slony1-engine.git;a=snapshot;h=0a9346436b0369b887945219bd43b611fd79c6ca;sf=tgz" -O - | tar xzf -
  cd slony1-engine-0a93464/

• Configure, compile

  ./configure \
  --prefix=/opt/slony/2.1.0 \
  --with-pgconfigdir=/opt/PostgresPlus/8.4AS/bin/ \
  --with-perlsharedir=/opt/slony/2.1.0/lib \
  --with-pgsharedir=/opt/slony/2.1.0/share \
  --with-perltools=/opt/slony/2.1.0/scripts \
  --with-pglibdir=/opt/slony/2.1.0/lib \
  --with-pgpkglibdir=/opt/slony/2.1.0/lib
  
  grep -lriE '\$libdir' . | while read file;do sed -i 's:\$libdir:/opt/slony/2.1.0/lib:g' $file;done
  
  make
  

• Delete the symbolic link and install

  sudo rm -rf /opt/slony/
  sudo make install