gpu_accelerated_vm_with_kvm

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision		 Previous revision
gpu_accelerated_vm_with_kvm [01.10.2018 18:07] – [3rd Try: Nice DCV] Pascal Sutergpu_accelerated_vm_with_kvm [29.01.2023 22:39] (current) – [Re-Try RDP + passthrough] Pascal Suter
Line 162: Line 162:
 in order to get the middle mous button (wheel) to click, i had to modify the config of my citrix receiver on my client machine. I had to edit the file ''$HOME/.ICAClient/wfclient.ini'' and set the entry for ''MouseSendsControlV'' to ''False''. This setting is helpful if you are used to copy/paste by selecting text and then clicking the mouse wheel on the target text entry field. however it is useless if you CAD software makes use of the middle mous click to rotat your Model :)  in order to get the middle mous button (wheel) to click, i had to modify the config of my citrix receiver on my client machine. I had to edit the file ''$HOME/.ICAClient/wfclient.ini'' and set the entry for ''MouseSendsControlV'' to ''False''. This setting is helpful if you are used to copy/paste by selecting text and then clicking the mouse wheel on the target text entry field. however it is useless if you CAD software makes use of the middle mous click to rotat your Model :) 
 ===== GPU Forwarding for KVM ===== ===== GPU Forwarding for KVM =====
 +First of all you need to enable VT-d and VT-x or they might be also called something like Intel Virtualization tech, in the BIOS. without these guys you won't be able to load the vfio module later on. 
 +
 Helpful links:  Helpful links: 
   * [[https://medium.com/@calerogers/gpu-virtualization-with-kvm-qemu-63ca98a6a172]]   * [[https://medium.com/@calerogers/gpu-virtualization-with-kvm-qemu-63ca98a6a172]]
Line 279: Line 281:
  
 so far i could only test this from a very slow connection in a hotel abroad and it is amazing how much faster both DCV and HP RGS are compared to normal RDP. DCV seems to perform very similar or maybe even a bit better than RGS with this extremely low bandwidth (too low really to use any sort of remote desktop with full HD, not to mention 4k).. so at the moment, except for the high priche tag, DCV is in my eyes the winner so far.  so far i could only test this from a very slow connection in a hotel abroad and it is amazing how much faster both DCV and HP RGS are compared to normal RDP. DCV seems to perform very similar or maybe even a bit better than RGS with this extremely low bandwidth (too low really to use any sort of remote desktop with full HD, not to mention 4k).. so at the moment, except for the high priche tag, DCV is in my eyes the winner so far. 
 +
 +===== retry DCV =====
 +dcv was running very unreliably.. it crashed often when i went to full screen with 4k and it had performance problems with resolutions in the 4k range.. i noticed however, that my method of copying the windows image and then first uninstall the HP stuff before i installed DCV was probably not the best as for example the display driver for the virtual display was not called DCV Display but something else, so i think there might have been some leftovers around which interfere with DCV.. so in order to get rid of all that i will retry the dcv installation with a new windows install. 
 +also i noticed that the performance of my virtual disk was lousy (10MB/s) so i am now using an LVM volume on a physical disk rather than an image on a zfs storage. 
 +
 +here is the command line to start the vm: 
 +
 +  qemu-system-x86_64 -enable-kvm -k de-ch -name windows -drive format=raw,file=/dev/vms/dcv,if=virtio,format=raw,index=0 -m 9000 -device virtio-net-pci,netdev=net0,mac=52:54:00:01:00:01 -netdev tap,id=net0 -cdrom Win10_1803_English_x64.iso -drive file=virtio-win.iso,media=cdrom,index=1 -monitor tcp:127.0.0.1:5801,server,nowait -boot once=d -cpu host,kvm=off -smp cpus=8,cores=8 -device vfio-pci,host=05:00.0 -monitor vc -drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd -drive if=pflash,format=raw,file=/bulk/vm/OVMF_VARS.windows10.fd -rtc base=localtime -balloon none -serial none -parallel none -M q35 -vnc :1
 +  
 +as far as speed goes, this worked out a lot better, however i could not get DCV to run reliably. 
 +
 +this time i've created a snapshot prior to installing DCV and i later cloned that snapshot to install RGS on a "plain" clone of the windows install in a second LVM volume. Here is my command to start the RGS VM: 
 +  qemu-system-x86_64 -enable-kvm -k de-ch -name windows -drive format=raw,file=/dev/vms/rgs,if=virtio,format=raw,index=0 -m 9000 -device virtio-net-pci,netdev=net0,mac=52:54:00:01:00:01 -netdev tap,id=net0 -cdrom Win10_1803_English_x64.iso -drive file=virtio-win.iso,media=cdrom,index=1 -monitor tcp:127.0.0.1:5801,server,nowait -boot once=d -cpu host,kvm=off -smp cpus=8,cores=8 -monitor vc -drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd -drive if=pflash,format=raw,file=/bulk/vm/OVMF_VARS.windows10.fd -rtc base=localtime -balloon none -serial none -parallel none -M q35 -vnc :1 -device vfio-pci,host=05:00.0
 +
 +RGS runs smooth and windows feels reasonably fast too with the SSD as storage behind it. however, When the vm is left running over night usually the next morning it hangs or is so slow that one often can't even shut it down properly. I don't know why that is, i've deactivated all power saving features of windows, screen blanking etc. but I still have this issue. The machine hangs the same when I connect vie windows RDP. Also in those cases where i can still see something, I usually get alot of horizontal lines as artefacts on the screen both with RGS and RDP. The only solution is then to shut down the VM and start it again, a Reboot (instead of a power cycle) will make the VM fast again but the Artefacts remain. so there is definately something wrong with my VM here. This needs further investigation! 
 +
 +==== LVM stuff ====
 +just as a quick side note: here is how you create and revert to a snapshot in lvm.  using dd you can also copy a snapshot into a new lvm volume if the new volume is at least the size of the snapshotted one. 
 +
 +create a snapshot: 
 +  lvcreate -s -L 50G -n win10base /dev/vms/dcv
 +revert to a snapshot: 
 +  lvconvert --merge /dev/vms/win10base
 +
 +==== using a vm with just the forwarded GPU ====
 +this is the ultimate goal. so far i could not get this to work, whenver i booted a vm without a emulated graphics card i could no longer connect to it. and it was hard to debug as i had no "local" screen to connect to. however, i recently connected a monitor to the GPU and found out, that the reason why i could not connect was in fact very simple: once the emulated graphics adapter is no longer presence, the pci-id of the emulated NIC changes and windows detects it as a new ethernet adapter despite it having the same MAC. So the vm actually got a DHCP assigned ip rather than the fixed one i had configured. I could then connect to the dhcp assigned ip and set the same fixed ip on the "second" interface which now works fine. 
 +
 +to start a vm without a emulated graphics card, use ''-nographic -vga none'' in your command line. 
 +
 + qemu-system-x86_64 -enable-kvm -k de-ch -name windows -drive format=raw,file=/dev/vms/dcv,if=virtio,format=raw,index=0 -m 9000 -device virtio-net-pci,netdev=net0,mac=52:54:00:01:00:01 -netdev tap,id=net0 -cdrom Win10_1803_English_x64.iso -drive file=virtio-win.iso,media=cdrom,index=1 -monitor tcp:127.0.0.1:5801,server,nowait -boot once=d -cpu host,kvm=off -smp cpus=8,cores=8 -monitor vc -drive if=pflash,format=raw,readonly,file=/usr/share/OVMF/OVMF_CODE.fd -drive if=pflash,format=raw,file=/bulk/vm/OVMF_VARS.windows10.fd -rtc base=localtime -balloon none -serial none -parallel none -M q35 -vga none -nographic -device vfio-pci,host=05:00.0
 +
 +==== stability issues ====
 +i go to the point where whenever i booted my DCV VM the HOST system rebooted or rather reset! Finally I found a workaround that fixed this issue: I bought a little dummy hdmi display emulator which is basically an HDMI plug that you plug into the output of the graphics card and it emulates a display. Since i had no adapters readily available for my K1200 i switched to a GTX 1050Ti which i had in my other pc. So far it works the same as the K1200 did. DCV now starts and is usable same as RGS was before. However i Still have the issue, where after about a day or so, when i log back into my VM i get horizontal black lines across my remote desktop which move around as i re-size the window but stay stady "above" the various windows of Windows. This happend with RGS as well as with DCV and even when I use windows RDP! Usually the vm also becomes increasingly unresponsive when i try to re-connect after having this issue. 
 +
 +I have now updated my Bios hosts bios to the latest version (it was actually quite outdated already) and have also enabled Coherency support for the IOMMU stuff in the bios. I usually don't like changing two parameters at once to sovle an issue, but for now lets just try it as it takes at least a day to reproduce the problem.. i can then roll back the bios setting and see if it was the update alone that resolved the issue should my current trials be successful. 
 +
 +===== Re-Try RDP + passthrough =====
 +the key to a smooth user experience as far as framerate etc. goes seems to be to use AVC444 which on Windows supports NVENC (using the Nvidia Card to encode the video stream for RDP) 
 +
 +to enable this on the windows side: 
 +  - Open Run (Win+R). Type gpedit.msc
 +  - Computer Configuration -> Administrative Templates -> Windows Components -> Remote Desktop Services -> Remote Desktop Session Host -> Remote Session Environment
 +    - Enable "Prioritize H.264/AVC 444 Graphics mode for Remote Desktop..."
 +    - Enable "Configure H.264/AVC hardware encoding for Remote Desktop..."
 +
 +
 +on the client side with xrdp, get the latest version (i worked with 3.0) from https://www.freerdp.com/ where they offer pre-built binaries for many distributions. the binaries are built on a daily basis and include the stable branch. with that, use the ''/gfx:AVC444'' or ''/gfx-h264:AVC444'' option to connect to the server 
 +  /opt/freerdp-nightly/bin/xfreerdp /gfx-h264:AVC444 /u:"Pascal Suter" /v:192.168.168.101 /dynamic-resolution
 +
 +to verify that nvenc is being used, open the task manager and go to the performance tab. the GPU should show video encoding workload. 
  
  
  • gpu_accelerated_vm_with_kvm.1538410076.txt.gz
  • Last modified: 01.10.2018 18:07
  • by Pascal Suter