vdi-stream-client

VDI Stream Client

A very tiny and low latency desktop streaming client for remote Windows guests with GPU passthrough which supports Nvidia NVENC, AMD VCE, VCN and Intel Quick Sync Video.

Overview

VDI Stream Client is a tiny and low latency Linux client which connects to Parsec Host running on Windows. It allows to run fully GPU accelerated Windows desktop environment on a remote machine while streaming the content to a local Linux system which process keyboard and mouse input. The remote server can be a virtual machine running on-top of on-premise deployments of Virtuozzo Hybrid Server, KVM or Xen. It can also connect to public clouds like Amazon (EC2 G3 Accelerated) or Microsoft Azure (NV6). It uses Simple DirectMedia Layer (SDL3) for low-level access to audio, video, keyboard, mouse and clipboard.

Motivation

I’m using Linux on Desktop since more than 20 years already for several reasons - customization, small and tiny floating window managers like Fluxbox, ideal platform for engineers and developers and many more. However over the past couple of years my daily job required more attention which rely on Microsoft Windows applications like Microsoft Office and full fledged groupware solution like Microsoft Outlook. There exist open source alternative like LibreOffice and Thunderbird plus various Exchange connectors or even SOGo. They are all good in their niche but they didn’t reach compatibility to satisfy me. I tried a lot of different ways of solving the problem (see below in the comparison table) but at the end I have only three main goals:

1. GPU passthrough with remote support. Using hosted remote Windows desktop with full-fledged GPU acceleration for 3D, video decoding and encoding to run any kind of application with full GPU support as they would have been used locally.
2. Low latency network connection. Using laptop running Linux to connect from anywhere to the Windows desktop and do my daily work. This must work via fast network like ethernet, roaming wireless and low bandwidth with unstable mobile broadband.
3. Being very energy efficient on client. The laptop battery drain should not significantly increase while working on the remote desktop. Also the server should not become more energy hungry than needed for simple streaming.

Comparison

In Linux ecosystem there exist already wide range of options to run Windows guest environment as virtual machine on-top of a Linux host and access either its local display or via remote display protocol, all of them have their pros and cons. Look at the table below for brief comparison.

Method	Local	Remote	3D
QXL with Spice	Yes	Yes	No
GPU Passthrough	Yes	No	Yes
KVM FrameRelay	Yes	No	Yes
iGVT-g	Yes	No	Yes (Intel only)
Virgil 3D	Yes	Yes	Yes (Linux only)
SPICE Streaming Agent	Yes	Yes	Yes (Linux only)
Moonlight	Yes	Yes	Yes (Nvidia only)
Sunshine	Yes	Yes	Yes
Parsec	Yes	Yes	Yes

So why another streaming client is needed if one for Windows, Linux and macOS exist already? Well the Parsec client is focusing on gaming capabilities while VDI Stream Client is focusing on having a reliable daily working environment. The table below gives a brief overview about differences.

Features	VDI Stream Client	Parsec
Keyboard Input	Full	Partial
Mouse Input	Full	Partial
Gamepad Input	No	Yes
Clipboard Sharing	Text only	Text only
Remote	Yes	Yes
DirectX	Yes	Yes
SDL Renderer	Yes	No
Resolution Sync	Host-to-Client	Client-to-Host
Alt+Tab Integration	Yes	No
No GUI	Yes	No
System SDL3	Yes	No
Auto Reconnect	Yes	No
Screensaver Integration	Yes	No
USB Redirection	Yes	No
Multi Monitor	Yes	Yes
Color Mode 4:4:4	Yes	Yes

Requirements

VDI Stream Client currently targets Linux on x86_64. The client renders frames through the SDL renderer and uses the Parsec SDK for transport, audio, input and session handling. H.264 (AVC) and H.265 (HEVC) video are decoded through FFmpeg by replacing the public Linux SDK’s hidden FFmpeg decoder entry with a mandatory vdi-stream-client owned decoder implementation. The original SDK software and hardware decoder entries are disabled so all video decoding remains in the injected FFmpeg decoder.

Any recent GPU with VA-API support can be used for FFmpeg hardware decoding. The client queries libva profile and decode-entrypoint metadata without encoding or decoding a test frame. The --video-decoder option selects the codec, color mode, acceleration policy and ordered fallbacks. Its default hw-hevc-444 mode tries hardware H.265 4:4:4, hardware H.265 4:2:0, hardware H.264 4:2:0 and software H.264 4:2:0 in that order. H.265 4:4:4 is enabled only when the VA-API device exposes a VAProfileHEVC*444 VLD profile with a matching YUV444 render-target format. With libplacebo and Vulkan, retained VA-API frames are imported as DRM PRIME DMA-BUFs and rendered without copying frame pixels through CPU memory.

Nothing exist without drawbacks and that one for Parsec is that it is mandatory to have an account which is completely free. However communication between client and host is always made directly. Also the SDK is only available as pre-compiled library, so for those who fully rely on an open-source system, stop reading here. Some features are only available with a commercial subscription under Parsec Warp. It applies to 4:4:4 color mode which is required to encode images and streams without chroma subsampling for sharp and crystal clear text.

Features

• Fully CLI based customization. User can specify all required parameters via command line switches, allow perfect integration into different window managers.
• Desktop in window support with SDL_SetWindowKeyboardGrab while leaving mouse pointer ungrabbed. Supporting Alt+Tab and Windows special keys grab, even if assigned to the window manager.
• SDL window is created without SDL_WINDOW_RESIZABLE flag set, so window cannot be resized by window manager and always uses native host resolution (no more blurry desktop).
• H.264 (AVC) and H.265 (HEVC) decoding through FFmpeg with VA-API hardware acceleration when available and automatic fallback to FFmpeg software decoding.
• Host to client resolution sync with automatic window resize. User can change resolution in Windows control panel and client polls for the changes and adjust client window size.
• Multi monitor mode, requesting up to NUM_VSTREAMS host monitor streams by default. The client enables additional Parsec video streams and creates or destroys matching SDL windows dynamically when monitors are added or removed in Windows Display Settings. Use --monitors SPEC to restrict monitors or override resolutions.
• Specify per-monitor resolution via command line using --monitors, for example --monitors 1:1920x1080,2:default, and the host will change native resolution once the connection has been established.
• Configurable mouse wheel sensitivity. User can specify mouse scroll speed via command line switch serving different needs and requirements.
• Modular architecture and can be extended with additional streaming host services like Nvidia GameStream via Moonlight libraries.
• Screen saver and screen locker support. By default screen saver support is enabled but user can specify command line switch to avoid setting SDL_EnableScreenSaver and disallow X server to lock the local screen.
• Built-in USB redirection with automatic reconnect support using usbredir protocol. User can specify via command line which local USB devices will be redirected to the host (e.g. to forward a webcam with microphone). However usbredir protocol is currently only supported if using KVM, Xen or Virtuozzo Hybrid Server with QEMU. Other virtualization solutions may use standard Linux USB/IP server and native Windows client driver.
• Show render statistics, pipeline stage timings and video stream bandwidth with --stats SECONDS to identify bottlenecks in FFmpeg, SDL, Parsec or event loop.

FFmpeg Decoder

H.265 (HEVC) is enabled by default. The public Linux Parsec SDK exposes a hidden FFmpeg decoder entry, but its Linux implementation does not reliably initialize HEVC. VDI Stream Client therefore installs its own FFmpeg decoder callbacks into that SDK decoder entry at startup and uses it for both H.264 and H.265. The SDK still handles networking, transport and frame delivery, while FFmpeg decodes the encoded video packet and retains the decoded frame behind a ParsecFrame descriptor for the main-thread renderer.

The FFmpeg decoder tries VA-API first when hardware acceleration is enabled. It retains the decoded AV_PIX_FMT_VAAPI frame through the Parsec frame descriptor. Before connecting, the client queries the VA-API profiles and decode entrypoints on the same device FFmpeg will use. This is a metadata-only query and does not encode or decode a test frame. If no H.265 VLD profile is available, the client requests H.264 from the host immediately. H.264 uses VA-API when its VLD profile is available and FFmpeg software decoding otherwise. The renderer maps that frame to DRM PRIME and derives each Vulkan plane format from the VA-API software layout before importing its DMA-BUF objects through libplacebo. This accommodates drivers such as nvidia-vaapi-driver whose exported per-plane DRM fourcc does not match Vulkan’s preferred component mapping. On pre-GFX9 RADV devices without DRM modifier support, the renderer instead imports RadeonSI’s contiguous linear NV12 surface as one multi-planar Vulkan image using Mesa’s macroblock-aligned allocation dimensions through opaque external memory after validating both planes’ pitches, offsets and allocation bounds. If direct DMA-BUF import fails after Vulkan setup, the client transfers the frame to system memory and uploads it through libplacebo, avoiding SDL planar texture creation on the active Vulkan renderer. If Vulkan setup fails, the client recreates the window with the default SDL renderer. If VA-API initialization fails despite an advertised profile, that codec falls back to FFmpeg software decoding. If the complete H.265 startup attempt fails, the client reconnects once with H.265 disabled and uses the selected hardware or software FFmpeg H.264 decoder. Use --video-decoder MODE to select one of these policies. MODE has the form TYPE-CODEC-CHROMA:

• hw or sw selects a hardware or software decoder.
• hevc or h264 selects H.265 / HEVC or H.264 / AVC.
• 444 selects 4:4:4 chroma without subsampling.
• 420 selects 4:2:0 chroma subsampling.

Mode	Behavior
hw-hevc-444 (default)	HW H.265 4:4:4 -> HW H.265 4:2:0 -> HW H.264 4:2:0 -> SW H.264 4:2:0
hw-hevc-420	HW H.265 4:2:0 -> HW H.264 4:2:0 -> SW H.264 4:2:0
hw-h264-420	HW H.264 4:2:0 -> SW H.264 4:2:0
sw-hevc-420	SW H.265 4:2:0
sw-h264-420	SW H.264 4:2:0

Parsec Warp

• Support for disabling chroma subsampling to support color mode 4:4:4 with sharp and crisp text and better colors. Available for Nvidia Hosts running GTX 900 or better.

Known Issues

• Resolution changes from client connected to the host are not persistent and only valid within the session. Once the last client disconnects the host restores original resolution.
• No macOS and Windows support yet. However porting should be fairly easy but I haven’t tested it and pull requests are welcome.

Licensing

VDI Stream Client is licensed under the GNU General Public License version 3 or later. Because this project is intended to be used with the proprietary Parsec SDK, it also includes an additional permission under GPLv3 section 7. The permission is in COPYING.EXCEPTION and allows this program to link with, dynamically load, or otherwise combine with libparsec.so and other Parsec SDK files.

libparsec.so, Parsec SDK headers, and other Parsec SDK files are not part of VDI Stream Client and are not licensed under the GPL by this project. They remain subject to the Parsec SDK license. You may distribute those files only if the applicable Parsec SDK terms allow you to do so.

Building

VDI Stream Client uses GNU Build System to configure, build and install the application. It requires sdl3, sdl3-ttf, libusb, usbredir, libavcodec, libavutil, libavformat, libva, libdrm, libplacebo, Vulkan and the Parsec SDK. The original parsec-cloud/parsec-sdk repository is no longer available after Parsec was acquired by Unity, so this project references the mirrored SDK repository instead. FFmpeg is a mandatory dependency because video decoding is implemented in VDI Stream Client with those libraries. libdrm, libplacebo, Vulkan and libavformat are mandatory build dependencies for VA-API DRM PRIME zero-copy rendering. The build system will search the SDK first in the build directory and use DSO loading for libparsec.so. If not found, it will search in system-wide include and library directories and link against the system libparsec.so.

The GPLv3 additional permission in COPYING.EXCEPTION covers both build modes from the VDI Stream Client side. It does not grant permission to redistribute libparsec.so; check and follow the Parsec SDK license before shipping the Parsec SDK library or headers with source or binary packages.

On Debian or Ubuntu, install the build dependencies with:

sudo apt install build-essential autoconf automake pkg-config \
  libsdl3-dev libsdl3-ttf-dev libusb-1.0-0-dev \
  libusbredirhost-dev libusbredirparser-dev \
  libavcodec-dev libavformat-dev libavutil-dev libva-dev libdrm-dev \
  libplacebo-dev libvulkan-dev

On Arch Linux, install the build dependencies with:

sudo pacman -S base-devel autoconf automake pkgconf \
  sdl3 sdl3_ttf libusb usbredir ffmpeg libva libdrm libplacebo vulkan-headers

For build and install use the commands below and if --prefix=/usr is used, the make install command must be run as root user.

git clone https://github.com/mbroemme/parsec-sdk
./configure --prefix=/usr &&
make &&
make install

Arch Linux users can download ready-to-use PKGBUILD file available from Arch User Repository (AUR), following these build and install instructions.

Usage

VDI Stream Client requires that the Parsec Windows host (x86_64 or x86) is running and you have created a free account. There are several advanced configuration options for visual and network latency improvements described in the Parsec documentation in the “Hosting Settings” chapter. The command syntax is:

vdi-stream-client [OPTION]... --session ID --peer ID

The client requires a session ID and peer ID obtained through the Parsec API to identify users to make secure connections. For convenience you can use tools/parsec-login script to retrieve list of available hosts. This is one-time operation whenever you add Parsec host application to another Windows machine.

USB Redirection

If you are using libvirt for virtualization, you can redirect local USB devices to your Windows host. It is highly recommended to add XHCI controller because it is the only one which supports USB 1.1, USB 2.0 and USB 3.0, the QEMU UHCI and EHCI controllers support only their respective USB standard. Windows 10 is shipped by default with an XHCI USB 3.0 driver. Add the following to your <devices> section in domain XML for an USB 3.0 controller:

<controller type='usb' model='qemu-xhci'/>

With the controller you can start using USB redirection via TCP/IP protocol. Each USB device is redirected through an independent port and the firewall of your server must allow incoming TCP/IP connection for that. VDI Stream Client supports monitoring of hotplug events (e.g. plug and unplug) of the redirected USB devices to allow automatic reassignment to the Windows host. Add the following to your <devices> section in domain XML to enable redirection of one local USB device and replace <ip> with the IP address of your virtualization host:

<redirdev bus='usb' type='tcp'>
  <source mode='bind' host='<ip>' service='4000'/>
</redirdev>

You need to add additional <redirdev> sections to redirect multiple devices and increase the port number accordingly. After reloading libvirt and starting the Virtual Machine with the configuration above, you can redirect a local USB device. You can get a list of available devices with their <VENDOR> and <PRODUCT> with lsusb command. The --redirect SPEC argument uses this format:

VENDOR:PRODUCT@ADDRESS#PORT[,...]

vdi-stream-client --session ID --peer ID \
  --redirect VENDOR:PRODUCT@ADDRESS#4000

Once connection is established it will redirect the local USB device to the host and setup drivers. Multiple USB devices can be redirected using , separator to the --redirect switch. They will be reassigned if the client reconnects to the host. Moreover, devices are reconnected to the host automatically when you unplug and plug them again to the USB port. If the devices are not connected during startup of the client, they are monitored and redirected as soon as they are attached to the client.

Notes

USB redirection integration doesn’t support any kind of encryption. The main goal of this client is to support in-house desktop streaming between client and host. Please use a VPN like WireGuard if you are using it across WAN connections.

Future

Building a truly and fully open-source GPU accelerated desktop streaming solution requires a host service running inside the Windows environment. Hardware encoding is supported by FFmpeg for the major GPU vendors. VDI Stream Host application seems reasonable.

Thanks

A special thanks goes to Parsec Cloud, Inc. as they provided a Parsec Warp subscription to support this project.

Support

If you want to help out with development without providing code yourself, you can always donate to the project directly using the following platforms.

• GitHub
• PayPal
• Bitcoin - bc1qh2pssvhac7629k3ndmmpw3azsm4yjz9jvaaycr
• Litecoin - ltc1qfz93z928vr36d9ksp5mstt385t84mdx4shsv35

Videos