GPU Passthrough on NixOS

Setting up GPU passthrough on Linux can be tedious at the best of times. NixOS makes some parts of this process easier, but still requires many manual steps as of today. This guide will cover the steps necessary to enable passthrough, create a virtual machine, install Windows, and configure passthrough and Looking Glass for your VM.

Preparation

ISO Downloads

Before doing anything, make sure you have a recent ISO of Windows 10 (or Windows 11, though you may run into some incompatibilities with drivers). You will also need a VirtIO Windows driver ISO.

Bios Settings

To support virtualization, the feature (VT-d, VT-x, or SVM) must be enabled in your Bios. Consult your motherboard manual or explore the menus for information on how to do this.

Next, IOMMU groups must be enabled in the Bios. This information may be difficult to find on your own, typically a web search for how to enable IOMMU on your motherboard brand will help.

Graphics Processor

Ensure that your graphics processor that you want to pass through (either a second graphics card or a graphics card accompanying integrated graphics) is installed in the system. In order to function, you may require a “dummy” connector to be plugged into the graphics output. Here is an example of one such product on Amazon.

In order to specify the specific card to use, you will need to know its IOMMU group and id. To get this information you can either run nix run github:jakehamilton/config#list-iommu or create and run the following script yourself:

#! /usr/bin/env nix-shell
#! nix-shell -i bash -p pciutils

shopt -s nullglob
for d in /sys/kernel/iommu_groups/*/devices/*; do 
    n=${d#*/iommu_groups/*}; n=${n%%/*}
    printf 'IOMMU Group %s ' "$n"
    lspci -nns "${d##*/}"
done;

Note the entries for the graphics card that you would like to pass through. For example, an AMD RX480 may appear with the following entries:

IOMMU Group 23 23:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev c7)
IOMMU Group 23 23:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]

The important information from these entries are the groups (23.00.0, 23.00.1) and the ids (1002:67df, 1002:aaf0). Keep these on hand later to be used when working with the virtual machine configuration.

NixOS Configuration

Before creating a virtual machine, changes must be made to the system configuration. These changes will vary depending on the device you are using, but should typically require only specifying either amd or intel variants for kernel modules and options. Append or import the following configuration into your own.

{ pkgs, config, ... }:
let
  # Change this to your username.
  user = "my-user";
  # Change this to match your system's CPU.
  platform = "amd";
  # Change this to specify the IOMMU ids you wrote down earlier.
  vfioIds = [ "1002:67df" "1002:aaf0" ];
in {
  # Configure kernel options to make sure IOMMU & KVM support is on.
  boot = {
    kernelModules = [ "kvm-${platform}" "vfio_virqfd" "vfio_pci" "vfio_iommu_type1" "vfio" ];
    kernelParams = [ "${platform}_iommu=on" "${platform}_iommu=pt" "kvm.ignore_msrs=1" ];
    extraModprobeConfig = "options vfio-pci ids=${builtins.concatStringsSep "," vfioIds}";
  };

  # Add a file for looking-glass to use later. This will allow for viewing the guest VM's screen in a
  # performant way.
  systemd.tmpfiles.rules = [
      "f /dev/shm/looking-glass 0660 ${user} qemu-libvirtd -"
  ];

  # Add virt-manager and looking-glass to use later.
  environment.systemPackages = with pkgs; [
      virt-manager
      looking-glass-client
  ];

  # Enable virtualisation programs. These will be used by virt-manager to run your VM.
  virtualisation = {
     libvirtd = {
       enable = true;
       extraConfig = ''
         user="${user}"
       '';

       # Don't start any VMs automatically on boot.
       onBoot = "ignore";
       # Stop all running VMs on shutdown.
       onShutdown = "shutdown";

       qemu = {
         package = pkgs.qemu_kvm;
         ovmf = enabled;
         verbatimConfig = ''
            namespaces = []
           user = "+${builtins.toString config.users.users.${user}.uid}"
         '';
       };
    };
  };

  users.users.${user}.extraGroups = [ "qemu-libvirtd" "libvirtd" "disk" ];
}

Once updated, run sudo nixos-rebuild switch on your configuration and reboot.

VM Creation

Start by opening the virt-manager program. Once open, make sure that it has connected to the default KVM target (qemu:///system). This should be the default and require no extra work.

Now, create a new virtual machine, selecting the Windows ISO that you've downloaded. You may configure your system's RAM and CPU resources, but leave the disk configuration for later. We will be using a specific disk setup with VirtIO support. Before finishing VM creation, select “Configure the machine before installing.”

Within the VM Configuration window, add the following:

OS Installation

You may now boot the VM and begin installing Windows. When prompted to select a drive, you may not see any available if you're using a VirtIO disk. When this happens, select “Load Driver” and select the VirtIO driver for your operating system version.

Windows 11 Installation

Windows 11 does not support systems without certain features like a TPM and Secure Boot. Currently, these features can be avoided by performing the following actions.

Now you may proceed with installation.

Drivers

SPICE

Install the SPICE drivers for your Windows system.

VirtIO

Install the VirtIO drivers if you haven't already. Note that these drivers do not currently support Windows 11.

Looking Glass

In order to install Looking Glass, you must download the matching host version for the client version that you will be using on your NixOS machine. To find your client version, run the following command:

nix-instantiate --eval -E "(import <nixpkgs> {}).looking-glass-client.version"

Download and install the appropriate host version of Looking Glass for your Windows system, taking care to get the one that matches the version you just checked.

Once installed you will need to configure the program to run on startup. This can be done from an administrator powershell with the following commands:

# Done manually
SCHTASKS /Create /TN "Looking Glass" /SC  ONLOGON /RL HIGHEST /TR 'C:\Program Files\Looking Glass (host)\looking-glass-host.exe'

# Installed as a service
C:\Program Files\Looking Glass (host)\looking-glass-host.exe InstallService

Passthrough

With the OS, drivers, and services installed on your VM, you can begin configuring virt-manager to use full device passthrough. To do so, shut down the VM and open its configuration in virt-manager.

Add the following entries to the end of the <devices> section in your system's XML.

<shmem name="looking-glass">
  <model type="ivshmem-plain"/>
  <size unit="M">32</size>
  <address type="pci" domain="0x0000" bus="0x0b" slot="0x01" function="0x0"/>
</shmem>

Change the <video> model to none in the <devices> section in your system's XML.

<video>
  <model type="none"/>
</video>

Remove <input type="tablet"> from <devices> in your system's XML.

- <input type="tablet" bus="usb">
-   <address type="usb" bus="0" port="1"/>
- </input>

You may now boot the VM and open Looking Glass. If Looking Glass does not work, I recommend connecting a display to the graphics card that you are passing through and using the output there to diagnose the issue. Sometimes Looking Glass can take a while to start or Windows will refuse to start it. A reboot of the VM may fix the issue.

Troubleshooting

Installation may be done, but the trouble isn't over. Thanks Microsoft!

Booting ISO results in PAGE_FAULT_IN_NONPAGED_AREA

This issue was resolved by correcting permissions on the ISO file. For some reason it was read-only when it should have been read-write. It was also owned inexplicably by the root group. Running sudo chown my-user:users my-windows-installer.iso and chmod g+w my-windows-installer.iso corrected the problem.

Windows fails to boot

Sometimes Windows will break itself...

No drive found

Windows isn't able to find the drive for some reason. We must boot into the installation ISO, select Repair this PC, and then choose Troubleshoot > Command Line.

Run the following command to list the disks currently available.

fsutil fsinfo drives

Find the drive with the VirtIO drivers (likely E:) and run the following command to install the driver for a 64-bit system.

# For Windows 10 drivers on the E: drive
drvload E:\amd64\w10\viostor.inf

With the VirtIO driver loaded, exit out of the terminal and select Troubleshoot > Startup Repair. Choose the Windows installation and let the machine reboot. Repeat the process of loading drivers again once the windows prompt comes up, then select Continue to Windows this time.

In the OS, run the following commands.

# Scan the drive & fix if possible
sfc /scannow

# Scan the drive & fix (more thoroughly)
dism /online /cleanup-image /restorehealth

# Scan & fix one last time...
sfc /scannow

Reboot the VM.

Windows 11 fails to install

Make sure that you've followed the steps in the Installation section of this guide to disable installation checks. If those no longer work then you may be out of luck.

Resizing <shmem>

In order to change the size of shared memory for Looking Glass, first shut down the virtual machine. Then remove the existing device with the command sudo rm /dev/shm/looking-glass. Then modify the XML for the virtual machine and start it.