proxmox / virtio-blk / disk by-id

proxmox / virtio-blk / disk by-id

  • Written by Walter Doekes

  • Published on: 2021-04-08

Why does the virtio-blk /dev/vda block device not show up in /dev/disk/by-id?

Yesterday, I wrote about how Proxmox VE attaches scsi0 and virtio0 block devices differently. That is the starting point for todays question: how come do I get /dev/sda in /dev/disk/by-id while /dev/vda is nowhere to be found?

This question is relevant if you're used to referencing disks through /dev/disk/by-id (for example when setting up ZFS, using the device identifiers). The named devices can be a lot more convenient to keep track of.

If you're on a QEMU VM using virtio-scsi, the block devices do show up:

# ls -log /dev/disk/by-id/
total 0
lrwxrwxrwx 1  9 apr  8 14:50 ata-QEMU_DVD-ROM_QM00003 -> ../../sr0
lrwxrwxrwx 1  9 apr  8 14:50 scsi-0QEMU_QEMU_HARDDISK_drive-scsi0 -> ../../sda
lrwxrwxrwx 1 10 apr  8 14:50 scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-part1 -> ../../sda1

But if you're using virtio-blk, they do not:

# ls -log /dev/disk/by-id/
total 0
lrwxrwxrwx 1 9 apr  8 14:50 ata-QEMU_DVD-ROM_QM00003 -> ../../sr0

There: no symlinks to /dev/vda, while it does exist and it does show up in /dev/disk/by-path:

# ls -l /dev/vda{,1}
brw-rw---- 1 root disk 254, 0 apr  8 14:50 /dev/vda
brw-rw---- 1 root disk 254, 1 apr  8 14:50 /dev/vda1
# ls -log /dev/disk/by-path/
total 0
lrwxrwxrwx 1  9 apr  8 14:50 pci-0000:00:01.1-ata-2 -> ../../sr0
lrwxrwxrwx 1  9 apr  8 14:50 pci-0000:00:0a.0 -> ../../vda
lrwxrwxrwx 1 10 apr  8 14:50 pci-0000:00:0a.0-part1 -> ../../vda1
lrwxrwxrwx 1  9 apr  8 14:50 virtio-pci-0000:00:0a.0 -> ../../vda
lrwxrwxrwx 1 10 apr  8 14:50 virtio-pci-0000:00:0a.0-part1 -> ../../vda1

udev rules

Who creates these? It's udev.

If you look at the udev rules in 60-persistent-storage.rules, you'll see a bunch of these:

# grep -E '"(sd|vd)' /lib/udev/rules.d/60-persistent-storage.rules
KERNEL=="vd*[!0-9]", ATTRS{serial}=="?*", ENV{ID_SERIAL}="$attr{serial}", SYMLINK+="disk/by-id/virtio-$env{ID_SERIAL}"
KERNEL=="vd*[0-9]", ATTRS{serial}=="?*", ENV{ID_SERIAL}="$attr{serial}", SYMLINK+="disk/by-id/virtio-$env{ID_SERIAL}-part%n"
...
KERNEL=="sd*[!0-9]|sr*", ENV{ID_SERIAL}!="?*", SUBSYSTEMS=="scsi", ATTRS{vendor}=="ATA", IMPORT{program}="ata_id --export $devnode"
KERNEL=="sd*[!0-9]|sr*", ENV{ID_SERIAL}!="?*", SUBSYSTEMS=="scsi", ATTRS{type}=="5", ATTRS{scsi_level}=="[6-9]*", IMPORT{program}="ata_id --export $devnode"
...
KERNEL=="sd*|sr*|cciss*", ENV{DEVTYPE}=="disk", ENV{ID_SERIAL}=="?*", SYMLINK+="disk/by-id/$env{ID_BUS}-$env{ID_SERIAL}"
KERNEL=="sd*|cciss*", ENV{DEVTYPE}=="partition", ENV{ID_SERIAL}=="?*", SYMLINK+="disk/by-id/$env{ID_BUS}-$env{ID_SERIAL}-part%n"
...

So, udev is in the loop, and would create symlinks, if it matched the appropriate rules.

Comparing output from udevadm:

# udevadm info /dev/sda
P: /devices/pci0000:00/0000:00:05.0/virtio1/host2/target2:0:0/2:0:0:0/block/sda
N: sda
...
E: DEVNAME=/dev/sda
E: DEVTYPE=disk
...
E: ID_SERIAL=0QEMU_QEMU_HARDDISK_drive-scsi0
E: ID_SERIAL_SHORT=drive-scsi0
E: ID_BUS=scsi
E: ID_PATH=pci-0000:00:05.0-scsi-0:0:0:0
...

and:

# udevadm info /dev/vda
P: /devices/pci0000:00/0000:00:0a.0/virtio1/block/vda
N: vda
...
E: DEVNAME=/dev/vda
E: DEVTYPE=disk
...
E: ID_PATH=pci-0000:00:0a.0
...

The output from /dev/vda is a lot shorter. And there is no ID_BUS nor ID_SERIAL. And the lack of a serial is what causes this rule to be skipped:

KERNEL=="vd*[!0-9]", ATTRS{serial}=="?*", ENV{ID_SERIAL}="$attr{serial}", SYMLINK+="disk/by-id/virtio-$env{ID_SERIAL}"

We could hack the udev rules, adding a default serial when it's unavailable:

KERNEL=="vd*[!0-9]", ATTRS{serial}!="?*", ENV{ID_SERIAL}="MY_SERIAL", SYMLINK+="disk/by-id/virtio-$env{ID_SERIAL}"
# udevadm control --reload
# udevadm trigger --action=change
# ls -log /dev/disk/by-id/
lrwxrwxrwx 1 9 apr  8 14:50 ata-QEMU_DVD-ROM_QM00003 -> ../../sr0
lrwxrwxrwx 1 9 apr  8 14:50 virtio-MY_SERIAL -> ../../vda

But that's awkward. And it breaks things if we ever add a second disk.

Adding a serial through Proxmox

Instead, we can hand-hack the Proxmox VE QEMU configuration file and add a (custom 20 bytes) ,serial=MY_SERIAL parameter to the disk configuration. We'll use disk0 as serial for now:

--- /etc/pve/qemu-server/NNN.conf
+++ /etc/pve/qemu-server/NNN.conf
@@ -10,5 +10,5 @@ ostype: l26
 scsihw: virtio-scsi-pci
 smbios1: uuid=d41e78ad-4ff6-4000-8882-c343e3233945
 sockets: 1
-virtio0: somedisk:vm-NNN-disk-0,size=32G
+virtio0: somedisk:vm-NNN-disk-0,serial=disk0,size=32G
 vmgenid: 2ffdfa16-769a-421f-91f3-71397562c6b9

Stop the VM, start it again, and voilà, the disk is matched:

# ls -log /dev/disk/by-id/
total 0
lrwxrwxrwx 1  9 apr  8 14:50 ata-QEMU_DVD-ROM_QM00003 -> ../../sr0
lrwxrwxrwx 1  9 apr  8 14:50 virtio-disk0 -> ../../vda
lrwxrwxrwx 1 10 apr  8 14:50 virtio-disk0-part1 -> ../../vda1

As long as you don't create duplicate serials in the same VM, this should be fine.


Back to overview Older post: proxmox / alter default create vm parameters