zpool import / no pools / stale zdb labels
Today, when trying to import a newly created ZFS pool, we had to supply the -d DEV argument to find the pool.
# zpool import
no pools available to import
But I know it's there.
# zpool import local-storage
cannot import 'local-storage': no such pool available
And by specifying -d with a device search path, it can be found:
# zpool import local-storage -d /dev/disk/by-id
Success!
# zpool list -oname
NAME
bpool
local-storage
rpool
Manually specifying a search path is not real convenient. It would make the boot process a lot less smooth. We'd have to alter the distribution provided scripts, which in turn makes upgrading more painful.
The culprit — it turned out — was an older zpool that had existed on this device. This caused the zeroth label to be cleared, but the first label to be used:
# zdb -l /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T8TDP_1234
failed to read label 0
------------------------------------
LABEL 1
------------------------------------
version: 5000
name: 'local-storage'
state: 0
...
The easy fix here is to flush the data and start from scratch:
# zfs destroy local-storage
At this point, zdb -l still lists LABEL 1 as used.
# zfs labelclear /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T8TDP_1234
Now the labels are gone:
# zdb -l /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T8TDP_1234
failed to unpack label 0
failed to unpack label 1
failed to unpack label 2
failed to unpack label 3
And after recreating the pool everything works normally:
# zpool create -O compression=lz4 -O mountpoint=/data local-storage \
/dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T8TDP_1234
# zdb -l /dev/disk/by-id/nvme-Micron_9300_MTFDHAL3T8TDP_1234
------------------------------------
LABEL 0
------------------------------------
version: 5000
name: 'local-storage'
state: 0
...
# zpool export local-storage
# zpool import
pool: local-storage
id: 8392074971509924158
state: ONLINE
action: The pool can be imported using its name or numeric identifier.
...
# zpool import local-storage
All good. Imports without having to specify a device.