Bug 268116 - ZFS: panic (VERIFY0(0 == nvlist_lookup_uint64(nvl, name, &rv)) failed (0 == 22)) – reboot after readding ZIL device
Summary: ZFS: panic (VERIFY0(0 == nvlist_lookup_uint64(nvl, name, &rv)) failed (0 == 2...
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 13.1-RELEASE
Hardware: amd64 Any
: --- Affects Only Me
Assignee: freebsd-fs (Nobody)
URL:
Keywords: crash, needs-qa
Depends on:
Blocks:
 
Reported: 2022-12-02 06:29 UTC by Cristian Miu
Modified: 2022-12-15 19:59 UTC (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Cristian Miu 2022-12-02 06:29:10 UTC
Hi, i'm facing a weird problem. I tested this on multiple machines with multiple ZFS array (zfs stripe, mirror, raidz1).
This problem appears only on FreeBSD 13.1-RELEASE, 13.1-STABLE and 14-CURRENT.

So, here is the scenario:

1. I create a new log and cache device using the following commands:
gpart create -s gpt nvd0
gpart add -a 4k -s 50G -t freebsd-zfs -l log nvd0
gpart add -a 4k -s 50G -t freebsd-zfs -l cache nvd0
zpool add zroot log /dev/gpt/log
zpool add zroot cache /dev/gpt/cache 

2. I reboot the machine, everything works as it should, ZFS devices are displayed correctly. 

3. I realise that i want to allocate less space for log and more for cache for example. I remove the log and cache from the pool with the following commands:

zpool remove zroot /dev/gpt/cache
zpool remove zroot /dev/gpt/log

4. I reboot the machine, to ensure that everything is ok. The machine is booting correctly.

5. I destroy the array and recreate it, then added to the pool using the following commands:

gpart destroy -F /dev/nvd0
gpart create -s gpt /dev/nvd0
gpart add -a 4k -s 16G -t freebsd-zfs -l log nvd0
gpart add -a 4k -s 84G -t freebsd-zfs -l cache nvd0
zpool add zroot log /dev/gpt/log
zpool add zroot cache /dev/gpt/cache 

6. Everything is ok, is working for the moment. 
7. I reboot the system, the bootloader is loading, and the system start to boot and when trying to mount the log device the system reboots automatically. 
8. I tried to remove only the cache device from the pool using the command

zpool remove zroot /dev/gpt/cache

9. I reboot the system, same result, the system reboots.

10. Getting back to usb drive, import the zfs, remove the log, add the cache. The system booting normally using only the cache device.
11. I tried using different ssd, separate log and cache to different psychical  devices, same result.

12. I made a fresh install of freebsd 14-CURRENT and reproduce all the steps above... the system reboots exactly the same only with log device attached.

Here is the kernel panic message i got:

VERIFY0(0 == nvlist_lookup_uint64(nvl, name, &rv)) failed (0 == 22)


Is there a workaround to fix this?

Thank you!
Comment 1 Graham Perrin freebsd_committer freebsd_triage 2022-12-04 16:39:40 UTC
Can you provide more information about the crash? 

Maybe, the most recent core.txt.⋯ file from /var/crash (if there's no such file, please see savecore(8) and related pages). 

<https://www.freebsd.org/cgi/man.cgi?query=savecore&sektion=8&manpath=FreeBSD>

Thanks
Comment 2 titus m 2022-12-06 09:25:24 UTC
seems to be the same/close to as https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256368
patch, etc at
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256368#c4