Bug 221066 - kernel panic with vdev_geom_close_locked: cp->private is NULL
Summary: kernel panic with vdev_geom_close_locked: cp->private is NULL
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: Alan Somers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-07-28 09:49 UTC by David NewHamlet
Modified: 2018-03-05 20:38 UTC (History)
3 users (show)

See Also:
asomers: mfc-stable11+
asomers: mfc-stable10+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description David NewHamlet 2017-07-28 09:49:47 UTC
when using a zpool in a readonly device as root filesystem, kernel panic after failed to open the device for write. 

This comes from a recent changeset(https://svnweb.freebsd.org/base?view=revision&revision=318189). 


error=30 means device is readonly.

md0: Embedded image 13798912 bytes at 0xffff000000909538

ZFS WARNING: Unable to open /dev/md0.uzip for writing (error=30).
panic: vdev_geom_close_locked: cp->private is NULL

----- serial console text from raspberry pi 3 ----
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
Timecounters tick every 1.000 msec
md0: Embedded image 13798912 bytes at 0xffff000000909538
usbus0: 480Mbps High Speed USB v2.0
ugen0.1: <DWCOTG OTG Root HUB> at usbus0
uhub0: <DWCOTG OTG Root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
mmc0: No compatible cards found on bus
uhub0: 1 port with 1 removable, self powered
ugen0.2: <vendor 0x0424 product 0x9514> at usbus0
uhub1 on uhub0
uhub1: <vendor 0x0424 product 0x9514, class 9/0, rev 2.00/2.00, addr 2> on usbus0
uhub1: MTT enabled
uhub1: 5 ports with 4 removable, self powered
ugen0.3: <vendor 0x0424 product 0xec00> at usbus0
smsc0 on uhub1
smsc0: <vendor 0x0424 product 0xec00, rev 2.00/2.00, addr 3> on usbus0
smsc0: chip 0xec00, rev. 0002
miibus0: <MII bus> on smsc0
smscphy0: <SMC LAN8700 10/100 interface> PHY 1 on miibus0
smscphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
ue0: <USB Ethernet> on smsc0
ue0: Ethernet address: 02:00:00:00:00:00
ugen0.4: <ADATA ADATA USB Flash Drive> at usbus0
umass0 on uhub1
umass0: <ADATA ADATA USB Flash Drive, class 0/0, rev 2.10/11.00, addr 4> on usbus0
umass0:  SCSI over Bulk-Only; quirks = 0x8100
umass0:0:0: Attached to scbus0
Release APs
da0 at umass-sim0 bus 0 scbus0 target 0 lun 0
da0: <ADATA USB Flash Drive 1100> Removable Direct Access SPC-4 SCSI device
da0: Serial Number 1472516211270070
da0: 40.000MB/s transfers
da0: 29600MB (60620800 512 byte sectors)
da0: quirks=0x2<NO_6_BYTE>
CPU  0: ARM Cortex-A53 r0p4 affinity:  0
 Instruction Set Attributes 0 = <CRC32>
 Instruction Set Attributes 1 = <0>
         Processor Features 0 = <AdvSIMD,Float,EL3 32,EL2 32,EL1 32,EL0 32>
         Processor Features 1 = <0>
      Memory Model Features 0 = <4k Granule,64k Granule,MixedEndian,S/NS Mem,16bit ASID,1TB PA>
      Memory Model Features 1 = <>
             Debug Features 0 = <2 CTX Breakpoints,4 Watchpoints,6 Breakpoints,PMUv3,Debug v8>
             Debug Features 1 = <0>
         Auxiliary Features 0 = <0>
         Auxiliary Features 1 = <0>
CPU  1: ARM Cortex-A53 r0p4 affinity:  1
CPU  2: ARM Cortex-A53 r0p4 affinity:  2
CPU  3: ARM Cortex-A53 r0p4 affinity:  3
WARNING: WITNESS option enabled, expect reduced performance.
random: unblocking device.
arc4random: no preloaded entropy cache
Trying to mount root from zfs:initmdz [rw]...
arc4random: no preloaded entropy cache
arc4random: no preloaded entropy cache
ZFS WARNING: Unable to open /dev/md0.uzip for writing (error=30).
panic: vdev_geom_close_locked: cp->private is NULL
cpuid = 1
time = 14
KDB: stack backtrace:
db_trace_self() at db_trace_self_wrapper+0x28
	 pc = 0xffff00000073e6b0  lr = 0xffff00000018b2ec
	 sp = 0xffff000078bbd960  fp = 0xffff000078bbdb70

db_trace_self_wrapper() at vpanic+0x184
	 pc = 0xffff00000018b2ec  lr = 0xffff00000043114c
	 sp = 0xffff000078bbdb80  fp = 0xffff000078bbdc00

vpanic() at kassert_panic+0x158
	 pc = 0xffff00000043114c  lr = 0xffff000000430fc4
	 sp = 0xffff000078bbdc10  fp = 0xffff000078bbdcd0

kassert_panic() at vdev_geom_close_locked+0x174
	 pc = 0xffff000000430fc4  lr = 0xffff0000000e1774
	 sp = 0xffff000078bbdce0  fp = 0xffff000078bbdd10

vdev_geom_close_locked() at vdev_geom_open+0x6c0
	 pc = 0xffff0000000e1774  lr = 0xffff0000000e0b48
	 sp = 0xffff000078bbdd20  fp = 0xffff000078bbdda0

vdev_geom_open() at vdev_open+0x1bc
	 pc = 0xffff0000000e0b48  lr = 0xffff0000000d92fc
	 sp = 0xffff000078bbddb0  fp = 0xffff000078bbddf0

vdev_open() at vdev_open_children+0x34
	 pc = 0xffff0000000d92fc  lr = 0xffff0000000d9114
	 sp = 0xffff000078bbde00  fp = 0xffff000078bbde20

vdev_open_children() at vdev_root_open+0x38
	 pc = 0xffff0000000d9114  lr = 0xffff0000000ea1d8
	 sp = 0xffff000078bbde30  fp = 0xffff000078bbde70

vdev_root_open() at vdev_open+0x1bc
	 pc = 0xffff0000000ea1d8  lr = 0xffff0000000d92fc
	 sp = 0xffff000078bbde80  fp = 0xffff000078bbdec0

vdev_open() at spa_load_impl+0x2c0
	 pc = 0xffff0000000d92fc  lr = 0xffff0000000c9d48
	 sp = 0xffff000078bbded0  fp = 0xffff000078bbdfa0

spa_load_impl() at spa_load+0x1d8
	 pc = 0xffff0000000c9d48  lr = 0xffff0000000c493c
	 sp = 0xffff000078bbdfb0  fp = 0xffff000078bbe000

spa_load() at spa_load_impl+0xc78
	 pc = 0xffff0000000c493c  lr = 0xffff0000000ca700
	 sp = 0xffff000078bbe010  fp = 0xffff000078bbe0e0

spa_load_impl() at spa_load+0x1d8
	 pc = 0xffff0000000ca700  lr = 0xffff0000000c493c
	 sp = 0xffff000078bbe0f0  fp = 0xffff000078bbe140

spa_load() at spa_load_best+0x78
	 pc = 0xffff0000000c493c  lr = 0xffff0000000c41e8
	 sp = 0xffff000078bbe150  fp = 0xffff000078bbe1b0

spa_load_best() at spa_open_common+0xf4
	 pc = 0xffff0000000c41e8  lr = 0xffff0000000c0818
	 sp = 0xffff000078bbe1c0  fp = 0xffff000078bbe220

spa_open_common() at dsl_pool_hold+0x20
	 pc = 0xffff0000000c0818  lr = 0xffff0000000a8ac8
	 sp = 0xffff000078bbe230  fp = 0xffff000078bbe250

dsl_pool_hold() at dmu_objset_own+0x38
	 pc = 0xffff0000000a8ac8  lr = 0xffff00000007eff4
	 sp = 0xffff000078bbe260  fp = 0xffff000078bbe2a0

dmu_objset_own() at zfsvfs_create+0x8c
	 pc = 0xffff00000007eff4  lr = 0xffff00000011301c
	 sp = 0xffff000078bbe2b0  fp = 0xffff000078bbe300

zfsvfs_create() at zfs_mount+0x314
	 pc = 0xffff00000011301c  lr = 0xffff000000111790
	 sp = 0xffff000078bbe310  fp = 0xffff000078bbe4d0

zfs_mount() at vfs_donmount+0xd0c
	 pc = 0xffff000000111790  lr = 0xffff0000004e597c
	 sp = 0xffff000078bbe4e0  fp = 0xffff000078bbe720

vfs_donmount() at kernel_mount+0x58
	 pc = 0xffff0000004e597c  lr = 0xffff0000004e863c
	 sp = 0xffff000078bbe730  fp = 0xffff000078bbe780

kernel_mount() at parse_mount+0x618
	 pc = 0xffff0000004e863c  lr = 0xffff0000004ead28
	 sp = 0xffff000078bbe790  fp = 0xffff000078bbe8e0

parse_mount() at vfs_mountroot+0x508
	 pc = 0xffff0000004ead28  lr = 0xffff0000004e8f90
	 sp = 0xffff000078bbe8f0  fp = 0xffff000078bbeac0

vfs_mountroot() at start_init+0x64
	 pc = 0xffff0000004e8f90  lr = 0xffff0000003d0cf8
	 sp = 0xffff000078bbead0  fp = 0xffff000078bbeb50

start_init() at fork_exit+0x7c
	 pc = 0xffff0000003d0cf8  lr = 0xffff0000003f6388
	 sp = 0xffff000078bbeb60  fp = 0xffff000078bbeb90

fork_exit() at fork_trampoline+0x10
	 pc = 0xffff0000003f6388  lr = 0xffff000000756b64
	 sp = 0xffff000078bbeba0  fp = 0x0000000000000000

KDB: enter: panic
[ thread pid 1 tid 100002 ]
Stopped at      kdb_enter+0x40: undefined       d4200000
db>
Comment 1 Alan Somers freebsd_committer freebsd_triage 2017-07-28 15:31:52 UTC
Can you please post detailed reproduction steps?
Comment 2 David NewHamlet 2017-07-28 21:10:54 UTC
(In reply to Alan Somers from comment #1)


I hit this issue when trying to use a zfs as rootfs which use a compressed md_image as underlayer device. A compressed md_image can not be opened in write mode.

Use a zfs as rootfs(vfs.root.mountfrom) in compressed md_image or a memory stick/Sd card which have a hard switch to turn device into readoly mode will reproduce this panic.
Comment 3 Alan Somers freebsd_committer freebsd_triage 2017-11-27 20:58:50 UTC
Here's a minimal reproduction case:

$ sudo gnop create -S 16384 /dev/da0 
$ sudo zpool create foo /dev/da0.nop
Comment 4 commit-hook freebsd_committer freebsd_triage 2017-11-30 15:29:34 UTC
A commit references this bug:

Author: asomers
Date: Thu Nov 30 15:28:29 UTC 2017
New revision: 326399
URL: https://svnweb.freebsd.org/changeset/base/326399

Log:
  Fix assertion when ZFS fails to open certain devices

  "panic: vdev_geom_close_locked: cp->private is NULL"
  This panic will result if ZFS fails to open a device due to either of the
  following reasons:

  1) The device's sector size is greater than 8KB.
  2) ZFS wants to open the device RW, but it can't be opened for writing.

  The solution is to change the initialization order to ensure that the
  assertion will be satisfied.

  PR:		221066
  Reported by:	David NewHamlet <wheelcomplex@gmail.com>
  Reviewed by:	avg
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp
  Differential Revision:	https://reviews.freebsd.org/D13278

Changes:
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c
Comment 5 commit-hook freebsd_committer freebsd_triage 2017-11-30 15:36:42 UTC
A commit references this bug:

Author: asomers
Date: Thu Nov 30 15:36:06 UTC 2017
New revision: 326401
URL: https://svnweb.freebsd.org/changeset/base/326401

Log:
  Fix assertion when ZFS fails to open certain devices

  "panic: vdev_geom_close_locked: cp->private is NULL"
  This panic will result if ZFS fails to open a device due to either of the
  following reasons:

  1) The device's sector size is greater than 8KB.
  2) ZFS wants to open the device RW, but it can't be opened for writing.

  The solution is to change the initialization order to ensure that the
  assertion will be satisfied.

  PR:		221066
  Reported by:	David NewHamlet <wheelcomplex@gmail.com>
  Reviewed by:	avg
  MFC after:	3 weeks
  Sponsored by:	Spectra Logic Corp
  Differential Revision:	https://reviews.freebsd.org/D13278

Changes:
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c
Comment 6 commit-hook freebsd_committer freebsd_triage 2017-12-26 04:02:43 UTC
A commit references this bug:

Author: asomers
Date: Tue Dec 26 04:02:35 UTC 2017
New revision: 327188
URL: https://svnweb.freebsd.org/changeset/base/327188

Log:
  MFC r326401:

  Fix assertion when ZFS fails to open certain devices

  "panic: vdev_geom_close_locked: cp->private is NULL"
  This panic will result if ZFS fails to open a device due to either of the
  following reasons:

  1) The device's sector size is greater than 8KB.
  2) ZFS wants to open the device RW, but it can't be opened for writing.

  The solution is to change the initialization order to ensure that the
  assertion will be satisfied.

  PR:		221066
  Reported by:	David NewHamlet <wheelcomplex@gmail.com>
  Reviewed by:	avg
  Sponsored by:	Spectra Logic Corp
  Differential Revision:	https://reviews.freebsd.org/D13278

Changes:
_U  stable/11/
  stable/11/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c
Comment 7 commit-hook freebsd_committer freebsd_triage 2018-02-22 01:26:16 UTC
A commit references this bug:

Author: asomers
Date: Thu Feb 22 01:25:28 UTC 2018
New revision: 329780
URL: https://svnweb.freebsd.org/changeset/base/329780

Log:
  MFC r326399:

  Fix assertion when ZFS fails to open certain devices

  "panic: vdev_geom_close_locked: cp->private is NULL"
  This panic will result if ZFS fails to open a device due to either of the
  following reasons:

  1) The device's sector size is greater than 8KB.
  2) ZFS wants to open the device RW, but it can't be opened for writing.

  The solution is to change the initialization order to ensure that the
  assertion will be satisfied.

  PR:		221066
  Reported by:	David NewHamlet <wheelcomplex@gmail.com>
  Reviewed by:	avg
  Sponsored by:	Spectra Logic Corp
  Differential Revision:	https://reviews.freebsd.org/D13278

Changes:
_U  stable/11/
  stable/11/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_raidz.c
Comment 8 commit-hook freebsd_committer freebsd_triage 2018-03-05 20:29:31 UTC
A commit references this bug:

Author: asomers
Date: Mon Mar  5 20:28:49 UTC 2018
New revision: 330522
URL: https://svnweb.freebsd.org/changeset/base/330522

Log:
  MFC r326401:

  Fix assertion when ZFS fails to open certain devices

  "panic: vdev_geom_close_locked: cp->private is NULL"
  This panic will result if ZFS fails to open a device due to either of the
  following reasons:

  1) The device's sector size is greater than 8KB.
  2) ZFS wants to open the device RW, but it can't be opened for writing.

  The solution is to change the initialization order to ensure that the
  assertion will be satisfied.

  PR:		221066
  Reported by:	David NewHamlet <wheelcomplex@gmail.com>
  Reviewed by:	avg
  Sponsored by:	Spectra Logic Corp
  Differential Revision:	https://reviews.freebsd.org/D13278

Changes:
_U  stable/10/
  stable/10/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_geom.c