Bug 225877 - panic: solaris assert: newds == os->os_dsl_dataset during "zpool upgrade"
Summary: panic: solaris assert: newds == os->os_dsl_dataset during "zpool upgrade"
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: Andriy Gapon
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-02-13 22:39 UTC by Alan Somers
Modified: 2018-03-27 11:41 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alan Somers freebsd_committer 2018-02-13 22:39:47 UTC
The following panic is generated by zpool_upgrade_007_pos from the ZFS test suite:

panic: solaris assert: newds == os->os_dsl_dataset (0xfffff80045f4c000 == 0xfffff80021ab4800), file: /usr/home/alans/freebsd/head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c, line: 677
cpuid = 10
time = 1518552986
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00c24953a0
vpanic() at vpanic+0x18d/frame 0xfffffe00c2495400
panic() at panic+0x43/frame 0xfffffe00c2495460
assfail3() at assfail3+0x2c/frame 0xfffffe00c2495480
dmu_objset_refresh_ownership() at dmu_objset_refresh_ownership+0x161/frame 0xfffffe00c24955c0
zfs_ioc_userspace_upgrade() at zfs_ioc_userspace_upgrade+0x97/frame 0xfffffe00c2495600
zfs_prop_set_special() at zfs_prop_set_special+0x465/frame 0xfffffe00c2495670
zfs_set_prop_nvlist() at zfs_set_prop_nvlist+0x23f/frame 0xfffffe00c24956f0
zfs_ioc_set_prop() at zfs_ioc_set_prop+0x129/frame 0xfffffe00c2495740
zfsdev_ioctl() at zfsdev_ioctl+0x76b/frame 0xfffffe00c24957e0
devfs_ioctl() at devfs_ioctl+0xcb/frame 0xfffffe00c2495830
VOP_IOCTL_APV() at VOP_IOCTL_APV+0xd9/frame 0xfffffe00c2495860
vn_ioctl() at vn_ioctl+0x124/frame 0xfffffe00c2495970
devfs_ioctl_f() at devfs_ioctl_f+0x1f/frame 0xfffffe00c2495990
kern_ioctl() at kern_ioctl+0x2b9/frame 0xfffffe00c24959f0
sys_ioctl() at sys_ioctl+0x15c/frame 0xfffffe00c2495ac0
amd64_syscall() at amd64_syscall+0x79b/frame 0xfffffe00c2495bf0
fast_syscall_common() at fast_syscall_common+0xfc/frame 0x7fffffff8880

I saw this on 12.0-CURRENT amd64, at 328401.  Steps to reproduce:
$ svn co https://svn.freebsd.org/base/projects/zfsd/head
$ cd head
$ mtree -deU -f etc/mtree/BSD.tests.dist
$ cd tests/sys/cddl
$ make && sudo make install
$ # Now write kyua.conf.  Modify as appropriate for your system
$ echo >> /usr/local/etc/kyua/kyua.conf <HERE
-- $FreeBSD$
--
-- System-wide configuration file for kyua(1).  See kyua.conf(5) for details
-- on the syntax.
--

syntax(2)

-- User to drop privileges to when invoking kyua(1) as root and a test case
-- requests to be run with non-root permissions.
unprivileged_user = 'tests'

-- An example to set a configuration property specific to FreeBSD.
test_suites.FreeBSD.disks="/dev/da1 /dev/da2 /dev/da3 /dev/da4 /dev/da5 /dev/da6"
test_suites.FreeBSD.keep_pools="zroot tank"

HERE
$ cd /usr/tests/sys/cddl/zfs/tests/cli_root/zpool_upgrade
$ sudo kyua debug zpool_upgrade_test:zpool_upgrade_007_pos
Comment 1 commit-hook freebsd_committer 2018-02-13 22:43:00 UTC
A commit references this bug:

Author: asomers
Date: Tue Feb 13 22:42:25 UTC 2018
New revision: 329248
URL: https://svnweb.freebsd.org/changeset/base/329248

Log:
  Skip zpool_upgrade_007_pos

  This test frequently panics.  Skip it until the panic is fixed

  PR:		225877
  Sponsored by:	Spectra Logic Corp

Changes:
  projects/zfsd/head/tests/sys/cddl/zfs/tests/cli_root/zpool_upgrade/zpool_upgrade_test.sh
Comment 2 Alan Somers freebsd_committer 2018-03-10 01:54:55 UTC
Reassign back to the pool.  I'm no longer working on ZFS.
Comment 3 commit-hook freebsd_committer 2018-03-15 08:49:45 UTC
A commit references this bug:

Author: avg
Date: Thu Mar 15 08:49:22 UTC 2018
New revision: 330974
URL: https://svnweb.freebsd.org/changeset/base/330974

Log:
  MFV r330973: 9164 assert: newds == os->os_dsl_dataset

  illumos/illumos-gate@5f5913bb83405db87f982abee80162a479d363af
  https://github.com/illumos/illumos-gate/commit/5f5913bb83405db87f982abee80162a479d363af

  https://www.illumos.org/issues/9164
    This issue has been reported by Alan Somers as
    https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=225877

    dmu_objset_refresh_ownership() first disowns a dataset (and releases
    it) and then owns it again. There is an assert that the new dataset
    object is the same as the old dataset object.  When running ZFS Test
    Suite on FreeBSD we see this panic from zpool_upgrade_007_pos test:

    panic: solaris assert: newds == os->os_dsl_dataset (0xfffff80045f4c000
    == 0xfffff80021ab4800)

    I see that the old dataset has dsl_dataset_evict_async() pending in
    ds_dbu.dbu_tqent and its ds_dbuf is NULL.

  Reviewed by: Matt Ahrens <mahrens@delphix.com>
  Reviewed by: Don Brady <don.brady@delphix.com>
  Approved by: Richard Lowe <richlowe@richlowe.net>
  Author: Andriy Gapon <avg@FreeBSD.org>

  PR:		225877
  Reported by:	asomers
  MFC after:	1 week

Changes:
_U  head/sys/cddl/contrib/opensolaris/
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu_objset.h
  head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c
Comment 4 commit-hook freebsd_committer 2018-03-15 08:53:51 UTC
A commit references this bug:

Author: avg
Date: Thu Mar 15 08:52:49 UTC 2018
New revision: 330975
URL: https://svnweb.freebsd.org/changeset/base/330975

Log:
  re-enable zpool_upgrade_007_pos test after the fix in r330974

  The test was disabled in r329248.

  PR:		225877

Changes:
  head/tests/sys/cddl/zfs/tests/cli_root/zpool_upgrade/zpool_upgrade_test.sh
Comment 5 commit-hook freebsd_committer 2018-03-27 11:33:29 UTC
A commit references this bug:

Author: avg
Date: Tue Mar 27 11:33:21 UTC 2018
New revision: 331611
URL: https://svnweb.freebsd.org/changeset/base/331611

Log:
  MFC r330974: MFV r330973: 9164 assert: newds == os->os_dsl_dataset

  PR:		225877

Changes:
_U  stable/11/
  stable/11/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c
  stable/11/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu_objset.h
  stable/11/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c
Comment 6 commit-hook freebsd_committer 2018-03-27 11:35:33 UTC
A commit references this bug:

Author: avg
Date: Tue Mar 27 11:34:34 UTC 2018
New revision: 331612
URL: https://svnweb.freebsd.org/changeset/base/331612

Log:
  MFC r330974: MFV r330973: 9164 assert: newds == os->os_dsl_dataset

  PR:		225877

Changes:
_U  stable/10/
  stable/10/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dmu_objset.c
  stable/10/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/sys/dmu_objset.h
  stable/10/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/zfs_ioctl.c