Bug 242047 - ctld zvol hangups
Summary: ctld zvol hangups
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.1-RELEASE
Hardware: amd64 Any
: --- Affects Many People
Assignee: freebsd-fs mailing list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-11-18 05:33 UTC by pprocacci
Modified: 2019-11-23 19:14 UTC (History)
1 user (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description pprocacci 2019-11-18 05:33:15 UTC
I believe I'm having similar issues to the following two bug reports but am not 100% sure:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229958
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=226499

I'm therefore creating a new bug report detailing the problems I have happening.

I've created a 1TB ZVOL and have giving it to ctld for iSCSI attachment elsewhere.
When the host that is copying large amount of data to this ZVOL fires up over iSCSI, the iSCSI disk hangs and eventually disappears.

Furthermore, while this is happening, some processes (not all) on the host itself hang entirely.  zpool related commands hang, attempts to restart ctld hang, a forceful kill of ctld does nothing.

I'm really not sure what information to provide other than my zfs configuration:

-------------------------------
zpool status -v
  pool: tank
 state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          raidz2-0  ONLINE       0     0     0
            ada0    ONLINE       0     0     0
            ada1    ONLINE       0     0     0
            ada2    ONLINE       0     0     0
            ada3    ONLINE       0     0     0

errors: No known data errors

  pool: zroot
 state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        zroot       ONLINE       0     0     0
          nvd0p2    ONLINE       0     0     0

errors: No known data errors
-------------------------------

Other than the above, the only thing I can think to provide are system specs:

CPU: 29.7% user,  0.2% nice,  2.5% system,  0.1% interrupt, 67.4% idle
Mem: 737M Active, 3927M Inact, 13M Laundry, 55G Wired, 2334M Free
ARC: 46G Total, 21G MFU, 21G MRU, 2295M Anon, 204M Header, 1635M Other
     40G Compressed, 41G Uncompressed, 1.02:1 Ratio


.. and perhaps OS version:  # uname -a
FreeBSD nas.myhome 12.1-RELEASE-p1 FreeBSD 12.1-RELEASE-p1 GENERIC  amd64


If there's a way I can debug this problem while it's occuring, I'd like to know so I can pass it along, but it appears in the other two bug reports I've listed, the problem area is around ZVOL/locking.
Comment 1 pprocacci 2019-11-18 14:29:31 UTC
I was able to gather a procstat of one of my hung `mv` commands:

# procstat -kk 73788
  PID    TID COMM                TDNAME              KSTACK
73788 101264 cp                  -                   mi_switch+0xe2 sleepq_wait+0x2c _cv_wait+0x152 txg_wait_synced+0xa5 dmu_tx_assign+0x48 zfs_freebsd_write+0x4a2 VOP_WRITE_APV+0xec vn_write+0x260 vn_io_fault_doio+0x43 vn_io_fault1+0x2d4 vn_io_fault+0x18f dofilewrite+0xb0 sys_write+0xc1 amd64_syscall+0x364 fast_syscall_common+0x101