Bug 255131

Summary: kernel panic using zfs on shsec
Product: Base System Reporter: nvass
Component: kernAssignee: Mark Johnston <markj>
Status: Closed FIXED    
Severity: Affects Some People CC: markj, nvass
Priority: --- Keywords: crash
Version: CURRENT   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
Disabling BIO_FLUSH fixes the panic
none
proposed patch none

Description nvass 2021-04-16 19:55:51 UTC 
Trying to use zfs on a shsec device will result to a kernel panic.

Use the following script to reproduce the problem:
#!/bin/sh
gshsec load
m0=`mdconfig -at swap -s 120M`
m1=`mdconfig -at swap -s 120M`
m2=`mdconfig -at swap -s 120M`

gshsec label secret1234 $m0 $m1 $m2
zpool create secret1234 /dev/shsec/secret1234



Panic message:
Unread portion of the kernel message buffer:
panic: non-NULL bp->data in g_io_request(cmd=5)
cpuid = 1
time = 1618601239
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe0063abfa30
vpanic() at vpanic+0x181/frame 0xfffffe0063abfa80
panic() at panic+0x43/frame 0xfffffe0063abfae0
g_io_request() at g_io_request+0x365/frame 0xfffffe0063abfb10
g_shsec_start() at g_shsec_start+0x1db/frame 0xfffffe0063abfb70
g_io_schedule_down() at g_io_schedule_down+0x134/frame 0xfffffe0063abfba0
g_down_procbody() at g_down_procbody+0x5c/frame 0xfffffe0063abfbb0
fork_exit() at fork_exit+0x80/frame 0xfffffe0063abfbf0
fork_trampoline() at fork_trampoline+0xe/frame 0xfffffe0063abfbf0
--- trap 0, rip = 0, rsp = 0, rbp = 0 ---
KDB: enter: panic
Comment 1 nvass 2022-01-26 15:17:03 UTC 
Created attachment 231357 [details]
Disabling BIO_FLUSH fixes the panic

A quick fix in the attached patch is to disable BIO_FLUSH handling. A more proper fix would be to create a BIO_FLUSH handler.
Comment 2 nvass 2022-01-26 15:20:14 UTC 
This should be assigned to geom@
Comment 3 Mark Johnston freebsd_committer freebsd_triage 2022-01-26 16:00:11 UTC 
Created attachment 231358 [details]
proposed patch

The problem really appears to be that shsec erroneously allocates a buffer for BIO_FLUSH commands that get sent to child providers.  It should only be doing this for reads and writes, so the attached patch is sufficient, I believe.
Comment 4 nvass 2022-01-27 13:48:38 UTC 
Hi Mark,

Thank you for the analysis and the patch. I've tested it and it works fine. Please, commit it. It'd be nice to MFC too

Thanks, Nikos
Comment 5 commit-hook freebsd_committer freebsd_triage 2022-01-27 15:15:35 UTC 
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=a2dfffb98917a57bfacb155b9d7d423c3e8ff792

commit a2dfffb98917a57bfacb155b9d7d423c3e8ff792
Author:     Mark Johnston <markj@FreeBSD.org>
AuthorDate: 2022-01-27 14:53:02 +0000
Commit:     Mark Johnston <markj@FreeBSD.org>
CommitDate: 2022-01-27 14:56:07 +0000

    shsec: Allocate data blocks only for BIO_READ/WRITE requests

    In particular, there is no need to allocate a data block when passing
    BIO_FLUSH requests to child providers, and g_io_request() asserts that
    bp->bio_data == NULL for such requests.

    PR:             255131
    Reported and tested by: nvass@gmx.com
    MFC after:      2 weeks
    Sponsored by:   The FreeBSD Foundation

 sys/geom/shsec/g_shsec.c | 34 +++++++++++++++++++---------------
 1 file changed, 19 insertions(+), 15 deletions(-)
Comment 6 commit-hook freebsd_committer freebsd_triage 2022-02-10 13:47:36 UTC 
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=14c0cde468e136a007a5a7e1bcb7b3a50a2a8b57

commit 14c0cde468e136a007a5a7e1bcb7b3a50a2a8b57
Author:     Mark Johnston <markj@FreeBSD.org>
AuthorDate: 2022-01-27 14:53:02 +0000
Commit:     Mark Johnston <markj@FreeBSD.org>
CommitDate: 2022-02-10 13:46:12 +0000

    shsec: Allocate data blocks only for BIO_READ/WRITE requests

    In particular, there is no need to allocate a data block when passing
    BIO_FLUSH requests to child providers, and g_io_request() asserts that
    bp->bio_data == NULL for such requests.

    PR:             255131
    Reported and tested by: nvass@gmx.com
    Sponsored by:   The FreeBSD Foundation

    (cherry picked from commit a2dfffb98917a57bfacb155b9d7d423c3e8ff792)

 sys/geom/shsec/g_shsec.c | 34 +++++++++++++++++++---------------
 1 file changed, 19 insertions(+), 15 deletions(-)