Let's start ngctl without arguments (interactive mode) and run the following commands to create ng_ksocket(4) instance listening tcp/9 port and connected to ng_hole(4) to implement TCP "discard" service for demonstration: mkpeer hole dummy dummy name .:dummy hole mkpeer hole: ksocket hohook inet/stream/tcp name hole:hohook kso # setsockopt(SOL_SOCKET, SO_REUSEPORT, 1) msg kso: setopt { level=0xffff name=0x0200 value = [1] } msg kso: bind inet/0.0.0.0:9 msg kso: listen 1 msg kso: accept For FreeBSD releases before 12.0, it works: one can do "telnet localhost 9" to make connection to the port. Also ngctl command "shutdown kso:" works just fine thereafter. For FreeBSD versions after 12.0-RELEASE, "shutdown kso:" works if we run it before "msg kso: listen 1". After that stage, "shutdown kso:" produces instant panic: Fatal trap 12: page fault while in kernel mode cpuid = 0; apic id = 00 fault virtual address = 0x18 fault code = supervisor read data, page not present instruction pointer = 0x20:0xffffffff80b26298 stack pointer = 0x28:0xfffffe00085ff9c0 frame pointer = 0x28:0xfffffe00085ffa00 code segment = base rx0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 730 (ngctl) rdi: 0000000000000000 rsi: fffffe00099651e0 rdx: 0000000000000000 rcx: 00000000000003aa r8: 0000000000000001 r9: 0000000000010000 rax: fffff80004064b40 rbx: 0000000000000018 rbp: fffffe00085ffa00 r10: 0000000000000001 r11: 0000000000010000 r12: 00000000000003aa r13: 0000000000000000 r14: fffff8000352c400 r15: ffffffff823256a5 trap number = 12 panic: page fault cpuid = 0 time = 1700061288 KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe00085ff780 vpanic() at vpanic+0x150/frame 0xfffffe00085ff7d0 panic() at panic+0x43/frame 0xfffffe00085ff830 trap_fatal() at trap_fatal+0x40a/frame 0xfffffe00085ff890 trap_pfault() at trap_pfault+0xae/frame 0xfffffe00085ff8f0 calltrap() at calltrap+0x8/frame 0xfffffe00085ff8f0 --- trap 0xc, rip = 0xffffffff80b26298, rsp = 0xfffffe00085ff9c0, rbp = 0xfffffe00085ffa00 --- __mtx_lock_flags() at __mtx_lock_flags+0x48/frame 0xfffffe00085ffa00 ng_ksocket_shutdown() at ng_ksocket_shutdown+0x39/frame 0xfffffe00085ffa20 ng_rmnode() at ng_rmnode+0x188/frame 0xfffffe00085ffa50 ng_apply_item() at ng_apply_item+0x4fb/frame 0xfffffe00085ffae0 ng_snd_item() at ng_snd_item+0x2cc/frame 0xfffffe00085ffb20 ngc_send() at ngc_send+0x1b3/frame 0xfffffe00085ffbc0 sosend_generic() at sosend_generic+0x5fe/frame 0xfffffe00085ffc70 sousrsend() at sousrsend+0x79/frame 0xfffffe00085ffcd0 kern_sendit() at kern_sendit+0x1c0/frame 0xfffffe00085ffd60 sendit() at sendit+0xb7/frame 0xfffffe00085ffdb0 sys_sendto() at sys_sendto+0x4d/frame 0xfffffe00085ffe00 amd64_syscall() at amd64_syscall+0x139/frame 0xfffffe00085fff30 fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe00085fff30 --- syscall (133, FreeBSD ELF64, sendto), rip = 0x556e6b62ffa, rsp = 0x556e4385f98, rbp = 0x556e4385fe0 --- KDB: enter: panic [ thread pid 730 tid 100070 ] Stopped at kdb_enter+0x32: movq $0,0xde93a3(%rip) db> I got this panic using somewhat old 14.0-CURRENT system running in my bhyve. This bug makes it impossible running net/mpd5 as PPP over TCP server as mpd5 uses ng_ksocket.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=43f7e21668105cc5a3c66eae5ecef0203c2df62f commit 43f7e21668105cc5a3c66eae5ecef0203c2df62f Author: Gleb Smirnoff <glebius@FreeBSD.org> AuthorDate: 2023-11-17 17:24:30 +0000 Commit: Gleb Smirnoff <glebius@FreeBSD.org> CommitDate: 2023-11-17 17:24:30 +0000 ng_ksocket: fix accept(2) - Provide listen upcall and set it on NGM_KSOCKET_LISTEN - Mask EWOULDBLOCK on NGM_KSOCKET_ACCEPT Reviewed by: afedorov Differential Revision: https://reviews.freebsd.org/D42637 PR: 272319 PR: 275106 Fixes: 779f106aa169256b7010a1d8f963ff656b881e92 sys/netgraph/ng_ksocket.c | 41 +++++++++++++++++++++++++++++++++++------ 1 file changed, 35 insertions(+), 6 deletions(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=efad7cbfdc06e92bcc589a6c0cae2f3bea0d5cb9 commit efad7cbfdc06e92bcc589a6c0cae2f3bea0d5cb9 Author: Gleb Smirnoff <glebius@FreeBSD.org> AuthorDate: 2023-11-17 17:23:58 +0000 Commit: Gleb Smirnoff <glebius@FreeBSD.org> CommitDate: 2023-11-17 17:23:58 +0000 ng_ksocket: fix upcall clearing on node shutdown Note: imho, the proper solution would be to guarantee that upcalls won't ever be called after soclose(), but this isn't the case, yet. This change at least makes the node work the way it always worked. Reviewed by: afedorov Differential Revision: https://reviews.freebsd.org/D42636 PR: 272319 PR: 275106 Fixes: 779f106aa169256b7010a1d8f963ff656b881e92 sys/netgraph/ng_ksocket.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=09f4b840bd7cb6427af2a28a10bd839da6dd76d5 commit 09f4b840bd7cb6427af2a28a10bd839da6dd76d5 Author: Gleb Smirnoff <glebius@FreeBSD.org> AuthorDate: 2023-11-17 17:23:58 +0000 Commit: Gleb Smirnoff <glebius@FreeBSD.org> CommitDate: 2023-11-30 17:01:39 +0000 ng_ksocket: fix upcall clearing on node shutdown Note: imho, the proper solution would be to guarantee that upcalls won't ever be called after soclose(), but this isn't the case, yet. This change at least makes the node work the way it always worked. Reviewed by: afedorov Differential Revision: https://reviews.freebsd.org/D42636 PR: 272319 PR: 275106 Fixes: 779f106aa169256b7010a1d8f963ff656b881e92 (cherry picked from commit efad7cbfdc06e92bcc589a6c0cae2f3bea0d5cb9) sys/netgraph/ng_ksocket.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-)
A commit in branch stable/14 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=ae3c8991cf0db9beff762f90b55e8995326eb894 commit ae3c8991cf0db9beff762f90b55e8995326eb894 Author: Gleb Smirnoff <glebius@FreeBSD.org> AuthorDate: 2023-11-17 17:24:30 +0000 Commit: Gleb Smirnoff <glebius@FreeBSD.org> CommitDate: 2023-11-30 17:01:40 +0000 ng_ksocket: fix accept(2) - Provide listen upcall and set it on NGM_KSOCKET_LISTEN - Mask EWOULDBLOCK on NGM_KSOCKET_ACCEPT Reviewed by: afedorov Differential Revision: https://reviews.freebsd.org/D42637 PR: 272319 PR: 275106 Fixes: 779f106aa169256b7010a1d8f963ff656b881e92 (cherry picked from commit 43f7e21668105cc5a3c66eae5ecef0203c2df62f) sys/netgraph/ng_ksocket.c | 41 +++++++++++++++++++++++++++++++++++------ 1 file changed, 35 insertions(+), 6 deletions(-)
Fixes to ng_ksocket merged to stable/14. With INVARIANTS stable/14 would still panic. However, problem now being catched by INVARIANTS was there always, so can be ignored. The problem reported by INVARIANTS fixed in the main branch. The change is too intrusive to be merged to a stable branch.
(In reply to Gleb Smirnoff from comment #5) Thank you. Should it work if I merge both fixes to stable/13?
I don't know for sure, but very likely. Try it.