Bug 254610 - [tcp] tcp_bbr broken after 69a34e8d0235c0304a28bf8ab076b931aa61835f
Summary: [tcp] tcp_bbr broken after 69a34e8d0235c0304a28bf8ab076b931aa61835f
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Some People
Assignee: Randall Stewart
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-03-27 20:27 UTC by Xin LI
Modified: 2021-06-16 18:28 UTC (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Xin LI freebsd_committer 2021-03-27 20:27:19 UTC
After https://cgit.freebsd.org/src/commit/?id=69a34e8d0235c0304a28bf8ab076b931aa61835f , a kernel configured with tcp_bbr would panic on start when a network facing service starts:

panic: mtx_lock() of spin mutex socket @ /usr/src/sys/kern/uipc_socket.c:3615
cpuid = 3
time = 1616820332
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe01f70315f0
vpanic() at vpanic+0x181/frame 0xfffffe01f7031640
panic() at panic+0x43/frame 0xfffffe01f70316a0
__mtx_lock_flags() at __mtx_lock_flags+0x13b/frame 0xfffffe01f70316f0
soo_kqfilter() at soo_kqfilter+0x93/frame 0xfffffe01f7031730
kqueue_register() at kqueue_register+0x680/frame 0xfffffe01f70317c0
kqueue_kevent() at kqueue_kevent+0x103/frame 0xfffffe01f7031a90
kern_kevent_fp() at kern_kevent_fp+0x95/frame 0xfffffe01f7031ae0
kern_kevent() at kern_kevent+0x80/frame 0xfffffe01f7031b40
kern_kevent_generic() at kern_kevent_generic+0x70/frame 0xfffffe01f7031ba0
sys_kevent() at sys_kevent+0x61/frame 0xfffffe01f7031c00
amd64_syscall() at amd64_syscall+0x12e/frame 0xfffffe01f7031d30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01f7031d30
--- syscall (560, FreeBSD ELF64, sys_kevent), rip = 0x800c7464a, rsp = 0x7fffdf9ead98, rbp = 0x7fffdf9eade0 ---
.

With the revision reverted locally (commits: e13e4fa6c4eba72de5c4685942de5dbe8f43db73, 0a4f851074a3ca74cd4859c20e7d9807b2aeca65, ab4fad4be14462e347ed24ee3663a18eacfb138e and 69a34e8d0235c0304a28bf8ab076b931aa61835f), the issue is gone.

This is not reproducible if tcp_bbr is not loaded and net.inet.tcp.functions_default=bbr removed from /etc/sysctl.conf, with or without these revisions reverted.
Comment 1 Michael Tuexen freebsd_committer 2021-03-27 20:35:51 UTC
Does this affect releng/13.0?
Comment 2 Xin LI freebsd_committer 2021-03-27 21:21:54 UTC
(In reply to Michael Tuexen from comment #1)
No, the change was not MFC'ed to stable/13 so it won't affect releng/13.0.

I haven't tried BBR on my production systems yet, though, and at home they were mostly running -CURRENT.
Comment 3 Michael Tuexen freebsd_committer 2021-03-30 19:38:24 UTC
Channeling rrs: what is the kernel config you are using?

I also tried to reproduce it on an arm64 system with:
tuexen@parallels:~ % cat /boot/loader.conf
tcp_rack_load="YES"
tcp_bbr_load="YES"
tuexen@parallels:~ % cat /etc/sysctl.conf 
# $FreeBSD$
#
#  This file is read when going to multi-user and its contents piped thru
#  ``sysctl'' to adjust kernel values.  ``man 5 sysctl.conf'' for details.
#

# Uncomment this to prevent users from seeing information about processes that
# are being run under another UID.
#security.bsd.see_other_uids=0
vm.old_mlock=1
net.inet.tcp.functions_default=bbr

I can ssh in that VM.
Comment 4 Mark Johnston freebsd_committer 2021-06-16 15:52:26 UTC
Is this still a problem?
Comment 5 Michael Tuexen freebsd_committer 2021-06-16 16:44:48 UTC
(In reply to Mark Johnston from comment #4)
RACK and BBR have been updated substantially in main and stable/13.
Comment 6 Xin LI freebsd_committer 2021-06-16 18:28:27 UTC
(In reply to Mark Johnston from comment #4)
Sorry, no, this is no longer a problem after a recent update.  Closing as fixed.