Steps to reproduce: 1. Run a guest. I use Ubuntu Linux for testing: bhyve -c 2 -m 4096 -S -A -I -u -H -P -s 0:0,hostbridge -l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd -s 1:0,lpc -s 7:0,virtio-blk,/data/img/u buntu2510.img -l com1,stdio -s 8:0,virtio-net,slirp,mac=94:06:31:1a:a6:f7,open ubuntu2510 2. Generate traffic from the guest. I use the following command: while wget https://download.freebsd.org/releases/ISO-IMAGES/15.0/FreeBSD-15.0-RELEASE-arm64-aarch64-dvd1.iso -O /dev/null; do : ; done bhyve crashes with: thr_kill () at thr_kill.S:4 4 RSYSCALL(thr_kill) /dev/null 68%[===========================================================================================================> ] 2.63G --.-KB/s eta 3m 13s (gdb) bt #0 thr_kill () at thr_kill.S:4 #1 0x000000080114c934 in __raise (s=s@entry=6) at /usr/home/novel/code/freebsd-src/lib/libc/gen/raise.c:48 #2 0x00000008011fe7a9 in abort () at /usr/home/novel/code/freebsd-src/lib/libc/stdlib/abort.c:61 #3 0x000000080112f521 in __assert (func=<optimized out>, file=<optimized out>, line=line@entry=113, failedexpr=<optimized out>) at /usr/home/novel/code/freebsd-src/lib/libc/gen/assert.c:47 #4 0x0000000001024f2e in slirp_cb_send_packet (buf=<optimized out>, len=1514, param=<optimized out>) at /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/slirp/slirp-helper.c:113 #5 0x000000080141ad02 in ?? () from /usr/local/lib/libslirp.so.0 #6 0x0000000801412f7d in ?? () from /usr/local/lib/libslirp.so.0 #7 0x0000000801416366 in ?? () from /usr/local/lib/libslirp.so.0 #8 0x0000000801421197 in ?? () from /usr/local/lib/libslirp.so.0 #9 0x000000080141a718 in slirp_pollfds_poll () from /usr/local/lib/libslirp.so.0 #10 0x0000000001024c73 in slirp_pollfd_loop (priv=0x7fffffffe7b0) at /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/slirp/slirp-helper.c:263 #11 main (argc=<optimized out>, argv=<optimized out>) at /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/slirp/slirp-helper.c:566 (gdb) fr 4 #4 0x0000000001024f2e in slirp_cb_send_packet (buf=<optimized out>, len=1514, param=<optimized out>) at /usr/home/novel/code/freebsd-src/usr.sbin/bhyve/slirp/slirp-helper.c:113 113 assert((size_t)n == len); (gdb) p n $1 = 0 (gdb) Host system is: FreeBSD tulp 16.0-CURRENT FreeBSD 16.0-CURRENT #1 main-n282364-69e041521a80: Fri Dec 5 16:24:19 CET 2025 novel@tulp:/usr/obj/usr/home/novel/code/freebsd-src/amd64.amd64/sys/TULP amd64
Additional node: commenting out this assert appears to "fix" the issue. At least the wget loop runs for a few hours without issues. With iperf run I get the following result: client$ iperf -c <server> -u -i 1 -P 8 -t 600 -b 100M server: [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 46] 0.00-600.02 sec 7.32 GBytes 105 Mbits/sec 0.048 ms 2727/5347621 (0.051%) [ 46] 0.00-600.02 sec 7 datagrams received out-of-order [ 47] 0.00-600.02 sec 7.32 GBytes 105 Mbits/sec 0.047 ms 2867/5347593 (0.054%) [ 42] 0.00-600.02 sec 7.32 GBytes 105 Mbits/sec 0.048 ms 3005/5349831 (0.056%) [ 42] 0.00-600.02 sec 4 datagrams received out-of-order [ 44] 0.00-600.02 sec 7.32 GBytes 105 Mbits/sec 0.047 ms 2673/5349876 (0.05%) [ 44] 0.00-600.02 sec 12 datagrams received out-of-order [ 45] 0.00-600.02 sec 7.32 GBytes 105 Mbits/sec 0.048 ms 2777/5349872 (0.052%) [ 45] 0.00-600.02 sec 16 datagrams received out-of-order [ 49] 0.00-600.02 sec 7.32 GBytes 105 Mbits/sec 0.048 ms 3017/5347630 (0.056%) [ 48] 0.00-600.02 sec 7.32 GBytes 105 Mbits/sec 0.048 ms 2795/5347580 (0.052%) [ 48] 0.00-600.02 sec 2 datagrams received out-of-order [ 43] 0.00-600.02 sec 7.32 GBytes 105 Mbits/sec 0.048 ms 2775/5349856 (0.052%) [ 43] 0.00-600.02 sec 4 datagrams received out-of-order [ ID] Interval Transfer Bandwidth [SUM] 0.00-600.02 sec 58.6 GBytes 838 Mbits/sec 22636/42789859 (0.053%) [SUM] 0.00-600.02 sec 45 datagrams received out-of-order On server I also see warnings like: WARN: recvmsg MSG_CTRUNC occured.
I believe https://reviews.freebsd.org/D54340 will fix the problem. Can you confirm this?
(In reply to Mark Johnston from comment #2) Yes, this works, no failures, thanks! iperf same client command (iperf -c <client> -u -i 1 -P 8 -t 60 -b 500M), server: [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 38] 0.00-60.01 sec 412 MBytes 57.6 Mbits/sec 0.077 ms 110/294004 (0.037%) [ 40] 0.00-60.00 sec 710 MBytes 99.2 Mbits/sec 0.073 ms 286/506505 (0.056%) [ 34] 0.00-60.00 sec 1.04 GBytes 149 Mbits/sec 0.070 ms 173/761382 (0.023%) [ 36] 0.00-60.00 sec 699 MBytes 97.8 Mbits/sec 0.074 ms 142/499064 (0.028%) [ 39] 0.00-60.00 sec 489 MBytes 68.3 Mbits/sec 0.075 ms 153/348742 (0.044%) [ 35] 0.00-60.01 sec 754 MBytes 105 Mbits/sec 0.072 ms 117/537620 (0.022%) [ 33] 0.00-60.00 sec 1.19 GBytes 170 Mbits/sec 0.071 ms 310/866389 (0.036%) [ 37] 0.00-60.01 sec 807 MBytes 113 Mbits/sec 0.071 ms 169/575468 (0.029%) [ 37] 0.00-60.01 sec 1 datagrams received out-of-order [ ID] Interval Transfer Bandwidth [SUM] 0.00-60.01 sec 6.01 GBytes 860 Mbits/sec 1460/4389174 (0.033%) [SUM] 0.00-60.01 sec 1 datagrams received out-of-order JFYI, this is from a guest running on hostA, and iperf server is running on hostB. Running the same client command on hostA directly: [ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams [ 54] 0.00-60.01 sec 864 MBytes 121 Mbits/sec 0.120 ms 3/616558 (0.00049%) [ 54] 0.00-60.01 sec 9 datagrams received out-of-order [ 56] 0.00-60.01 sec 847 MBytes 118 Mbits/sec 0.123 ms 17/603955 (0.0028%) [ 51] 0.00-60.01 sec 851 MBytes 119 Mbits/sec 0.122 ms 0/607005 (0%) [ 51] 0.00-60.01 sec 4 datagrams received out-of-order [ 49] 0.00-60.01 sec 864 MBytes 121 Mbits/sec 0.122 ms 1/616129 (0.00016%) [ 50] 0.00-60.01 sec 858 MBytes 120 Mbits/sec 0.122 ms 3/612051 (0.00049%) [ 50] 0.00-60.01 sec 2 datagrams received out-of-order [ 52] 0.00-60.03 sec 850 MBytes 119 Mbits/sec 0.123 ms 1/606176 (0.00016%) [ 52] 0.00-60.03 sec 4 datagrams received out-of-order [ 53] 0.00-60.03 sec 855 MBytes 119 Mbits/sec 0.122 ms 11/609725 (0.0018%) [ 53] 0.00-60.03 sec 2 datagrams received out-of-order [ 55] 0.00-61.01 sec 855 MBytes 118 Mbits/sec 0.123 ms 12/609799 (0.002%) [ 55] 0.00-61.01 sec 4 datagrams received out-of-order [ ID] Interval Transfer Bandwidth [SUM] 0.00-61.01 sec 6.68 GBytes 941 Mbits/sec 47/4881398 (0.00096%) [SUM] 0.00-61.01 sec 25 datagrams received out-of-order
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=daef625cf884dea33d50ad7a0e2da0879a442495 commit daef625cf884dea33d50ad7a0e2da0879a442495 Author: Mark Johnston <markj@FreeBSD.org> AuthorDate: 2025-12-22 19:12:34 +0000 Commit: Mark Johnston <markj@FreeBSD.org> CommitDate: 2025-12-29 14:49:24 +0000 bhyve: Use a blocking socket in the helper process The send_packet callback does not handle EAGAIN, and on the recv side we already use poll() and MSG_DONTWAIT to implement a non-blocking loop. PR: 291616 Tested by: novel Fixes: 0e62ebd20172 ("bhyve: Move the slirp backend out into a separate process") Differential Revision: https://reviews.freebsd.org/D54340 usr.sbin/bhyve/slirp/slirp-helper.c | 2 ++ 1 file changed, 2 insertions(+)
Thanks for the report!
A commit in branch stable/15 references this bug: URL: https://cgit.FreeBSD.org/src/commit/?id=777f67d8a9c3ad8f3b90c378cb5c1754a7d4c554 commit 777f67d8a9c3ad8f3b90c378cb5c1754a7d4c554 Author: Mark Johnston <markj@FreeBSD.org> AuthorDate: 2025-12-22 19:12:34 +0000 Commit: Mark Johnston <markj@FreeBSD.org> CommitDate: 2026-06-25 15:34:06 +0000 bhyve: Use a blocking socket in the helper process The send_packet callback does not handle EAGAIN, and on the recv side we already use poll() and MSG_DONTWAIT to implement a non-blocking loop. PR: 291616 Tested by: novel Fixes: 0e62ebd20172 ("bhyve: Move the slirp backend out into a separate process") Differential Revision: https://reviews.freebsd.org/D54340 (cherry picked from commit daef625cf884dea33d50ad7a0e2da0879a442495) usr.sbin/bhyve/slirp/slirp-helper.c | 2 ++ 1 file changed, 2 insertions(+)