Bug 245653

Summary: dns/bind916: 9.16.2 crashes on 12.1 after libuv update: general: critical: netmgr.c:995: REQUIRE(worker->recvbuf_inuse) failed, back trace
Product: Ports & Packages Reporter: lysfjord.daniel
Component: Individual Port(s)Assignee: Mathieu Arnold <mat>
Status: Closed FIXED    
Severity: Affects Many People CC: cmt, ohartmann, tatsuki_makino, thierry, vvd
Priority: --- Keywords: crash, needs-qa, regression
Version: LatestFlags: bugzilla: maintainer-feedback? (mat)
thierry: maintainer-feedback+
Hardware: amd64   
OS: Any   
Attachments:
Description Flags
just an experiment for devel/libuv. tatsuki_makino: maintainer-approval-

Description lysfjord.daniel 2020-04-15 23:30:57 UTC 
Packages built by poudriere.

I upgraded bind916 to 9.16.2 from 9.16.1, upon restarting bind, the following log entries appeared in /var/log/messages: (the built with is identical between 9.16.2 and 9.16.1)

named 55088 - - starting BIND 9.16.2 (Stable Release) <id:b310dc7>
named 55088 - - running on FreeBSD amd64 12.1-RELEASE-p2 FreeBSD 12.1-RELEASE-p2 GENERIC
named 55088 - - built with '--disable-linux-caps' '--localstatedir=/var' '--sysconfdir=/usr/local/etc/namedb' '--with-dlopen=yes' '--with-libxml2' '--with-openssl=/usr/local' '--with-readline=-L/usr/local/lib -ledit' '--with-dlz-filesystem=yes' '--disable-dnstap' '--disable-fixed-rrset' '--disable-geoip' '--without-maxminddb' '--without-gssapi' '--with-libidn2=/usr/local' '--with-json-c' '--disable-largefile' '--with-lmdb=/usr/local' '--disable-native-pkcs11' '--without-python' '--disable-querytrace' 'STD_CDEFINES=-DDIG_SIGCHASE=1' '--enable-tcp-fastopen' '--with-tuning=default' '--disable-symtable' '--prefix=/usr/local' '--mandir=/usr/local/man' '--infodir=/usr/local/share/info/' '--build=amd64-portbld-freebsd12.1' 'build_alias=amd64-portbld-freebsd12.1' 'CC=cc' 'CFLAGS=-O2 -pipe -march=skylake -DLIBICONV_PLUG -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing ' 'LDFLAGS= -L/usr/local/lib -ljson-c -Wl,-rpath,/usr/local/lib -fstack-protector-strong ' 'LIBS=-L/usr/local/lib' 'CPPFLAGS=-DLIBICONV_PLUG -isystem /usr/local/include' 'CPP=cpp' 'PKG_CONFIG=pkgconf'
named 55088 - - running as: named -4 -u bind -c /usr/local/etc/namedb/named.conf
named 55088 - - compiled by CLANG 4.2.1 Compatible FreeBSD Clang 8.0.1 (tags/RELEASE_801/final 366581)
named 55088 - - compiled with OpenSSL version: LibreSSL 3.0.2
named 55088 - - linked to OpenSSL version: LibreSSL 3.0.2
named 55088 - - compiled with libxml2 version: 2.9.10
named 55088 - - linked to libxml2 version: 20910
named 55088 - - compiled with json-c version: 0.13.1
named 55088 - - linked to json-c version: 0.13.1
named 55088 - - compiled with zlib version: 1.2.11
named 55088 - - linked to zlib version: 1.2.11
-----8<----SNIP ZONE LOADING---->8-----
named 55088 - - command channel listening on 127.0.0.1#953
kernel: [6745829] pid 55088 (named), jid 0, uid 53: exited on signal 6


The following lines appeared in /var/log/bind/named.log:
16-Apr-2020 00:51:34.182 general: critical: netmgr.c:995: REQUIRE(worker->recvbuf_inuse) failed, back trace
16-Apr-2020 00:51:34.182 general: critical: #0 0x2cae20 in ??
16-Apr-2020 00:51:34.182 general: critical: #1 0x4a8e4a in ??
16-Apr-2020 00:51:34.182 general: critical: #2 0x4be749 in ??
16-Apr-2020 00:51:34.182 general: critical: #3 0x4c2b11 in ??
16-Apr-2020 00:51:34.182 general: critical: #4 0x800923c00 in ??
16-Apr-2020 00:51:34.182 general: critical: #5 0x800925dda in ??
16-Apr-2020 00:51:34.182 general: critical: #6 0x800915321 in ??
16-Apr-2020 00:51:34.182 general: critical: #7 0x4bcb74 in ??
16-Apr-2020 00:51:34.182 general: critical: #8 0x800941736 in ??
16-Apr-2020 00:51:34.182 general: critical: exiting (due to assertion failure)

Downgrading to 9.16.1 resulted in the same assertion failure, even if that was the previously running version. /var/log/bin/named.log from 9.6.1:
16-Apr-2020 00:53:42.055 general: critical: netmgr.c:1000: REQUIRE(worker->recvbuf_inuse) failed, back trace
16-Apr-2020 00:53:42.055 general: critical: #0 0x2c9d90 in ??
16-Apr-2020 00:53:42.055 general: critical: #1 0x4a707a in ??
16-Apr-2020 00:53:42.055 general: critical: #2 0x4bc819 in ??
16-Apr-2020 00:53:42.055 general: critical: #3 0x4c0a41 in ??
16-Apr-2020 00:53:42.055 general: critical: #4 0x800921c00 in ??
16-Apr-2020 00:53:42.055 general: critical: #5 0x800923dda in ??
16-Apr-2020 00:53:42.055 general: critical: #6 0x800913321 in ??
16-Apr-2020 00:53:42.055 general: critical: #7 0x4bac44 in ??
16-Apr-2020 00:53:42.055 general: critical: #8 0x80093f736 in ??
16-Apr-2020 00:53:42.055 general: critical: exiting (due to assertion failure)

Currently backed up to 9.14.11, whom started just fine.
Comment 1 Tatsuki Makino 2020-04-15 23:54:31 UTC 
This is due to an upgrade of libuv to 1.36.0.
Comment 2 Kubilay Kocak freebsd_committer freebsd_triage 2020-04-15 23:59:27 UTC 
Report [1] of ports r531786 being indicated in this regression

[1] https://lists.freebsd.org/pipermail/svn-ports-all/2020-April/248926.html
Comment 3 Tatsuki Makino 2020-04-16 03:27:08 UTC 
Created attachment 213441 [details]
just an experiment for devel/libuv.

This is an experiment that combines libuv-1.35.0/src/unix/udp.c with a source　excluding libuv-1.36.0/src/unix/udp.c.
If we start named using this, the problem does not seem to occur.
Comment 4 O. Hartmann 2020-04-16 05:57:19 UTC 
(In reply to lysfjord.daniel from comment #0)

The same here on 12-STABLE and CURRENT (FreeBSD 13.0-CURRENT #76 r359859: Mon Apr 13 11:45:44 CEST 2020 amd64), built the traditional way (make).

poudriere bin packages also affected as reported and deployment of those broken BIND 9.16 broke DNS facilities.
Comment 5 Thierry Thomas freebsd_committer freebsd_triage 2020-04-16 07:04:26 UTC 
I can either revert libuv to its previous version, or apply the patch proposed by tatsuki_makino.
Let me know.
Comment 6 Vladimir Druzenko freebsd_committer freebsd_triage 2020-04-16 07:58:03 UTC 
(In reply to Thierry Thomas from comment #5)
> apply the patch proposed by tatsuki_makino.
Your patch work fine for me on two hosts with 12.1 amd64.
Comment 7 Thierry Thomas freebsd_committer freebsd_triage 2020-04-16 08:53:57 UTC 
Upstream just reverted some commits: see https://github.com/libuv/libuv/pull/2792

I'm about to commit this.
Comment 8 commit-hook freebsd_committer freebsd_triage 2020-04-16 09:38:02 UTC 
A commit references this bug:

Author: thierry
Date: Thu Apr 16 09:37:41 UTC 2020
New revision: 531835
URL: https://svnweb.freebsd.org/changeset/ports/531835

Log:
  Adding patches from upstream (by bnoordhuis) to solve an issue with dns/bind916).

  See <https://github.com/libuv/libuv/issues/2791>
  and <https://github.com/libuv/libuv/pull/2792>

  PR:		245653
  Reported by:	lysfjord.daniel(at)smokepit.net and many

Changes:
  head/devel/libuv/Makefile
  head/devel/libuv/files/
  head/devel/libuv/files/patch-docs_src_udp.rst
  head/devel/libuv/files/patch-include_uv.h
  head/devel/libuv/files/patch-src_unix_freebsd.c
  head/devel/libuv/files/patch-src_unix_internal.h
  head/devel/libuv/files/patch-src_unix_linux-syscalls.c
  head/devel/libuv/files/patch-src_unix_linux-syscalls.h
  head/devel/libuv/files/patch-src_unix_udp.c
Comment 9 O. Hartmann 2020-04-16 11:22:33 UTC 
Just installed new devel/libuv after this commit, dns/bind916 now seems to work as expected. Thank you very much for the fast response.
Comment 10 Thierry Thomas freebsd_committer freebsd_triage 2020-04-16 11:51:22 UTC 
Fixed, thanks everyone for the quick report!