Bug 246536

Summary: dns/powerdns-recursor: Segmentation fault running pdns_recursor on i386
Product: Ports & Packages Reporter: jose.belejo
Component: Individual Port(s)Assignee: freebsd-ports-bugs (Nobody) <ports-bugs>
Status: Closed Overcome By Events    
Severity: Affects Some People CC: fernape, jose.belejo, kevans, tremere
Priority: --- Keywords: needs-patch, needs-qa
Version: LatestFlags: tremere: maintainer-feedback+
koobs: merge-quarterly?
Hardware: i386   
OS: Any   
See Also: https://github.com/PowerDNS/pdns/issues/4974
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=256530

Description jose.belejo 2020-05-18 11:42:19 UTC
Hi,

I believe I've found a critical bug on powerdns recursor 4.2.1 for i386 FreeBSD 12.1.

When I run pdns_recursor I get a Segmentation fault at the start. This is a persistent behaviour.

I'm using the default configuration file for recursor, but that happens with a customized conf file or even withou any configuration file.

This bug seems to be specific to i386 port.

System was fresh installed.

root@magalhaes:~ # uname -a
FreeBSD magalhaes 12.1-RELEASE-p5 FreeBSD 12.1-RELEASE-p5 GENERIC  i386
root@magalhaes:~ #
root@magalhaes:~ # freebsd-version
12.1-RELEASE-p5
root@magalhaes:~ #
root@magalhaes:~ # pkg info
boost-libs-1.72.0_1            Free portable C++ libraries (without Boost.Python)
gettext-runtime-0.20.1         GNU gettext runtime libraries and programs
icu-66.1,1                     International Components for Unicode (from IBM)
indexinfo-0.3.1                Utility to regenerate the GNU info page index
libedit-3.1.20191211,1         Command line editor library
lua52-5.2.4                    Small, compilable scripting language providing easy access to C code
pkg-1.13.2                     Package manager
powerdns-recursor-4.2.1        Advanced DNS recursor
protobuf-3.11.4,1              Data interchange format library
sudo-1.8.31p1                  Allow others to run commands as root
root@magalhaes:~ #
root@magalhaes:~ # pdns_recursor
May 18 12:31:09 Asked to run with pdns-distributes-queries set but no distributor threads, raising to 1
May 18 12:31:09 PowerDNS Recursor 4.2.1 (C) 2001-2019 PowerDNS.COM BV
May 18 12:31:09 Using 32-bits mode. Built using clang 8.0.1 (tags/RELEASE_801/final 366581) on Apr 24 2020 19:00:05 by root@121i386-quarterly-job-07.
May 18 12:31:09 PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it according to the terms of the GPL version 2.
May 18 12:31:09 NOT using IPv6 for outgoing queries - set 'query-local-address6=::' to enable
May 18 12:31:09 Only allowing queries from: 127.0.0.0/8, 10.0.0.0/8, 100.64.0.0/10, 169.254.0.0/16, 192.168.0.0/16, 172.16.0.0/12, ::1/128, fc00::/7, fe80::/10
May 18 12:31:09 Will not send queries to: 127.0.0.0/8, 10.0.0.0/8, 100.64.0.0/10, 169.254.0.0/16, 192.168.0.0/16, 172.16.0.0/12, ::1/128, fc00::/7, fe80::/10, 0.0.0.0/8, 192.0.0.0/24, 192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24, 240.0.0.0/4, ::/96, ::ffff:0:0/96, 100::/64, 2001:db8::/32, 0.0.0.0, ::
May 18 12:31:09 PowerDNS Recursor itself will distribute queries over threads
May 18 12:31:09 Inserting rfc 1918 private space zones
May 18 12:31:09 Listening for UDP queries on 127.0.0.1:53
May 18 12:31:09 Listening for TCP queries on 127.0.0.1:53
May 18 12:31:09 Set effective group id to 120
May 18 12:31:09 Set effective user id to 120
May 18 12:31:09 Launching 1 distributor threads
May 18 12:31:09 Launching 2 worker threads
May 18 12:31:09 Done priming cache with root hints
May 18 12:31:09 Done priming cache with root hints
May 18 12:31:09 Done priming cache with root hints
Segmentation fault
root@magalhaes:~ #

log from messages:
May 18 12:22:01 magalhaes pdns_recursor[948]: PowerDNS Recursor 4.2.1 (C) 2001-2019 PowerDNS.COM BV
May 18 12:22:01 magalhaes pdns_recursor[948]: Using 32-bits mode. Built using clang 8.0.1 (tags/RELEASE_801/final 366581) on Apr 24 2020 19:00:05 by root@121i386-quarterly-job-07.
May 18 12:22:01 magalhaes pdns_recursor[948]: PowerDNS comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it according to the terms of the GPL version 2.
May 18 12:22:01 magalhaes pdns_recursor[948]: NOT using IPv6 for outgoing queries - set 'query-local-address6=::' to enable
May 18 12:22:01 magalhaes pdns_recursor[948]: Only allowing queries from: 127.0.0.0/8, 10.0.0.0/8, 100.64.0.0/10, 169.254.0.0/16, 192.168.0.0/16, 172.16.0.0/12, ::1/128, fc00::/7, fe80::/10
May 18 12:22:01 magalhaes pdns_recursor[948]: Will not send queries to: 127.0.0.0/8, 10.0.0.0/8, 100.64.0.0/10, 169.254.0.0/16, 192.168.0.0/16, 172.16.0.0/12, ::1/128, fc00::/7, fe80::/10, 0.0.0.0/8, 192.0.0.0/24, 192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24, 240.0.0.0/4, ::/96, ::ffff:0:0/96, 100::/64, 2001:db8::/32, 0.0.0.0, ::
May 18 12:22:01 magalhaes pdns_recursor[948]: PowerDNS Recursor itself will distribute queries over threads
May 18 12:22:01 magalhaes pdns_recursor[948]: Inserting rfc 1918 private space zones
May 18 12:22:01 magalhaes pdns_recursor[948]: Launching 1 distributor threads
May 18 12:22:01 magalhaes pdns_recursor[948]: Launching 2 worker threads
May 18 12:22:01 magalhaes pdns_recursor[948]: Done priming cache with root hints
May 18 12:22:01 magalhaes kernel: pid 948 (pdns_recursor), jid 0, uid 120: exited on signal 11

Thanks in advance for your attention.

Best regards,
Jose Belejo
Comment 1 Ralf van der Enden 2020-05-20 09:06:50 UTC
I've talked to people upstream and unfortunately it was decided to not look into this issue on i386 by them. See this for reference: https://github.com/PowerDNS/pdns/issues/4974

So unless you or somebody else can supply a patch to fix it I'm afraid I'll have to mark the recursor as broken on i386.
Comment 2 jose.belejo 2020-05-20 12:21:44 UTC
(In reply to Ralf van der Enden from comment #1)
Hi Ralf,

First please let me thank you your concern and trouble you had investigating this subject.

Considering PowerDNS team is not willing to invest more time on i386 platform, I believe it doesn't make much sense for me or anyone else to work on a fix for a specific release without any roadmap for this platform. Strangely, or not, PowerDNS is yet supported on Raspberry PI early models that are as you know 32 bits.

On the other hand, PowerDNS Authoritive Server seems to work fine on i386.

Anyway, no worries about it, although I'm very comfortable to work with PowerDNS recursor, I'll look for another alternative.

Thank you once again.

Best regards,
Jose Belejo
Comment 3 Kubilay Kocak freebsd_committer freebsd_triage 2020-06-29 12:31:03 UTC
@Ralf (Maintainer) If you have reproduced the issue independently, feel free to submit a patch to set BROKEN_i386
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2020-06-29 12:32:17 UTC
(In reply to jose.belejo from comment #2)

Upstream mentioned they would welcome a patch to fix it. Perhaps we should get a backtrace of the crash after building boost and powerdns-recursor (at least) with debug symbols (/etc/make.conf: WITH_DEBUG=yes)
Comment 5 Ralf van der Enden 2020-06-29 12:54:18 UTC
I've tried (working with people from upstream) to get some useful information using gdb, truss, etc. about the crash. Nothing useful however. Upstream suggested running the recursor through valgrind, but unfortunately that's broken on i386 (and possible other archs as well) in more than one way.

In short: I'm at a loss on how to see what's causing the crash (it's 99.9% chance of being related to threading).
Comment 6 Ralf van der Enden 2020-06-29 13:35:13 UTC
With the update to 4.3.1 I already set the BROKEN_i386 (committed May 27th 2020).
Comment 7 Kyle Evans freebsd_committer 2020-06-29 14:10:13 UTC
(In reply to Ralf van der Enden from comment #5)

If you're interested, you can try building devel/valgrind-devel with D25452[0] applied; I believe this largely fixes 12.1/i386.

[0] https://reviews.freebsd.org/D25452
Comment 8 Ralf van der Enden 2020-06-30 08:00:09 UTC
(In reply to Kyle Evans from comment #7)

I've managed to compile valgrind-devel using the review you mentioned. However, running the recursor through several tools didn't give me any useful results. I have to confess I'm a noob at using valgrind, but I would have expected memcheck, helgrind or DRD to show me at least one error.

Any help would be appreciated in what arguments to pass to valgrind in order to get some decent information from it.
Comment 9 Ralf van der Enden 2020-07-01 10:04:37 UTC
Maybe the output from truss for both arch's is useful for somebody more skilled in debugging. I've put both files here: https://www.cainites.net/files/FreeBSD/
Comment 10 Fernando ApesteguĂ­a freebsd_committer 2020-07-07 18:06:11 UTC
Port was marked as broken in ports r536690
Comment 11 Ralf van der Enden 2021-06-12 12:25:21 UTC
Please close this PR. Upstream no longer supports archs with 32 bits time_t.

Set NOT_FOR_ARCHS=i386 in PR bug #256530
Comment 12 Mark Linimon freebsd_committer freebsd_triage 2021-06-12 12:36:26 UTC
^Triage: OBE, see PR 256530.