Bug 244600

Summary: dns/knot-resolver: Fix critical cache space pre-allocation failure bug and add rc scripts
Product: Ports & Packages Reporter: yds <yds>
Component: Individual Port(s)Assignee: Kurt Jaeger <pi>
Status: Closed FIXED    
Severity: Affects Many People CC: freebsd, pi, vcunat
Priority: --- Flags: koobs: maintainer-feedback+
Version: Latest   
Hardware: Any   
OS: Any   
URL: https://GitLab.Labs.NIC.cz/knot/knot-resolver/issues/549
Bug Depends on: 247699    
Bug Blocks:    
Attachments:
Description Flags
[patch] fix critical cache space pre-allocation failure bug and add rc scripts
none
knot-resolver freebsd: maintainer-approval-

Description yds 2020-03-04 21:47:15 UTC
Created attachment 212149 [details]
[patch] fix critical cache space pre-allocation failure bug and add rc scripts

1. add rc service scripts for `kresd` and `kres-cache-gc`
2. change dependency back to `dns/knot2` otherwise both ports cannot be installed at the same time. `dns/knot2` needs to be fixed to also depend on `dns/knot2-lib`
3. add creation of RUNDIR with fixed perms at install time
4. add proper PORTEXAMPLES installation
5. prevent `kresd.conf` from getting deleted or overwritten on reinstall [bug]
6. move `root.keys` to RUNDIR to fix the following error:
```
PANIC: unprotected error in call to Lua API (/usr/local/lib/knot-resolver/trust_anchors.lua:186: /usr/local/etc/knot-resolver/root.keys.lock.35621: Permission denied)
```
7. added patch to fix the following critical bug:
   <https://GitLab.Labs.NIC.cz/knot/knot-resolver/issues/549>
Comment 1 Vladimír Čunát 2020-03-05 12:42:56 UTC
Trust anchors (say root.keys): there are two main approaches, with this file being either read-only or read-write.  The error you were getting (before this patch) was due to the default appearing to be a mix between the two.  Docs: https://knot-resolver.readthedocs.io/en/stable/build.html#trust-anchors

If you update the package at least once a year (and thus the root.keys), I'd say the read-only way is more practical, so you may want to re-consider this.


I see the lua dependencies can be tweaked more after 5.0.0 (from NEWS):
- lua: remove dependency on lua-socket and lua-sec, used lua-http and cqueues (#512, #521, !894)

(Note: my FreeBSD/ports knowledge is minimial.)
Comment 2 Vladimír Čunát 2020-03-05 12:52:18 UTC
Oh and nitpick: the second link in files/pkg-message.in has been broken for a few months, but it's not unclear to me what kind of information you want in "To run as daemon".  Perhaps [config-overview]?  But given the previous link it doesn't seem very useful to add this one.

[config-overview] https://knot-resolver.readthedocs.io/en/stable/config-overview.html

BTW, our documentation URLs can be conditioned by version, e.g. using /v5.0.1/ instead of /stable/, in case that's supported and desirable in pkgs-message.in
Comment 3 yds 2020-03-05 14:00:50 UTC
(In reply to Vladimír Čunát from comment #2)

Vladimír, first of all, thank you so much for fixing the "cache space pre-allocation" bug so quick and taking your time to look over this patch.
  
probably best to remove pkg-message at this point if the links are stale. there's nothing very useful there considering that with the new rc scripts from this patch, to run as daemon, all that needs to be done is setting /etc/rc.conf:

    kresd_enable="YES"
    krescachegc_enable="YES"

FWIW, I'm already running kresd with this patch, it works but (as the first comment points out) the Lua dependencies have to be revised for full functionality. Some of the new Lua dependencies need to be ported to FreeBSD first. 

Re: managed TA: This patch satisfies this requirement from the docs:

> In case you want to have automatically managed DNSSEC trust anchors
> instead, set -Dmanaged_ta=enabled and make sure both keyfile_default
> file and its parent directories are writable by kresd process
> (after package installation!).

if the end user wants to disable managed_ta it's much easier and more secure to simply make root.keys not writable by the kresd user in the RUNDIR rather than requiring the end user to make ETCDIR writable by the kresd user. It also looks better to not be throwing errors upon startup, IMHO.

FWIW, the Unbound port updates its TA via rc script upon startup as well.
Comment 4 Vladimír Čunát 2020-03-05 14:40:21 UTC
> probably best to remove pkg-message at this point if the links are stale.

The other two links should be very stable, but I don't know FreeBSD conventions.
Comment 5 Leo Vandewoestijne 2020-07-01 13:31:54 UTC
Created attachment 216109 [details]
knot-resolver

Took a very long time, but I finally have a patch that (hopefully) solves everything addressed here and more.

1. Thanks for the rc.d scripts! I did minor adjustments to them.
Only "problem" I still have is being not able to set user/group, but it can be done in kresd.conf

2. The dns/knot2-lib port is desired by those who wish to have only knot-resolver and not knot2.
Consequence of that is that you need to install knot2 before knot-resolver in case you wish to have both.

5. The `kresd.conf` overwrite problem should be solved using @sample (I hope).

6. I have not been able to replicate that exact same error, but clearly still needs attention. Problem I currently have in getting root.keys is because of missing lua modules.

7. Solved per PR 246578

- Before I remove lua-socket and lua-sec I still encountered errors with cqueues
So I've also switched to `luajit` instead of `luajit-openresty` hoping it would maybe help (still needs testing).
In ports/dnsdist we switched from luajit luajit-openresty because it was told to be faster, so maybe it better should remain unchanged?

- Links in pkg-messages are removed/updated.

- The `-Dmanaged_ta=enabled` issue got it's own knob also, hope it allows you to run as desired - but to me looks not OK yet.

Further also here I removed python as a requirement; I get 2 GB of software just to never ever regenerate a few tiny docs. Seems to be fine without.

I'm using this patch myself, but please let me know what you think of it.
Comment 6 Leo Vandewoestijne 2020-07-01 14:59:38 UTC
command_args in kresd.in is missing -q argument
Comment 7 Vladimír Čunát 2020-07-01 15:23:25 UTC
What kind of cqueues problems are you getting?  I don't think the root of the issue will be in using vanilla luajit - I haven't such an issue with kresd so far.  (I hope I'm not getting too off-topic here.)
Comment 8 Leo Vandewoestijne 2020-07-02 00:40:55 UTC
The cqueues error I encounter when starting up without root.keys is

Jul  2 00:22:16 loc daemon[70929]: [ ta ] keyfile '/usr/local/etc/knot-resolver/root.keys': doesn't exist, bootstrapping
Jul  2 00:22:16 loc daemon[70929]: [system] error /usr/local/lib/knot-resolver/kluautil.lua:3: module 'cqueues.errno' not found:
Jul  2 00:22:16 loc daemon[70929]:      no field package.preload['cqueues.errno']
Jul  2 00:22:16 loc daemon[70929]:      no file '/usr/local/lib/knot-resolver/cqueues/errno.lua'
Jul  2 00:22:16 loc daemon[70929]:      no file '/usr/local/lib/knot-resolver/cqueues/errno/init.lua'
Jul  2 00:22:16 loc daemon[70929]:      no file './cqueues/errno.lua'
Jul  2 00:22:16 loc daemon[70929]:      no file '/usr/local/share/luajit-2.1.0-beta3/cqueues/errno.lua'
Jul  2 00:22:16 loc daemon[70929]:      no file '/usr/local/share/lua/5.1/cqueues/errno.lua'
Jul  2 00:22:16 loc daemon[70929]:      no file '/usr/local/share/lua/5.1/cqueues/errno/init.lua'
Jul  2 00:22:16 loc daemon[70929]:      no file '/usr/local/lib/knot-resolver/cqueues/errno.so'
Jul  2 00:22:16 loc daemon[70929]:      no file './cqueues/errno.so'
Jul  2 00:22:16 loc daemon[70929]:      no file '/usr/local/lib/lua/5.1/cqueues/errno.so'
Jul  2 00:22:16 loc daemon[70929]:      no file '/usr/local/lib/lua/5.1/loadall.so'
Jul  2 00:22:16 loc daemon[70929]:      no file '/usr/local/lib/knot-resolver/cqueues.so'
Jul  2 00:22:16 loc daemon[70929]:      no file './cqueues.so'
Jul  2 00:22:16 loc daemon[70929]:      no file '/usr/local/lib/lua/5.1/cqueues.so'
Jul  2 00:22:16 loc daemon[70929]:      no file '/usr/local/lib/lua/5.1/loadall.so'

When defining -Dconfig_tests=enabled it's basically that same problem.

So for now I've added:
TESTCONF_BROKEN=		Lua cqueues package seems missing


When defining -Dextra_tests=enabled I end up with:

CMake Error: The source directory "/wrkdirs/usr/ports/dns/knot-resolver/work/knot-resolver-5.1.2" does not appear to contain CMakeLists.txt.

So for now I've added:
TESTEXTRA_BROKEN=		CMake Error: The source directory misses CMakeLists.txt

I cannot find CMakeLists.txt in the working dir, and cannot find any filename containing "cqueues" on the entire filesystem.
Comment 9 Leo Vandewoestijne 2020-07-02 00:44:25 UTC
Comment on attachment 216109 [details]
knot-resolver

Overcome by event: 5.1.2 was released, see PR 247699
This PR can be closed.
Comment 10 Vladimír Čunát 2020-07-02 09:53:56 UTC
Well, lua-cqueues package is missing (or how you'd call it), and I didn't even find it packaged in ports.  Switching to different luajit implementation won't help there.  Some kresd features need such extra dependencies, e.g. bootstrapping initial root keys.
Comment 11 Kurt Jaeger freebsd_committer freebsd_triage 2020-07-11 20:58:44 UTC
Fixed with update to 5.1.2 in https://svnweb.freebsd.org/changeset/ports/542054