Bug 278395 - security/krb5: KDC has some issues if its LDAP servers are down
Summary: security/krb5: KDC has some issues if its LDAP servers are down
Status: In Progress
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Cy Schubert
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-04-16 21:43 UTC by Lexi Winter
Modified: 2024-04-29 12:51 UTC (History)
0 users

See Also:
bugzilla: maintainer-feedback? (cy)


Attachments
POC patch (565 bytes, patch)
2024-04-18 18:04 UTC, Cy Schubert
no flags Details | Diff
Try 2 (568 bytes, patch)
2024-04-18 18:07 UTC, Cy Schubert
no flags Details | Diff
Src patch. (901 bytes, patch)
2024-04-18 18:49 UTC, Cy Schubert
no flags Details | Diff
This should work. (948 bytes, patch)
2024-04-18 18:51 UTC, Cy Schubert
no flags Details | Diff
Commit yet to be pushed (1.50 KB, patch)
2024-04-18 19:01 UTC, Cy Schubert
no flags Details | Diff
New, improved (2.59 KB, patch)
2024-04-22 02:55 UTC, Cy Schubert
no flags Details | Diff
Simplified (2.54 KB, patch)
2024-04-22 03:15 UTC, Cy Schubert
no flags Details | Diff
Final patch (2.54 KB, patch)
2024-04-22 03:23 UTC, Cy Schubert
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Lexi Winter 2024-04-16 21:43:12 UTC
i might work up a patch for this but i thought i'd submit the bug first in case you have any ideas.

the problem is this: if KDC is configured with an LDAP backend, and it can't connect to its LDAP server(s) on startup, it will exit and never restart -- which makes it awkward to run if e.g. slapd and kdc are in different jails.

this also makes me concerned that, if it's running and all its LDAP servers go down, it might exit and never restart.

so i'm wondering if the rc.d script should start kdc using daemon(8) with -r, to ensure it's restarted if it exits.  but... it seems like security/krb5 doesn't provide its own init script, it relies on /etc/rc.d/kdc, so this might be more of a src issue.
Comment 1 Cy Schubert freebsd_committer freebsd_triage 2024-04-16 22:27:54 UTC
We can try daemon -r but my testing with this has been less satisfactory so far.

And don't forget if using daemon -r kdc must be started using -n. Best might be to create a kdc daemon that uses the same syntax as the Heimdal kdc that will ensure the kdc remains running. It could take an argument to test for a remote listening LDAP socket before krb5kdc starts. This is the most elegant solution.

To answer the question of where to put the source for this daemon? In security/krb5/files or in its own port, called krb5-ldap-glue or some other name.

I think mucking around with libexec/rc/rc.d/kdc is a bad idea. Let's not complicate it for this application. I can worry about that once MIT has been imported to replace Heimdal. I want to avoid complicating the rc script in order to avoid PRs when it breaks.

Let's do a separate management daemon.

You know, another alternative (hack) is to put the krb5kdc startup in /etc/ttys, just like one would xdm. This will automatically restart the daemon. This is a bit of a hack though. I'd only recommend that for a one-off application.
Comment 2 Lexi Winter 2024-04-18 17:50:13 UTC
> We can try daemon -r but my testing with this has been less satisfactory so far.

out of interest, what was the problem you ran into?

i'm fine with fixing this by shipping a 'kdc_daemon' script or something, although i would suggest there's no need to put it in a separate package.
Comment 3 Cy Schubert freebsd_committer freebsd_triage 2024-04-18 17:53:49 UTC
There is already a kdc daemon script in the security/krb5* ports.
Comment 4 Lexi Winter 2024-04-18 17:59:09 UTC
what's it called?  i'm looking at 'pkg info -l krb5' and i can't see it.
Comment 5 Cy Schubert freebsd_committer freebsd_triage 2024-04-18 18:04:51 UTC
Created attachment 250061 [details]
POC patch

Here's a patch do restart krb5kdc.

I haven't tested this. Give it a spin and I'll apply it to the other ports too.
Comment 6 Cy Schubert freebsd_committer freebsd_triage 2024-04-18 18:07:05 UTC
Created attachment 250062 [details]
Try 2

Haste makes waste. krb5kdc must have -n flag too.

Putting out fire at $JOB. Let me know if this fixes it for you.
Comment 7 Lexi Winter 2024-04-18 18:21:01 UTC
i can't take the KDC down during business hours but i'll test this tonight - thanks.
Comment 8 Cy Schubert freebsd_committer freebsd_triage 2024-04-18 18:49:23 UTC
Created attachment 250064 [details]
Src patch.

Now I recall. There was another PR that suggested the kdc script was insufficient. I updated the kdc rc script to handle both Heimdal and MIT. This adds a restart variable. Try this instead.

Again, this hasn't been tested. I'll test this myself tonight.
Comment 9 Cy Schubert freebsd_committer freebsd_triage 2024-04-18 18:51:37 UTC
Created attachment 250065 [details]
This should work.

This should work.
Comment 10 Cy Schubert freebsd_committer freebsd_triage 2024-04-18 19:01:34 UTC
Created attachment 250066 [details]
Commit yet to be pushed

This is the path forward. I'll commit this after your testing. (Works here.)
Comment 11 Lexi Winter 2024-04-18 19:03:56 UTC
okay, just to be clear the most recent patch (250066) is the only one i need to apply, right?
Comment 12 Cy Schubert freebsd_committer freebsd_triage 2024-04-18 19:29:17 UTC
(In reply to Lexi Winter from comment #11)

Correct. I obsoleted the others.
Comment 13 Lexi Winter 2024-04-20 18:11:39 UTC
tested this and it seems to work (sorry for the delay):

- start and stop still works normally when LDAP is running
- start fails when kdc_restart isn't set and LDAP is down
- restarting works correctly when LDAP is down, and the KDC starts when LDAP comes up

it does log a lot of console messages (about once per second):

# service kdc start
Starting kdc.
# krb5kdc: cannot initialize realm LE-FAY.ORG - see log file for details
krb5kdc: cannot initialize realm LE-FAY.ORG - see log file for details
krb5kdc: cannot initialize realm LE-FAY.ORG - see log file for details
krb5kdc: cannot initialize realm LE-FAY.ORG - see log file for details
krb5kdc: cannot initialize realm LE-FAY.ORG - see log file for details
krb5kdc: cannot initialize realm LE-FAY.ORG - see log file for details
krb5kdc: cannot initialize realm LE-FAY.ORG - see log file for details
krb5kdc: starting...
Comment 14 Cy Schubert freebsd_committer freebsd_triage 2024-04-21 02:35:10 UTC
(In reply to Lexi Winter from comment #13)
> tested this and it seems to work (sorry for the delay):
> 
> - start and stop still works normally when LDAP is running
> - start fails when kdc_restart isn't set and LDAP is down

daemon(8) managing krb5kdc startup doesn't magically start the LDAP server.

> - restarting works correctly when LDAP is down, and the KDC starts when LDAP comes up
> 
> it does log a lot of console messages (about once per second):
> 
> # service kdc start
> Starting kdc.
> # krb5kdc: cannot initialize realm LE-FAY.ORG - see log file for details
> krb5kdc: cannot initialize realm LE-FAY.ORG - see log file for details
> krb5kdc: cannot initialize realm LE-FAY.ORG - see log file for details
> krb5kdc: cannot initialize realm LE-FAY.ORG - see log file for details
> krb5kdc: cannot initialize realm LE-FAY.ORG - see log file for details
> krb5kdc: cannot initialize realm LE-FAY.ORG - see log file for details
> krb5kdc: cannot initialize realm LE-FAY.ORG - see log file for details
> krb5kdc: starting...

This is because it is restarting krb5kdc immediately after it terminates because it cannot connect to your LDAP server. Try starting your LDAP server and the messages should stop.

Maybe what you need is an MIT krb5kdc startup script that is configured to start after openldap26-server starts. If this is what you want, the attached patch is totally wrong for the application. Is this what you're looking for? MIT krb5 port may need multiple startup scripts, each with different dependencies, or a helper app that tests for an open LDAP port before it allows krb5kdc to start.

But what you describe, daemon(8) is working properly.
Comment 15 Lexi Winter 2024-04-21 15:14:45 UTC
sorry, in case my comment wasn't clear: everything is working fine and expected with this patch, so this fixes the issue for me.
Comment 16 Cy Schubert freebsd_committer freebsd_triage 2024-04-22 02:55:09 UTC
Created attachment 250145 [details]
New, improved

Excellent. Thanks for testing.

The improved patch adds default to defaults/rc.conf. It also allows tuning of the restart delay (-R). I will submit this for phabricator review.
Comment 17 Cy Schubert freebsd_committer freebsd_triage 2024-04-22 03:15:27 UTC
Created attachment 250146 [details]
Simplified

This is the one I will submit for review following local testing this week.
Comment 18 Cy Schubert freebsd_committer freebsd_triage 2024-04-22 03:23:15 UTC
Created attachment 250147 [details]
Final patch

This fixes an unquoted string error. Not sure how I missed that.
Comment 19 Cy Schubert freebsd_committer freebsd_triage 2024-04-22 03:35:00 UTC
Obsoleted patch. Latest patch can be found at https://reviews.freebsd.org/D44898.
Comment 20 commit-hook freebsd_committer freebsd_triage 2024-04-24 05:55:29 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=9e248b7f645cdf05615a49cc15295df49b54d659

commit 9e248b7f645cdf05615a49cc15295df49b54d659
Author:     Cy Schubert <cy@FreeBSD.org>
AuthorDate: 2024-04-18 18:56:54 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2024-04-24 05:50:38 +0000

    kdc: Add restart option

    Add a new kdc_restart rc variable that manages kdc (or krb5kdc) under
    daemon(8). This automatically restarts the kdc should it fail, i.e.
    when it's configured to use LDAP as a backend and cannot connect to its
    LDAP directory.

    Set kdc_restart="YES" to auto restart kdc on abnormal termination.

    Set kdc_restart_delay="N" to the number of seconds to delay before
    restarting the kdc. The daemon(8) default seconds applies when not set.

    Reported by:            Lexi Winter <lexi.freebsd@le-fay.org>
    PR:                     278395
    Differential Revision:  https://reviews.freebsd.org/D44898

 libexec/rc/rc.conf  |  2 ++
 libexec/rc/rc.d/kdc | 29 +++++++++++++++++++++++++++--
 2 files changed, 29 insertions(+), 2 deletions(-)
Comment 21 Cy Schubert freebsd_committer freebsd_triage 2024-04-24 06:10:33 UTC
I've put a calendar entry into plan(1) to MFC this on Monday April 29 at 0800U.
Comment 22 commit-hook freebsd_committer freebsd_triage 2024-04-29 12:51:50 UTC
A commit in branch stable/14 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=abc4b3088941d772728c28146d466010f816ea66

commit abc4b3088941d772728c28146d466010f816ea66
Author:     Cy Schubert <cy@FreeBSD.org>
AuthorDate: 2024-04-18 18:56:54 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2024-04-29 12:50:19 +0000

    kdc: Add restart option

    Add a new kdc_restart rc variable that manages kdc (or krb5kdc) under
    daemon(8). This automatically restarts the kdc should it fail, i.e.
    when it's configured to use LDAP as a backend and cannot connect to its
    LDAP directory.

    Set kdc_restart="YES" to auto restart kdc on abnormal termination.

    Set kdc_restart_delay="N" to the number of seconds to delay before
    restarting the kdc. The daemon(8) default seconds applies when not set.

    Reported by:            Lexi Winter <lexi.freebsd@le-fay.org>
    PR:                     278395
    Differential Revision:  https://reviews.freebsd.org/D44898

    (cherry picked from commit 9e248b7f645cdf05615a49cc15295df49b54d659)

 libexec/rc/rc.conf  |  2 ++
 libexec/rc/rc.d/kdc | 29 +++++++++++++++++++++++++++--
 2 files changed, 29 insertions(+), 2 deletions(-)
Comment 23 commit-hook freebsd_committer freebsd_triage 2024-04-29 12:51:56 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=e750111ced0370a90f585467a79a773d20f14360

commit e750111ced0370a90f585467a79a773d20f14360
Author:     Cy Schubert <cy@FreeBSD.org>
AuthorDate: 2024-04-18 18:56:54 +0000
Commit:     Cy Schubert <cy@FreeBSD.org>
CommitDate: 2024-04-29 12:51:11 +0000

    kdc: Add restart option

    Add a new kdc_restart rc variable that manages kdc (or krb5kdc) under
    daemon(8). This automatically restarts the kdc should it fail, i.e.
    when it's configured to use LDAP as a backend and cannot connect to its
    LDAP directory.

    Set kdc_restart="YES" to auto restart kdc on abnormal termination.

    Set kdc_restart_delay="N" to the number of seconds to delay before
    restarting the kdc. The daemon(8) default seconds applies when not set.

    Reported by:            Lexi Winter <lexi.freebsd@le-fay.org>
    PR:                     278395
    Differential Revision:  https://reviews.freebsd.org/D44898

    (cherry picked from commit 9e248b7f645cdf05615a49cc15295df49b54d659)

 libexec/rc/rc.conf  |  2 ++
 libexec/rc/rc.d/kdc | 29 +++++++++++++++++++++++++++--
 2 files changed, 29 insertions(+), 2 deletions(-)