Bug 241347 - security/sssd: Update to 1.16.4
Summary: security/sssd: Update to 1.16.4
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-ports-bugs (Nobody)
URL:
Keywords:
Depends on:
Blocks: 231846 240708
  Show dependency treegraph
 
Reported: 2019-10-19 22:00 UTC by lukas.slebodnik
Modified: 2020-06-17 08:04 UTC (History)
14 users (show)

See Also:


Attachments
Update to 1.16.4 (84.01 KB, patch)
2019-10-19 22:00 UTC, lukas.slebodnik
no flags Details | Diff
sssd_binalias_python3.patch (553 bytes, patch)
2020-01-20 15:23 UTC, Phillip R. Jaenke
no flags Details | Diff
Add krb5 v1.18 support (1001 bytes, patch)
2020-03-17 13:38 UTC, Rick
no flags Details | Diff
SSSD 1.16.4 Patch (81.64 KB, patch)
2020-03-30 16:29 UTC, Rick
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description lukas.slebodnik 2019-10-19 22:00:39 UTC
Created attachment 208428 [details]
Update to 1.16.4

sssd-1.13 branch should have been LTM in upstream but latest version 1.13.4 is 2.5 years old.

sssd-1.16 branch is more suitable for the update.
Comment 1 Rick 2019-10-20 07:23:06 UTC
Reconsider defining an explicit Samba dependency. This factored into LDB conflicts between security/sssd and net/samba4* in the first place. Ports defines behavior for selecting the Samba version by setting a default that can be over-ridden via DEFAULT_VERSIONS, which puts greater control in the users hands.

Explicit dependency mitigates current behavior, but when net/samba411 is added to ports and users want to install 4.11 as opposed to 4.10, it requires overriding SMB_LIB_DEPENDS instead of using DEFAULT_VERSIONS like "DEFAULT_VERSIONS=samba=4.11".

Addressing the current failing behavior is achievable through a note in Makefile or UPDATING explaining net/samba410 to be the minimum Samba version required for SMB support. So, users know to deploy a configuration similar to that described in bug #238465.

Also, the SSSD project is shipping version 2.2. Are there compelling reasons for not updating to the project's most recent version, or at least adding it as a new port, security/sssd2 for example?
Comment 2 Rick 2019-10-20 08:18:32 UTC
Attached patch appears to fail citing the following. The config.log is at https://pastebin.com/Bc5ydiPH


checking for python2... no
checking for python3... no
configure: error: 
The program python3 was not found in search path.
Please ensure that it is installed and its directory is included in the search
path. It is required for building python3 bindings. If you do not want to build
them please use argument --without-python3-bindings when running configure.
===>  Script "configure" failed unexpectedly.
Please report the problem to lukas.slebodnik@intrak.sk [maintainer] and attach
the "/wrkdirs/usr/ports/security/sssd/work/sssd-1.16.4/config.log" including
the output of the failure of your make command. Also, it might be a good idea
to provide an overview of all packages installed on your system (e.g. a
/usr/local/sbin/pkg-static info -g -Ea).
*** Error code 1

Stop.
make: stopped in /usr/ports/security/sssd
=>> Cleaning up wrkdir
===>  Cleaning for sssd-1.16.4_1
build of security/sssd | sssd-1.16.4_1 ended at Sun Oct 20 04:06:08 EDT 2019
build time: 00:01:03
!!! build failure encountered !!!
Comment 3 Phillip R. Jaenke 2019-10-20 14:14:59 UTC
Rick, not speaking for Lukas here obviously, but I am speaking as someone very familiar with sssd. The "latest" is often "too latest." Frequently does not do what it says on the tin, at best. It's focused on feature addition and the like. Hence, why they have LTMs. The LTMs are tied to RHEL.
So from the FreeBSD side, the port should track what sssd version is in the current mainstream release of Red Hat. For 7.7, that's 1.16. I'm a large RHEL customer at $dayjob and the in-house sssd expert there, so I'm reasonably familiar with this. 

All that said, Lukas, can this build with python3.6+? FreeBSD is EOL'ing 2.7 much more aggressively than RH. So I would recommend building only with 3.x if possible so it doesn't come up as broken in January.

The other concern I have is around the security/krb5 and samba dependency. We don't have a good way to enforce option dependencies in other ports. I think this can be worked around by depending on ${LOCALBASE}/lib/shared-modules/krb5/winbind_krb5_localauth.so and ${LOCALBASE}/lib/samba4/krb5/plugins/kdb/samba.so which are only present when GSSAPI_MIT is selected in samba48+. That SHOULD prevent user foot-shooting by installing a GSSAPI_BUILTIN samba48+ against sssd here.
Comment 4 lukas.slebodnik 2019-10-21 10:12:43 UTC
(In reply to Rick from comment #1)
> Also, the SSSD project is shipping version 2.2. Are there compelling reasons for not updating to the project's most recent version, or at least adding it as a new port, security/sssd2 for example?

You would need to follow upstream closer to know the context.

There is not any sssd-2.0 or sssd-2.1 branch and thus they cannot bachport fixes there. And there were 2 regressions quite serious regressions between 2.2.0 and 2.2.2 

Upstream promised to make sssd-1.13 LTS version but reality is different.
CVE-2019-3811 (https://pagure.io/SSSD/sssd/issue/3901) was fixed just in master and sssd-1-16 branch. Sure, we could backport patches ourselves. but it is more complicated to backport patches from sssd-1.16 branch to sssd-1.13.

The sssd-1-13 branch had the latest commit 14 months ago and sssd-1-16 11 days ago.

I am not sure whether adding security/sssd2 make a sense.
I would rather wait till sssd-2.x stabilize on Linux and then move from sssd-1.16 to sssd-2.x. So far sssd-1-16 is still in "active" state in usptream.
Comment 5 lukas.slebodnik 2019-10-21 10:18:22 UTC
(In reply to Phillip R. Jaenke from comment #3)
> All that said, Lukas, can this build with python3.6+? FreeBSD is EOL'ing 2.7 much more aggressively than RH. So I would recommend building only with 3.x if possible so it doesn't come up as broken in January.

My patch enforce python3 and disable python2 by default.
Upstream tests python3.4+ so python3.6 should be fine

I would probably need to update port to not rely on pyhton3 but directly use python3.6. Is that what you want?

> The other concern I have is around the security/krb5 and samba dependency. We don't have a good way to enforce option dependencies in other ports. I think this can be worked around by depending on ${LOCALBASE}/lib/shared-modules/krb5/winbind_krb5_localauth.so and ${LOCALBASE}/lib/samba4/krb5/plugins/kdb/samba.so which are only present when GSSAPI_MIT is selected in samba48+. That SHOULD prevent user foot-shooting by installing a GSSAPI_BUILTIN samba48+ against sssd here.

Do I understand that you worry about mixing MIT krb5 and heimdal when building with samba ?
Comment 6 lukas.slebodnik 2019-10-21 10:29:13 UTC
(In reply to Rick from comment #1)
> Reconsider defining an explicit Samba dependency. This factored into LDB conflicts between security/sssd and net/samba4* in the first place. Ports defines behavior for selecting the Samba version by setting a default that can be over-ridden via DEFAULT_VERSIONS, which puts greater control in the users hands.

Situation with LDB is different samba is more picky about version of ldb.
sssd can work without recompilation with ldb 1.2, 1.3, 1.4, 1.5
The only exception is ldb-1.1.30 due to ABI breakage in ldb.


> Explicit dependency mitigates current behavior, but when net/samba411 is added to ports and users want to install 4.11 as opposed to 4.10, it requires overriding SMB_LIB_DEPENDS instead of using DEFAULT_VERSIONS like "DEFAULT_VERSIONS=samba=4.11".

The tricky part is winbind_idmap_sss.so The might change ABI between versions
and thus you need to recompile the plugin and to "downgrade" version with --with-smb-idmap-interface-version=6

Sure that module needn't be used by anyone and ipa and ad provider would work.
But it is dangeroups therefore I decided to stick with samna-4.10


> Addressing the current failing behavior is achievable through a note in Makefile or UPDATING explaining net/samba410 to be the minimum Samba version required for SMB support. So, users know to deploy a configuration similar to that described in bug #238465.


which changes do you suggest after my explanation?
Comment 7 Rick 2019-10-21 17:17:47 UTC
Thanks for explaining 1.16 vs 2.x. It makes sense to track the same version being deployed in Red Hat.

In considering explicit Samba dependencies, my preference remains in favor of putting control in the users hands unless there is material information I am missing or misunderstanding. Explicit dependency works for users pulling in Samba as a direct consequence of sssd. However, users pulling in Samba as a consequence of sssd and one or more other ports may experience conflicts and/or failed builds.

Defining explicit dependencies also implies more effort to ensure compatibility. In other words, ports defining explicit dependencies must stay vigilant as Samba versions are deprecated. SSSD fell victim to this as Samba 4.[678] have and are reaching EOL as proven by its failure to build w/ SMB from July 2019 on after Samba 4.[67] were removed and thus defaults to 4.8.

Comments to Makefile or UPDATING explaining and recommending defining default Samba to 4.10 when SMB is enabled are appropriate w/ the addition that changing the default may result in undesirable behavior as a consequence of differing ABIs with winbind_idmap_sss.so. This is irrelevant to me as each repository stands on it’s own.

Having said all that, sssd w/ the attached patch for 1.16.4 continues to fail during configure citing the error above. It occurs even despite defining PYTHON_VERSION and/or PYTHON_CMD in make.conf for either Python 2.7 or 3.6 like:

.if ${.CURDIR:M*/security/sssd}
PYTHON_VERSION= 3.6
PYTHON_CMD=/usr/local/bin/python3.6
OPTIONS_FILE_SET+=SMB  
.endif
Comment 8 lukas.slebodnik 2019-10-21 21:20:49 UTC
(In reply to Rick from comment #7)
> Comments to Makefile or UPDATING explaining and recommending defining default
> Samba to 4.10 when SMB is enabled are appropriate w/ the addition that changing
> the default may result in undesirable behavior as a consequence of differing
> ABIs with winbind_idmap_sss.so. This is irrelevant to me as each repository
> stands on it’s own.

That's not really true.
winbind_idmap_sss.so is plugin for winbind to use idmapping of SID -> (UIG/GID) done by sssd. (samba and winbind use different algorithms)
And as I already mentioned the ABI for plugins is not very stable.
Therefore I decided to stick to samba 4.10

Sure the best would be more flexible and be able to build with 4.10 and 4.11
(cause one is in maintenance mode and another in stable release)
https://wiki.samba.org/index.php/Release_Planning_for_Samba_4.11

But i would like to enforce strict runtime dependencies between samba and sssd.
Use the same version as were used at build time. And because of this I decided to stick with samba 4.10 

Could you point me to other ports which have similar requirements
about build-time and run-time dependencies for different ports?

> Having said all that, sssd w/ the attached patch for 1.16.4 continues to fail
> during configure citing the error above. It occurs even despite defining
> PYTHON_VERSION and/or PYTHON_CMD in make.conf for either Python 2.7 or 3.6 like:
>
> .if ${.CURDIR:M*/security/sssd}
> PYTHON_VERSION= 3.6
> PYTHON_CMD=/usr/local/bin/python3.6
> OPTIONS_FILE_SET+=SMB  
> .endif

That is a different bug which will be addressed.

But I would like get an agreement about samba.
Comment 9 Rick 2019-10-22 00:06:04 UTC
(In reply to lukas.slebodnik from comment #8)

Ok. I understand. Fortunately, it is a workable config.
Comment 10 Phillip R. Jaenke 2019-10-23 18:28:32 UTC
> I would probably need to update port to not rely on pyhton3 but directly use python3.6. Is that what you want?

Definitely not; I just saw the python2 and was worried it might be attempting to use that. ports should still have 3.4+ for quite some time, so the flavors method (and SHEBANGFIX) that Rick mentioned would be the best way there.

> Do I understand that you worry about mixing MIT krb5 and heimdal when building with samba ?

Yes, this is a known issue we ran into back on 1.13 when it was working cleanly if you don't recall. By default, FreeBSD samba4x uses the built-in Heimdal due to AD_DC being a default option. So a user taking samba48 or samba410 from pkg.freebsd.org and building security/sssd locally will end up with a broken sssd. samba48+ now has the option for MIT due upstream updates, but it's a non-default option on FreeBSD. So some additional care needs to be taken in the context of FreeBSD. Basically the security/sssd port should tell the user that the MIT option is required on their samba48 or samba410.

> The tricky part is winbind_idmap_sss.so The might change ABI between versions
and thus you need to recompile the plugin and to "downgrade" version with --with-smb-idmap-interface-version=6

> Sure that module needn't be used by anyone and ipa and ad provider would work.
But it is dangeroups therefore I decided to stick with samna-4.10

I would argue that it is a LOT more dangerous to NOT have the AD provider. I don't think the IPA provider is going to work correctly anyways (but it has been a while and I don't have a test environment handy.) But the AD provider is basically the number one use case on FreeBSD. So not having it working is going to be a LOT worse than people having Samba without MIT kerberos.

The Samba API changes absolutely make it a challenge and a half though, ESPECIALLY because net/samba410 is Schroedinger's Port: both broken and working. And both with built-in ldb and ldb1.5 (I think, I'd have to look again.) And in a lot of flux because AD_DC provisioning is broken plus folks attempting to fix it on ZFS again. That really throws a wrench into the works. 

Rick, do you know of an approved or at least not-horrible way to do CONFIG+=--with-smb-idmap-interface-version=6 based on the DEFAULT_VERSIONS value for samba?
Comment 11 Rick 2019-10-23 21:07:26 UTC
(In reply to Phillip R. Jaenke from comment #10)

> do you know of an approved or at least not-horrible way to do
> CONFIG+=--with-smb-idmap-interface-version=6 based on the
> DEFAULT_VERSIONS value for samba?

I pondered this, but didn't have an answer. The challenge being that there already is a default samba and the question is 'how does one tell if the version defined is implicit vs user defined?'.
Comment 12 Rick 2019-11-07 12:36:54 UTC
Lukas, is there an updated patch that needs testing? Updating the port is important to mature its security posture.
Comment 13 lukas.slebodnik 2019-11-07 20:47:45 UTC
Sorry,
I have a longer holiday ATM. I should be able to find time for update at weekend (or maybe on Monday) I am sorry for delay.
Comment 14 Phillip R. Jaenke 2019-11-08 06:47:56 UTC
No worries on that, we all have lives.

Since SSSD has *always* been a "special handling required" port, I think the only real question here is whether to ship as-is (which means default builds of dependencies will leave it non-functioning) or update samba410 to use MIT default (that's a timur@ call and as MIT is experimental probably a bad idea.)

Considering all those moving pieces, I think we've re-exposed a shortcoming in ports infrastructure that isn't going to be resolved today - or this quarter, and probably not next year. Once building clean is confirmed and function is confirmed when dependencies are built to suit, commit it as is. Then exclude it from official pkgbuilds (maybe RESTRICTED?) so that users have to engage their brains and build everything the right way.

Once we have the key portions here, I'm happy to take up the torch of writing a pkg message explaining to users what they need to do to produce a working setup.
Comment 15 Phillip R. Jaenke 2019-12-05 19:04:23 UTC
Lukas, do you have any update here? I would like to get this into my testing infrastructure ASAP. Thanks!
Comment 16 Rick 2019-12-11 12:08:50 UTC
Hi Lukas, I too am anxious to get this in testing infrastructure. Are there additional patch(es) to test? Thanks.
Comment 17 Phillip R. Jaenke 2020-01-20 15:23:37 UTC
Created attachment 210896 [details]
sssd_binalias_python3.patch

This is currently being blocked by #242077 (databases/ldb14 PLIST error) since November, and I have submitted a patch to resolve that issue. It was suggested to use a newer LDB and indicated that ldb14 will be expired, so, now it builds with ldb15. Because we still don't have anything newer than 1.5. C'est la vie.

The attached patch slaps a BINARY_ALIAS fix on the python3 issue in autoconf, which should be fine, since everything else is looking in the right place. It's just autoconf being autoconf. Note that this goes on top of Lukas' patch, not separately.

Since we haven't heard any updates in quite some time now, I'm inclined to suggest we go ahead and commit the combined patches in hopes of getting broader testing, and do some additional cleanup based on results. I have not had time to do ANY proper testing of functionality yet due to Samba also being broken. We need people to get testing whether or not this functions as intended at this point.

------------
The following make.conf (or equivalent) settings are REQUIRED to generate a fully functional SSSD for both AD and LDAP(S) environments. If you use defaults for the dependencies, it should NOT be expected to work in reasonable environments through no fault of it's own. (OpenLDAP does not have SASL by default, see D21855.)

## make.conf snippet
# DEFAULT_VERSIONS must be set exactly this way! openssl can be base or ports.
DEFAULT_VERSIONS+=perl5=5.30
DEFAULT_VERSIONS+=python3=3.6    # 3.6 is minimum, not maximum - dep safety
DEFAULT_VERSIONS+=samba=4.10

# security/sssd options for maximum testing
security_sssd_SET+=SMB

# Do not rely on these, not all ports obey or use these names.
OPTIONS_SET+=GSSAPI_MIT
OPTIONS_UNSET+=GSSAPI_BASE GSSAPI_HEIMDAL
# openldap is not SASL by default; AD requires SASL
WANT_OPENLDAP_SASL=yes
OPTIONS_SET+=WANT_OPENLDAP_SASL

# required for DNS registration to work 
dns_bind-tools_SET+=GSSAPI_MIT
dns_bind-tools_UNSET+=GSSAPI_BASE GSSAPI_HEIMDAL GSSAPI_NONE

# net/samba410 has very specific settings required
# mDNS/ZeroConf can be anything but AVAHI works best for now
# XXX: NEVER mix GSSAPI_MIT and AD_DC, you will have a BAD TIME.
net_samba410_set+=ADS GSSAPI_MIT NSUPDATE
net_samba410_unset+=AD_DC GSSAPI_BUILTIN BIND911 BIND914
Comment 18 Rick 2020-03-17 13:38:03 UTC
Created attachment 212458 [details]
Add krb5 v1.18 support

Include krb5 v1.18 support
Comment 19 Rick 2020-03-17 14:13:06 UTC
(In reply to Phillip R. Jaenke from comment #17)

My testing has proven successful. Successful in that sssd is handling authentication/authorization as expected.

Testing was done with the following make.conf settings after applying your BINARY_ALIAS patch. The port still failed to build until the attached krb5 patch was applied to add krb5 v1.18 support. The security/sssd ldb depend, as you indicated, must be 1.5.

WITH_GSSAPI=YES
WANT_OPENLDAP_SASL=YES
BUILD_ALL_PYTHON_FLAVORS=YES
DEFAULT_VERSIONS+= python=3.6 python3=3.6 samba=410

.if ${.CURDIR:M*/security/sssd*}
OPTIONS_FILE_SET+=SMB
.endif

.if ${.CURDIR:M*/net/samba4*}
OPTIONS_FILE_UNSET+=AD_DC QUOTAS
SAMBA4_BUNDLED_LDB=no
.endif

I'd like these patches committed also. It would prevent jumping through these hoops every time this port is built.
Comment 20 Rick 2020-03-30 16:29:36 UTC
Created attachment 212864 [details]
SSSD 1.16.4 Patch

While this patch successfully builds SSSD 1.16.4, it does not function as desired in my environment.
Comment 21 Rick 2020-03-30 16:30:16 UTC
The previous test I completed was flawed because it compiled SSSD 1.13, not 1.16.

So, I checked out a current ports tree and applied the three patches in this PR with the make.conf already described. Lukas' original patch seems to remove files/patch-src_external__pac__responder.m4 or I did something wrong. So, the third attached patch was manually added.

This fourth patch is intended to replace the previous three. With the 4th patch, SSSD 1.16.4 builds and packages, but it does not work as expected instead producing the following error with the same config file as deployed with 1.13:

[sssd] [sss_ini_call_validators] (0x0020): [rule/allowed_sections]: Section [domain/$domain] is not allowed. Check for typos.

I won't be moving to 1.16 until this is resolved and will continue to run 1.13 as opposed to v1.11 the current port is.
Comment 22 Kyle Evans freebsd_committer 2020-04-14 17:38:53 UTC
As a next step, can we please get an indication of which patches are obsolete (and should be marked thus) and which should be considered for maintainer (approval/timeout)?
Comment 23 the_mix_room 2020-05-14 08:19:01 UTC
I played around a little with this yesterday. Having missed this entry I initially tired to recreate it myself. 

The work generally looks good, and I only have some minor comments on all three patches, I think they seem quite similar and would benefit from being merged into one. 
 
* patch-src_external_pac__responder.m4 is renamed patch-src__external_pac__responder.m4, ( s/_/__/ which is correct - there was a _ missing previously) Inside the attachment 212864 [details] there are changes to which versions are included, previously 1.17* was included, now it is 1.17

* HOST_NAME_MAX is replaced by _POSIX_HOST_NAME_MAX in the code, instead of previously being defined in Makefile.am. Perhaps it would be easier to use an ifndef-block instead. I used sysconf(_SC_HOST_NAME_MAX) instead of the constant _POSIX_HOST_NAME_MAX. On my machine they are the same value, but I do not think there is guarantees for this. 

* CONFIGURE_ARGS in Makefile have changed order, this makes it quite difficult to follow what has been added or changed.

* 1.16.4 has been replaced by 1.16.5 in the meantime.

Other than that I will add myself to the list of people that would like to test this when there is a suitable patch.
Comment 24 Karli Sjöberg 2020-05-16 15:26:40 UTC
Hi everyone and thank you all so much for working on this!

I gotta say though, that right now sssd is just utterly broken.

First of all, the original port (1.11) only builds with 'SAMBA4_BUNDLED_LDB=YES' and fails to start for unknown reasons. The logs say very little about it, but it's got something to do with the version of ldb. Trying to switch up ldb from 14 to 15 in sssd's Makefile to match the ldb in samba410 makes it fail to build.

Second, the push to upgrade to sssd 1.13 (that allegedly works) fails to cleanly apply the patchset and therefore fails to build. The WIP to change it to 1.15 was never more than just that- a work in progress- and never really went anywhere.

Lastly, the patch to go to 1.16 also fails to apply and, of course, also fails to build because of it.

I would be more than happy to help with either one of these alternatives, I just want to be able to log in to my servers again!

Best Regards
/K