Bug 190055 - getgrouplist() unable to retrieve groups from nss "compat" source
Summary: getgrouplist() unable to retrieve groups from nss "compat" source
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: 10.0-RELEASE
Hardware: Any Any
: Normal Affects Many People
Assignee: Dag-Erling Smørgrav
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-05-21 09:30 UTC by Alex Mihajlov
Modified: 2024-01-10 12:16 UTC (History)
1 user (show)

See Also:


Attachments
Allow fallback methods to perform arbitrary lookups (2.21 KB, patch)
2014-11-14 03:51 UTC, Dag-Erling Smørgrav
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alex Mihajlov 2014-05-21 09:30:00 UTC
Hi all, 
Is anyone here using nss_ldap and can successfully get it to work with multiple group memberships? I would really like to get this to work here, but I only get the primary group.I get it to work only when I downgraded the port to r323088

root@lspm138:/home/amihailov # cd /usr/ports/net/nss_ldap/
root@lspm138:/usr/ports/net/nss_ldap # make install clean
Installing nss_ldap-1.265_10... done
root@lspm138:~/nss_ldap # getent group delphi
ldap_build_search_req ATTRS: cn userPassword memberUid uniqueMember gidNumber
=> ldap_bv2dn(cn=delphi,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=delphi,ou=groups,dc=lsoft,dc=ru)=0 
delphi:*:2004:addfs,alex,amangion,minlow,n0dwis,oliiiik,romastyi,rukavicyn,sgazin,valentin,vali,zinov,heman777
root@lspm138:~ # id himan777
ldap_build_search_req ATTRS: uid userPassword uidNumber gidNumber cn homeDirectory loginShell gecos description objectClass shadowLastChange shadowMax shadowExpire loginClass
ldap_build_search_req ATTRS:
ldap_build_search_req ATTRS: gidNumber
ldap_build_search_req ATTRS: gidNumber
ldap_build_search_req ATTRS: cn userPassword memberUid uniqueMember gidNumber
=> ldap_bv2dn(cn=repoupdater,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=repoupdater,ou=groups,dc=lsoft,dc=ru)=0 
ldap_build_search_req ATTRS: cn userPassword memberUid uniqueMember gidNumber
=> ldap_bv2dn(cn=repoupdater,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=repoupdater,ou=groups,dc=lsoft,dc=ru)=0 
ldap_build_search_req ATTRS: cn userPassword memberUid uniqueMember gidNumber
=> ldap_bv2dn(cn=repoupdater,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=repoupdater,ou=groups,dc=lsoft,dc=ru)=0 
uid=2020(himan777) gid=2011(repoupdater) groups=2011(repoupdater)
root@lspm138:~ # portdowngrade net/nss_ldap/ r323088
Checked out revision 323088.
root@lspm138:~/nss_ldap # make reinstall
Installing nss_ldap-1.265_7... done
root@lspm138:~ # id himan777
ldap_build_search_req ATTRS: uid userPassword uidNumber gidNumber cn homeDirectory loginShell gecos description objectClass shadowLastChange shadowMax shadowExpire loginClass
ldap_build_search_req ATTRS: cn userPassword memberUid uniqueMember gidNumber
=> ldap_bv2dn(cn=admins,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=admins,ou=groups,dc=lsoft,dc=ru)=0 
=> ldap_bv2dn(cn=users,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=users,ou=groups,dc=lsoft,dc=ru)=0 
=> ldap_bv2dn(cn=startup,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=startup,ou=groups,dc=lsoft,dc=ru)=0 
=> ldap_bv2dn(cn=delphi,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=delphi,ou=groups,dc=lsoft,dc=ru)=0 
=> ldap_bv2dn(cn=buxs,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=buxs,ou=groups,dc=lsoft,dc=ru)=0 
=> ldap_bv2dn(cn=techdep,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=techdep,ou=groups,dc=lsoft,dc=ru)=0 
=> ldap_bv2dn(cn=common user group,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=common user group,ou=groups,dc=lsoft,dc=ru)=0 
=> ldap_bv2dn(cn=common user group,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=common user group,ou=groups,dc=lsoft,dc=ru)=0 
=> ldap_bv2dn(cn=repoupdater,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=repoupdater,ou=groups,dc=lsoft,dc=ru)=0 
=> ldap_bv2dn(cn=sftponly,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=sftponly,ou=groups,dc=lsoft,dc=ru)=0 
=> ldap_bv2dn(cn=himan777,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=himan777,ou=groups,dc=lsoft,dc=ru)=0 
=> ldap_bv2dn(cn=computers,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=computers,ou=groups,dc=lsoft,dc=ru)=0 
ldap_build_search_req ATTRS: cn userPassword memberUid uniqueMember gidNumber
=> ldap_bv2dn(cn=repoupdater,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=repoupdater,ou=groups,dc=lsoft,dc=ru)=0 
ldap_build_search_req ATTRS: cn userPassword memberUid uniqueMember gidNumber
=> ldap_bv2dn(cn=repoupdater,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=repoupdater,ou=groups,dc=lsoft,dc=ru)=0 
ldap_build_search_req ATTRS: cn userPassword memberUid uniqueMember gidNumber
=> ldap_bv2dn(cn=repoupdater,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=repoupdater,ou=groups,dc=lsoft,dc=ru)=0 
ldap_build_search_req ATTRS: cn userPassword memberUid uniqueMember gidNumber
=> ldap_bv2dn(cn=users,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=users,ou=groups,dc=lsoft,dc=ru)=0 
ldap_build_search_req ATTRS: cn userPassword memberUid uniqueMember gidNumber
=> ldap_bv2dn(cn=delphi,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=delphi,ou=groups,dc=lsoft,dc=ru)=0 
ldap_build_search_req ATTRS: cn userPassword memberUid uniqueMember gidNumber
=> ldap_bv2dn(cn=techdep,ou=groups,dc=lsoft,dc=ru,0)
<= ldap_bv2dn(cn=techdep,ou=groups,dc=lsoft,dc=ru)=0 
uid=2020(himan777) gid=2011(repoupdater) groups=2011(repoupdater),2002(users),2004(delphi),2007(techdep)
root@lspm138:~ #

Fix: 

Downgrade port nss_ldap to r323088
Comment 1 Edwin Groothuis freebsd_committer freebsd_triage 2014-05-21 16:55:05 UTC
Responsible Changed
From-To: freebsd-ports-bugs->des

Over to maintainer (via the GNATS Auto Assign Tool)
Comment 2 Dag-Erling Smørgrav freebsd_committer freebsd_triage 2014-06-05 21:54:59 UTC
Confirmed, and near the top of my todo-list at $realjob.
Comment 3 Dag-Erling Smørgrav freebsd_committer freebsd_triage 2014-10-07 19:05:30 UTC
I've tried to debug this (it affects some of my production systems as well), but with no luck so far.  As, nss_ldap does not work at all in my environment without the krb5 / gssapi patches, I can't easily compare working and non-working versions.  Can you give me a full trace (loglevel 255) from both versions?  Can you provide access to a VM or jail in a test environment?
Comment 4 Dag-Erling Smørgrav freebsd_committer freebsd_triage 2014-10-07 19:20:53 UTC
By the way, did you build the port with default options?

The strange thing is that all I changed (yes, I know, famous last words...) was the code that enables the use of Kerberos to bind to the server, which is only needed for Active Directory.  Is your LDAP server in fact an Active Directory server, and are you using Kerberos to authenticate with it?
Comment 5 Oleg Sharoyko 2014-11-10 19:38:26 UTC
Hi,

I have the same problem with openldap and simple bind. It looks like the problem is specific to getgroupmembership(). If I remove it from the list of the methods provided by the module (which, I assume, forces the library to retrieve group membership via another code path) then id would successfully get all the groups.

Cheers,
Oleg.
Comment 6 Oleg Sharoyko 2014-11-10 22:20:30 UTC
BTW, rolling back to r323088 doesn't seem to help me.
Comment 7 Oleg Sharoyko 2014-11-11 17:58:30 UTC
Just dug a bit more and found that in my case the problem appeared because of

nss_connect_policy oneshot

With

nss_connect_policy persist

_nss_ldap_initgroups_dyn returns the correct set of groups.

Looking at some debugging output that I gathered when `nss_connect_policy' is set to `oneshot' then do_result() will successfully read the first result (first group the user is a member of) but then on next invocation it will fail the  condition shown below (ldap-nss.c line 2443) and that will terminate the loop in _nss_ldap_getent_ex() and cause _nss_ldap_initgroups_dyn() to return NSS_UNAVAIL (-1).

  if (__session.ls_state != LS_CONNECTED_TO_DSA)
    {
      debug ("<== do_result");
      return NSS_UNAVAIL;
    }

I would be surprised if this is a intended behaviour for _nss_ldap_initgroups_dyn to fail in `oneshot' mode but maybe I'm just missing something.
Comment 8 Dag-Erling Smørgrav freebsd_committer freebsd_triage 2014-11-12 15:16:50 UTC
Made no difference for me, and besides, persist is the default...
Comment 9 Dag-Erling Smørgrav freebsd_committer freebsd_triage 2014-11-13 21:26:44 UTC
I've spent all day adding debugging printf()s throughout nss_ldap and
libc (nsdispatch and nss_compat) and it looks like the bug is actually
in libc, which explains why rolling back did not help Oleg.

There are two axes here.  First, nss_ldap can be used in two ways:
either exclusively or via the compat mechanism.  Second, there are
three ways to access the group database: enumeration (getgrent()),
lookup by name or gid (getgrnam() / getgrgid()) and lookup by member
(getgrouplist()).

When used exclusively ("group: ldap"), nss_ldap works perfectly.

When used through the compat mechanism ("group: compat" and
"group_compat: ldap"), enumeration and lookup by name or gid work, but
lookup by member does not.

This is reflected in the fact that, say, "getent group" (enumeration)
and "getent group <group>" (lookup by name) work but "id <user>"
(lookup by member) does not.

So let's take a closer look at getgrouplist(3).  It's a very simple
function: all it does is call getgroupmembership().  This is
dispatched in the normal fashion, except that a fallback method is
available.

First bug: nss_compat does not provide a getgroupmembership() method,
so the fallback method is called instead of nss_ldap's.

The fallback method is fine, in principle.  It uses getgrent_r() to
iterate over the entire group database, looking for groups whose
membership includes the specified user, so it can be horrendously
slow, but it works.

The fallback method needs to restrict nsdispatch to a single source,
because it's intended to replace that source's nonexistent method.  If
it didn't do this, you would get a list that included the same groups
multiple times.  For instance, if a user has been added to wheel in
/etc/group, the list returned by getgrouplist() will contain wheel
twice: first from the "files" source, and then from the fallback
method which invokes the "files" source a second time.

Second and far more complicated bug: nsdispatch() fails to find a
getgroupmembership() method in nss_compat, so it calls the fallback
method.  The fallback method asks nsdispatch to call the setgrent()
method for the "compat" source, but "compat" is magic: it is
dispatched to compat_setgrent() which calls nsdispatch() again with
the name of the *real* source to use ("ldap" in this case).  But the
fallback flag is set, so instead of the "compat_group" list, it uses
the fallback source it got from compat_setgrent(), and the NIS
setgrent() is called instead of the LDAP one.

I have no idea how to fix the second bug.  The best I can do is fix
the first one so we don't encounter the second.  But even that is
non-trivial, due to the magic nature of compat.  I'll try to have a
patch ready by next week.
Comment 10 Dag-Erling Smørgrav freebsd_committer freebsd_triage 2014-11-13 21:42:15 UTC
Ah, fudge.  I just realized that the first bug is much harder to fix
than I initially thought.  I'm going to have to try to fix the second
instead.  In the meantime, I'll commit fixes for a couple of other
bugs I found while working on this.
Comment 11 Dag-Erling Smørgrav freebsd_committer freebsd_triage 2014-11-14 03:51:38 UTC
Created attachment 149388 [details]
Allow fallback methods to perform arbitrary lookups

Who needs sleep, anyway?  Here's a patch for the second bug.  It works
by recording the depth at which the fallback method was called, and
ensuring that while the fallback method is constrained to a single
source, it can still perform unconstrained lookups.

I'm not 100% sure this is correct in the general case, but it works
for nss_ldap and other combinations I've tested (including with nscd).
Comment 12 commit-hook freebsd_committer freebsd_triage 2014-11-25 09:48:04 UTC
A commit references this bug:

Author: des
Date: Tue Nov 25 09:47:16 UTC 2014
New revision: 275020
URL: https://svnweb.freebsd.org/changeset/base/275020

Log:
  The fallback flag in nsdispatch prevents the fallback implementation of
  getgroupmembership() from invoking the correct backend in the compat case.
  Replace it with a nesting depth counter so it only blocks one level (the
  first is the group -> group_compat translation, the second is the actual
  backend).  This is one of two bugs that break getgrouplist() in the compat
  case, the second being that the backend's own getgroupmembership() method
  is ignored.  Unfortunately, that is not easily fixable without a redesign
  of our nss implementation (which is also needed to implement the +@group
  syntax in /etc/passwd).

  PR:		190055
  MFC after:	1 week

Changes:
  head/lib/libc/net/nsdispatch.c
Comment 13 Glen Barber freebsd_committer freebsd_triage 2015-07-08 18:32:09 UTC
To originators/assignees of this PR:

A commit to the tree references this PR, however the PR is still in a non-closed state.

Please review this PR and close as appropriate, or if closing the PR requires a merge to stable/10, please let re@ know as soon as possible.

Thank you.

Glen
Comment 14 commit-hook freebsd_committer freebsd_triage 2015-07-09 13:31:17 UTC
A commit references this bug:

Author: des
Date: Thu Jul  9 13:30:38 UTC 2015
New revision: 285317
URL: https://svnweb.freebsd.org/changeset/base/285317

Log:
  MFH (r275020): partial fix for getgrouplist() in group_compat case

  PR:		190055
  Approved by:	re (marius)

Changes:
_U  stable/10/
  stable/10/lib/libc/net/nsdispatch.c
Comment 15 Eitan Adler freebsd_committer freebsd_triage 2018-05-23 10:27:14 UTC
batch change of PRs untouched in 2018 marked "in progress" back to open.
Comment 16 Dag-Erling Smørgrav freebsd_committer freebsd_triage 2018-05-23 11:48:08 UTC
Still being actively worked on.
Comment 17 Mark Linimon freebsd_committer freebsd_triage 2024-01-10 03:18:38 UTC
^Triage: apparently committed back in 2015.
Comment 18 Dag-Erling Smørgrav freebsd_committer freebsd_triage 2024-01-10 12:16:20 UTC
No, still not fully fixed.