Created attachment 214000 [details] Portmaster output Greetings, I tried to update curl via portmaster and ran into the attached error. I also got: pkg which /usr/local/include/gssapi/gssapi.h /usr/local/include/gssapi/gssapi.h was installed by package krb5-1.18.1 Is there any other useful information I could provide?
I'm not familiar with portmaster. What happened if you run "make deinstall clean install"? Besides, are you using default versions (DEFAULT_VERSIONS in /etc/make.conf) and options? If not, what are the changes?
(In reply to Sunpoet Po-Chuan Hsieh from comment #1) I am seeing the exact same error with `make deinstall clean install`. My make.conf looks like this: cat /etc/make.conf WITH_CCACHE_BUILD=yes DEFAULT_VERSIONS+= perl5=5.30 perl5 already reached that default version so I removed it from make.conf and tried it again with the same result.
Just to be sure, I downgraded to the previous version by running portdowngrade ftp/curl r530365 and this builds fine with: make deinstall install clean
The problem is related to the fact GSSAPI_BASE is on, but MIT Kerberos is installed. root@deblndw010x - /usr/ports/ftp/curl 2108 # make debug-krb cc -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -o /tmp/debug-krb.x -I"/usr/include" -lkrb5 -lgssapi -lgssapi_krb5 /tmp/debug-krb.c && ldd /tmp/debug-krb.x; /bin/rm -f /tmp/debug-krb.x ld: error: undefined symbol: krb5_gss_register_acceptor_identity >>> referenced by debug-krb.c >>> /tmp/debug-krb-a9d39b.o:(main) cc: error: linker command failed with exit code 1 (use -v to see invocation) PREFIX: /usr/local GSSAPIBASEDIR: /usr GSSAPIINCDIR: /usr/include GSSAPILIBDIR: /usr/lib GSSAPILIBS: -lkrb5 -lgssapi -lgssapi_krb5 GSSAPICPPFLAGS: -I/usr/include GSSAPILDFLAGS: GSSAPI_CONFIGURE_ARGS: CFLAGS=-I/usr/include -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing LDFLAGS= -L/usr/lib -L/usr/lib -fstack-protector-strong LIBS=-lkrb5 -lgssapi -lgssapi_krb5 -lkrb5 -lgssapi -lgssapi_krb5 -L/usr/local/lib KRB5CONFIG=/usr/bin/krb5-config KRB5CONFIG: /usr/bin/krb5-config CFLAGS: -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing LDFLAGS: -L/usr/lib -L/usr/lib -fstack-protector-strong LDADD: "-isystem /usr/local/include" gets passed. Therefore header files from MIT Kerberos is used. Base headers are ignored. This is also passed to ./configure. Removing it manually makes it work: root@deblndw010x - /usr/ports/ftp/curl 2110 # find . -name \*.so ./work/curl-7.70.0/lib/.libs/libcurl.so root@deblndw010x - /usr/ports/ftp/curl 2111 # ldd ./work/curl-7.70.0/lib/.libs/libcurl.so ./work/curl-7.70.0/lib/.libs/libcurl.so: libnghttp2.so.14 => /usr/local/lib/libnghttp2.so.14 (0x800701000) libssl.so.111 => /usr/lib/libssl.so.111 (0x80072e000) libheimntlm.so.11 => /usr/lib/libheimntlm.so.11 (0x8007c5000) libhx509.so.11 => /usr/lib/libhx509.so.11 (0x800e00000) libcom_err.so.5 => /usr/lib/libcom_err.so.5 (0x8007ce000) libcrypto.so.111 => /lib/libcrypto.so.111 (0x800e51000) libasn1.so.11 => /usr/lib/libasn1.so.11 (0x80111d000) libwind.so.11 => /usr/lib/libwind.so.11 (0x8007d2000) libheimbase.so.11 => /usr/lib/libheimbase.so.11 (0x8011c3000) libroken.so.11 => /usr/lib/libroken.so.11 (0x8011c9000) libcrypt.so.5 => /lib/libcrypt.so.5 (0x8011de000) libz.so.6 => /lib/libz.so.6 (0x8011fe000) libkrb5.so.11 => /usr/lib/libkrb5.so.11 (0x801219000) libgssapi.so.10 => /usr/lib/libgssapi.so.10 (0x80129a000) libgssapi_krb5.so.10 => /usr/lib/libgssapi_krb5.so.10 (0x8012a6000) libthr.so.3 => /lib/libthr.so.3 (0x8012c7000) libc.so.7 => /lib/libc.so.7 (0x80024b000) libprivateheimipcc.so.11 => /usr/lib/libprivateheimipcc.so.11 (0x8012f3000) I cannot exactly tell whether the cause is curl or /usr/ports/Mk/Uses/gssapi.mk.
Here is the cause failing the build: 27ea8fc2faa4e4753a9cff0a7f48d1dba68013d6 is the first bad commit commit 27ea8fc2faa4e4753a9cff0a7f48d1dba68013d6 Author: Daniel Stenberg <daniel@haxx.se> Date: Sun Mar 8 11:08:13 2020 +0100 configure: convert -I to -isystem as a last step As all the -I uses in CFLAGS at that point are for system headers and third party libraries this helps us remove/ignore warnings on those! Closes #5060 configure.ac | 2 ++ m4/curl-compilers.m4 | 9 ++++----- 2 files changed, 6 insertions(+), 5 deletions(-) I not sure whether this is really regression or simply reveals a bug in gssapi.mk. cyrus-sasl-gssapi2 compiles fine for both. I would recommend to talk to gssapi.mk's maintainer first before raising a bug upstream.
(In reply to hakan from comment #3) The bug was already present, but simply not visible: # svn up -r530365 Updating '.': U Makefile A files/patch-lib-easy.c U distinfo Updated to revision 530365. root@deblndw011x:/usr/ports/ftp/curl # make debug-krb cc -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -o /tmp/debug-krb.x -I"/usr/include" -lkrb5 -lgssapi -lgssapi_krb5 /tmp/debug-krb.c && ldd /tmp/debug-krb.x; /bin/rm -f /tmp/debug-krb.x ld: error: undefined symbol: krb5_gss_register_acceptor_identity >>> referenced by debug-krb.c >>> /tmp/debug-krb-a62152.o:(main) cc: error: linker command failed with exit code 1 (use -v to see invocation) PREFIX: /usr/local GSSAPIBASEDIR: /usr GSSAPIINCDIR: /usr/include GSSAPILIBDIR: /usr/lib GSSAPILIBS: -lkrb5 -lgssapi -lgssapi_krb5 GSSAPICPPFLAGS: -I/usr/include GSSAPILDFLAGS: GSSAPI_CONFIGURE_ARGS: CFLAGS=-I/usr/include -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing LDFLAGS= -L/usr/lib -L/usr/lib -fstack-protector-strong LIBS=-lkrb5 -lgssapi -lgssapi_krb5 -lkrb5 -lgssapi -lgssapi_krb5 -L/usr/local/lib KRB5CONFIG=/usr/bin/krb5-config KRB5CONFIG: /usr/bin/krb5-config CFLAGS: -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing LDFLAGS: -L/usr/lib -L/usr/lib -fstack-protector-strong LDADD:
OK, I have found the cause, but don't know how to fix it. Here is the bug: root@deblndw011x:/usr/ports/ftp/curl # make -V CPPFLAGS -I"/usr/include" -DCURL_DISABLE_NTLM -I/usr/include -isystem /usr/local/include root@deblndw011x:/usr/ports/Mk/Uses # grep -r isystem .. ../Uses/compiler.mk:CXXFLAGS+= -nostdinc++ -isystem /usr/include/c++/v1 ../Uses/localbase.mk:CPPFLAGS+= -isystem ${LOCALBASE}/include ../Uses/localbase.mk:CFLAGS+= -isystem ${LOCALBASE}/include ../Uses/localbase.mk:CXXFLAGS+= -isystem ${LOCALBASE}/include root@deblndw011x:/usr/ports/ftp/curl # grep -r localbase . ./Makefile:USES= cpe libtool localbase pathfix perl5 shebangfix tar:xz Basically, localbase kills the environment!
this patch makes it work again: root@deblndw011x:/usr/ports/ftp/curl # svn diff Index: Makefile =================================================================== --- Makefile (revision 533925) +++ Makefile (working copy) @@ -43,7 +43,7 @@ THREADED_RESOLVER_DESC= Threaded DNS resolver TLS_SRP_DESC= TLS-SRP (Secure Remote Password) support -USES= cpe libtool localbase pathfix perl5 shebangfix tar:xz +USES= cpe libtool pathfix perl5 shebangfix tar:xz USE_PERL5= build CONFIGURE_ARGS= --disable-werror \ root@deblndw011x:/usr/ports/ftp/curl # make debug-krb cc -O2 -pipe -fstack-protector-strong -fno-strict-aliasing -o /tmp/debug-krb.x -I"/usr/include" -lkrb5 -lgssapi -lgssapi_krb5 /tmp/debug-krb.c && ldd /tmp/debug-krb.x; /bin/rm -f /tmp/debug-krb.x /tmp/debug-krb.x: libkrb5.so.11 => /usr/lib/libkrb5.so.11 (0x80024a000) libgssapi.so.10 => /usr/lib/libgssapi.so.10 (0x8002cb000) libgssapi_krb5.so.10 => /usr/lib/libgssapi_krb5.so.10 (0x8002d7000) libc.so.7 => /lib/libc.so.7 (0x8002f8000) libasn1.so.11 => /usr/lib/libasn1.so.11 (0x8006ef000) libcom_err.so.5 => /usr/lib/libcom_err.so.5 (0x800795000) libcrypt.so.5 => /lib/libcrypt.so.5 (0x800799000) libcrypto.so.111 => /lib/libcrypto.so.111 (0x8007b9000) libhx509.so.11 => /usr/lib/libhx509.so.11 (0x800a85000) libroken.so.11 => /usr/lib/libroken.so.11 (0x800ad6000) libwind.so.11 => /usr/lib/libwind.so.11 (0x800aeb000) libheimbase.so.11 => /usr/lib/libheimbase.so.11 (0x800b15000) libprivateheimipcc.so.11 => /usr/lib/libprivateheimipcc.so.11 (0x800b1b000) libthr.so.3 => /lib/libthr.so.3 (0x800b20000) PREFIX: /usr/local GSSAPIBASEDIR: /usr GSSAPIINCDIR: /usr/include GSSAPILIBDIR: /usr/lib GSSAPILIBS: -lkrb5 -lgssapi -lgssapi_krb5 GSSAPICPPFLAGS: -I/usr/include GSSAPILDFLAGS: GSSAPI_CONFIGURE_ARGS: CFLAGS=-I/usr/include -O2 -pipe -fstack-protector-strong -fno-strict-aliasing LDFLAGS= -L/usr/lib -L/usr/lib -fstack-protector-strong LIBS=-lkrb5 -lgssapi -lgssapi_krb5 -lkrb5 -lgssapi -lgssapi_krb5 KRB5CONFIG=/usr/bin/krb5-config KRB5CONFIG: /usr/bin/krb5-config CFLAGS: -O2 -pipe -fstack-protector-strong -fno-strict-aliasing LDFLAGS: -L/usr/lib -L/usr/lib -fstack-protector-strong LDADD:
Port maintainer, can we make progress here? I have provided a lot of information to solve this issue.
I could build this port successfully with GSSAPI_BASE (default), GSSAPI_MIT or GSSAPI_HEIDMAL. (In reply to Michael Osipov from comment #4) > The problem is related to the fact GSSAPI_BASE is on, but MIT Kerberos is installed. You should always choose the proper GSSAPI option. If you use MIT Kerberos, you should choose GSSAPI_MIT. (In reply to Michael Osipov from comment #6) With GSSAPI_MIT: % make debug-krb cc -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -o /tmp/debug-krb.x -I"/usr/local/include" -lkrb5 -lgssapi_krb5 -L"/usr/local/lib" -Wl,-rpath,/usr/local/lib /tmp/debug-krb.c && ldd /tmp/debug-krb.x; /bin/rm -f /tmp/debug-krb.x /tmp/debug-krb.x: libkrb5.so.3.3 => /usr/local/lib/libkrb5.so.3.3 (0x800823000) libgssapi_krb5.so.2.2 => /usr/local/lib/libgssapi_krb5.so.2.2 (0x800b0d000) libc.so.7 => /lib/libc.so.7 (0x800d5f000) libk5crypto.so.3.1 => /usr/local/lib/libk5crypto.so.3.1 (0x80111a000) libcom_err.so.3.0 => /usr/local/lib/libcom_err.so.3.0 (0x80134a000) libkrb5support.so.0.1 => /usr/local/lib/libkrb5support.so.0.1 (0x80154d000) libintl.so.8 => /usr/local/lib/libintl.so.8 (0x80175c000) PREFIX: /usr/local GSSAPIBASEDIR: /usr/local GSSAPIINCDIR: /usr/local/include GSSAPILIBDIR: /usr/local/lib GSSAPILIBS: -lkrb5 -lgssapi_krb5 GSSAPICPPFLAGS: -I/usr/local/include GSSAPILDFLAGS: -L/usr/local/lib GSSAPI_CONFIGURE_ARGS: CFLAGS=-I/usr/local/include -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing LDFLAGS=-L/usr/local/lib -L/usr/local/lib -L/usr/lib -Wl,-rpath,/usr/local/lib:/usr/lib -L/usr/lib -fstack-protector-strong LIBS=-lkrb5 -lgssapi_krb5 -lkrb5 -lgssapi_krb5 -L/usr/local/lib KRB5CONFIG=/usr/local/bin/krb5-config KRB5CONFIG: /usr/local/bin/krb5-config CFLAGS: -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing LDFLAGS: -L/usr/local/lib -L/usr/lib -Wl,-rpath,/usr/local/lib:/usr/lib -L/usr/lib -fstack-protector-strong LDADD:
I do not fully agree. Even if MIT Kerberos is installed, this shouldn't force me to compile with it. I should work with base. Other ports do work too. I think this has to be discussed with LOCALBASE and GSSAPI .mk maintainers. Especially because that this is issue is very hard to find for most admins users as you can see I have put a lot of dev effort into it.
Created attachment 218834 [details] patch to flag if GSSAPI_BASE is selected while krb5 is installed This bug persists in 7.73.0, and I am providing a patch to flag the incompatible situation and propose three different remedies.
(In reply to Michael Osipov from comment #11) Michael, what do you propose instead, (worthy for upstream inclusion!), how should the curl build isolate itself from krb5 in /usr/local when you want base?
(In reply to Sunpoet Po-Chuan Hsieh from comment #1) portmaster ultimately is only a wrapper that (1) builds or upgrade requisite packages first, and (2) then does the equivalent (more or less) of "make all deinstall install clean". The key message in portmaster is that it's a build-from-ports in a live system, as opposed to poudriere with does a clean-room rebuild in a controlled environment. The minimal procedure to reproduce the issue without portmaster is: pkg install -y krb5 cd /usr/ports/ftp/curl make rmconfig deinstall clean make install BATCH=yes
(In reply to Matthias Andree from comment #12) With my patch, the attempt to reproduce the bug will fail early like so: $ make install BATCH=yes ===> curl-7.73.0 is marked as broken: The GSSAPI_BASE option is selected, but you have krb5 installed. Select GSSAPI_MIT, temporarily deinstall krb5 to build curl, or build in poudriere. *** Error code 1 Stop.
(In reply to Matthias Andree from comment #13) Let me get back to you in the next couple of days. I haven't looked at it for a while. If we can work out a decent patch, I'll be happy to merge it upstream. I am a curl committer.
(In reply to Michael Osipov from comment #16) Michael, my patch isn't suitable for upstream, it just catches the incompatible configuration in the port early on and flags it. Of course if upstream can fix things such as to make cURL more compatible across GSSAPI/Kerberos V implementations, that'll be good as well.
(In reply to Matthias Andree from comment #17) Have you already checked curl's configure.ac? The basic problem is as with most Makefiles, some other libs require -I/usr/local/include and -L/usr/local/lib. Eventhough you have configured GSS-API to be from /usr you cannot avoid having it from /usr/local. As sad as it sounds...
(In reply to Michael Osipov from comment #18) Michael, no, and I don't need to because I understand the nature of the bug, which is why I took the easy way to short-circuit the build and fail it if we can tell that it won't proceed. It might however have been the case that you had planned for cURL to actually check the real implementation that the system would provide and its features. :-)
(In reply to Matthias Andree from comment #19) What is the nature of the bug from your POV? I'd like to have common understanding.
IMP, the nature of the bug is that the /usr/local/SUBDIR and /usr/SUBDIR implementations of GSSAPI are incompatible if FreeBSD's krb5 package is installed and curl is supposed to use GSSAPI_BASE. It cannot achieve that because the GSSAPI stuff in /usr/local shadows the base implementation. The port or the upstream code (not sure which) might need to run actual feature/version checks and not be told from the outside what to expect, so it sees that it gets krb5. Of course one might also hack the port to detect the presence of krb5 and if it's installed, flip the switch from GSSAPI_BASE to GSSAPI_MIT. I doubt much of this is really upstream stuff. A few "make debug-krb5" later it seems that we pass the port garbage through LIBS and CFLAGS, and it ends up using the wrong libraries and headers. For one, we pass -lkrb5 believing it would be /usr/lib/libkrb5.so but in fact it will be /usr/local/lib/libkrb5.so. Garbage in, garbage out. My patch just traps the bogus configuration. I haven't tested if other misconfigurations explode in the user's face at run-time, I've only compile-tested those. I don't have Kerberized systems to test against.
I gave it another spin. I still stand by my previous statements. It is a flaw between localbase and gssapi.mk. These ports do use localbase and GSSAPI_* and they do not work too: > # grep -r localbase --include='**/Makefile*' -l . > out > # cat out | xargs -I ^ grep -l GSSAPI ^ > ./www/lighttpd/Makefile > ./security/heimdal/Makefile > ./mail/neomutt/Makefile > ./mail/mailutils/Makefile > ./net/freerdp/Makefile > ./net/samba413/Makefile > ./net/wireshark/Makefile > ./net/samba410/Makefile > ./net/ocserv/Makefile > ./net/samba411/Makefile > ./net/samba412/Makefile > ./ftp/curl/Makefile Let's see: > # cd ./mail/neomutt/Makefile > ### configure with base while MIT Kerberos is installed > # make debug-krb > cc -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -o /tmp/debug-krb.x -I"/usr/include" -lkrb5 -lgssapi -lgssapi_krb5 /tmp/debug-krb.c && ldd /tmp/debug-krb.x; /bin/rm -f /tmp/debug-krb.x > ld: error: undefined symbol: krb5_gss_register_acceptor_identity > >>> referenced by debug-krb.c > >>> /tmp/debug-krb-a78e68.o:(main) > cc: error: linker command failed with exit code 1 (use -v to see invocation) > PREFIX: /usr/local > GSSAPIBASEDIR: /usr > GSSAPIINCDIR: /usr/include > GSSAPILIBDIR: /usr/lib > GSSAPILIBS: -lkrb5 -lgssapi -lgssapi_krb5 > GSSAPICPPFLAGS: -I/usr/include > GSSAPILDFLAGS: > GSSAPI_CONFIGURE_ARGS: CFLAGS=-I/usr/include -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing LDFLAGS= -fstack-protector-strong LIBS=-lkrb5 -lgssapi -lgssapi_krb5 -L/usr/local/lib KRB5CONFIG=/usr/bin/krb5-config > KRB5CONFIG: /usr/bin/krb5-config > CFLAGS: -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing > LDFLAGS: -fstack-protector-strong > LDADD: So this is not a curl problem. This is a ports system problem. You cannot do -isystem /usr/local/include and expect everything to work. This can happen with EVERY library which is in base AND installed from ports and incompatible in API or ABI. Read https://gcc.gnu.org/onlinedocs/gcc/Directory-Options.html carefully and look at: > cc -O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -o /tmp/debug-krb.x -I"/usr/include" -lkrb5 -lgssapi -lgssapi_krb5 /tmp/debug-krb.c && ldd /tmp/debug-krb.x; /bin/rm -f /tmp/debug-krb.x Here is "-I"/usr/include"" is pointless because "-isystem /usr/local/include" is queried as system from left to right and those headers are found. I belive that this needs more localbase:args to fix this or turn into -I and make an exprun.
(In reply to Michael Osipov from comment #22) I won't object to your assessment that gssapi.mk is passing down nonsense if GSSAPI_BASE is selected for curl and krb5 is installed and the build ends up mixing up krb libraries and headers from base and localbase. Feel free to instead file a similar sanity check (as I filed for curl with https://bugs.freebsd.org/bugzilla/attachment.cgi?id=218834) for gssapi.mk instead, I was just trying to make curl fail gracefully. The quick solution I am proposing is telling the package to expect $LOCALBASE MIT Kerberos if it is present because it shadows /usr base system Kerberos, or making sure that you don't have MIT Kerberos in $LOCALBASE while building. Poudriere does the latter. I am not convinced that splitting and hacking and splicing include and linker paths is going to be sustainable. The next day someone adds a new option, or changes or updates your compiler, or updates base, you're in for re-sorting the mess. In some situations there's no proper solution and erroring out nicely with er nice instructive error message is easier. If you want to poke at the port a bit more, be my guest but. I will not spend any more time on this. https://bugs.freebsd.org/bugzilla/attachment.cgi?id=218834 is nice and clear when deliberately preventing the build early on, and offers three solutions. I think that's good. The default package build will use GSSAPI_BASE and build reliably in poudriere. So that's that. Don't overdo it.
Users have to choose the proper GSSAPI option when they build the port. The build failure caused by incorrect GSSAPI option is not a bug.