The conversion of bsd.databases.mk to Uses/bdb.mk doesn't work as expected. Even with a global override, the USE_BDB compatibility hook in bsd.port.mk is selecting the wrong version. eg: WITH_BDB_VER= 48 .if ${.CURDIR:M*/databases/db5*} != "" IGNORE= Damage control .endif .if ${.CURDIR:M*/databases/db6*} != "" IGNORE= Damage control .endif .. leads to this: [00:00:09] ====>> Checking packages for incremental rebuild needed [00:00:20] ====>> Deleting py27-bsddb-2.7.11_5.txz: missing dependency: db5-5.3.28_3 [..] [00:00:22] ====>> Hit CTRL+t at any time to see build progress and stats [00:00:22] ====>> [01][00:00:00] Starting build of databases/db5 [00:00:23] ====>> [01][00:00:01] Finished build of databases/db5: Ignored: Damage control [00:00:23] ====>> [01][00:00:01] Skipping build of databases/py-bsddb: Dependent port databases/db5 ignored [..] py-bsddb/Makefile has "USE_BDB=43+" (like many others) and until r414018, it used to work. It does seem to probe the installed system and uses whatever it found that is installed, but that doesn't help a poudriere run. In that case it seems to ignore any user preference of a global default and uses db5 instead.
PS: shouldn't this be using DEFAULT_VERSIONS?
From Uses/bdb.mk # WITH_BDB_VER - User defined global variable to set Berkeley DB version. .. which I have set and is being ignored.
With no bdb already installed: ports/databases/py-bsddb# make -V WITH_BDB_VER 48 ports/databases/py-bsddb# make debug-bdb --INPUTS---------------------------------------------------- PY27_BSDDB_WITH_BDB_VER: WITH_BDB_VER: 48 BDB_BUILD_DEPENDS: bdb_ARGS (original): 43+ WITH_BDB_HIGHEST (original): --PROCESSING------------------------------------------------ supported versions: 48 5 6 invalid versions: installed versions: eligible versions: 48 5 bdb_ARGS (effective): 48+ WITH_BDB_HIGHEST (override): yes --OUTPUTS--------------------------------------------------- BDB_VER=5 BDB_INCLUDE_DIR=/usr/local/include/db5 BDB_LIB_NAME=db-5.3 BDB_LIB_CXX_NAME=db_cxx-5.3 BDB_LIB_DIR=/usr/local/lib/db5 BUILD_DEPENDS= LIB_DEPENDS= ------------------------------------------------------------ It appears that if there's no pre-installed version, it ignores the user's choice and instead internally sets _WITH_BDB_HIGHEST. This is a serious POLA violation.
The old code had: -# Override USE_BDB with global WITH_BDB_VER -.if defined(WITH_BDB_VER) -. if ${WITH_BDB_VER} != 1 -USE_BDB= ${WITH_BDB_VER} -. endif -.endif - -# Override USE_BDB with maintainer's WANT_BDB_VER -.if defined(WANT_BDB_VER) -USE_BDB= ${WANT_BDB_VER} -.endif The new code has: +# Override _bdb_ARGS with global WITH_BDB_VER if the maintainer did not +# ask for a more specific version. +.if defined(WITH_BDB_VER) +. if ${WITH_BDB_VER} != 1 && ${_bdb_ARGS} == yes +_bdb_ARGS= ${WITH_BDB_VER} +. endif +.endif This is a loss of functionality. If the maintainer says "43+", then why does that have to mean "ignore the user's request for 48 and use db5 unconditionally" instead? At the very least it should exclude plus versions from this rule. "43+" is not "more specific" than "48" at all.
Yes, it should be moved to DEFAULT_VERSIONS, I'll try to see what I can do.
There was a reason for that change, maybe I collapsed too many variables. Can you try https://github.com/mat813/freebsd-ports/compare/origin/trunk...bdb https://github.com/mat813/freebsd-ports/compare/origin/trunk...bdb.patch And let me know if it solves the problem ?
A commit references this bug: Author: mat Date: Mon May 2 13:16:52 UTC 2016 New revision: 414444 URL: https://svnweb.freebsd.org/changeset/ports/414444 Log: Fix WITH_BDB_VER support, and switch to DEFAULT_VERSIONS. PR: 209183 Reported by: peter Sponsored by: Absolight Changes: head/Mk/Uses/bdb.mk head/Mk/bsd.default-versions.mk
Unfortunately, this is still broken. This code: # 3b. if there is no usable version installed, check defaults .if empty(_INST_BDB_VER) _DFLT_BDB_VER:=${_DB_DEFAULTS} # make sure we use a reasonable version for package builds _WITH_BDB_HIGHEST=yes . for i in ${_DFLT_BDB_VER} . if empty(_SUPP_BDB_VER:M${i}) _DFLT_BDB_VER:= ${_DFLT_BDB_VER:N${i}} . endif . endfor _ELIGIBLE_BDB_VER:=${_DFLT_BDB_VER} .. completely ignores the one from DEFAULT_VERSIONS and uses its hard-coded version list: With DEFAULT_VERSIONS=bdb=48 in make.conf, then: databases/py-bsddb # make debug-bdb --INPUTS---------------------------------------------------- PY27_BSDDB_WITH_BDB_VER: BDB_DEFAULT: 48 ^^^^ BDB_BUILD_DEPENDS: bdb_ARGS (original): 43+ WITH_BDB_HIGHEST (original): --PROCESSING------------------------------------------------ supported versions: 48 5 6 invalid versions: installed versions: eligible versions: 48 5 bdb_ARGS (effective): 48+ WITH_BDB_HIGHEST (override): yes ^^^ What? --OUTPUTS--------------------------------------------------- IGNORE= BDB_VER=5 BDB_INCLUDE_DIR=/usr/local/include/db5 BDB_LIB_NAME=db-5.3 BDB_LIB_CXX_NAME=db_cxx-5.3 BDB_LIB_DIR=/usr/local/lib/db5 BUILD_DEPENDS= LIB_DEPENDS= ------------------------------------------------------------ The code, as written, is if you don't have a version of bdb installed in /usr/local already before you start, it completely ignores what you requested and depends on bdb-5 instead. # WITH_BDB_HIGHEST # - Use the highest installed version of Berkeley DB. This is not what is happening.
Looking a little closer.. If I look over the tree, I see a million USE_BDB=40+ etc. The Uses/bdb.mk code converts all these to "48+" which it then interprets as "Use the latest version in _DB_PORTS, not the DEFAULT_VERSIONS one" One possible fix: sweep the ports tree and get rid of any USE_BDB that's older than 5. ie: USE_BDB=yes or USES+= bdb. This seems to work and should cover most of the tree. The real fix would be to rework the selection logic to actually respect the user's request like the rest of the ports tree does for DEFAULT_VERSIONS. A quick and dirty change to the logic is to leverage the "already installed" check. This appears to place nicely with the Version+ logic. Uses/bdb.mk: @@ -117,6 +116,8 @@ _INST_BDB_VER+=${bdb} . endif .endfor +_INST_BDB_VER:=${_INST_BDB_VER:N${BDB_DEFAULT}} +_INST_BDB_VER+=${BDB_DEFAULT} # 2. parse supported versions: # 2a. build list from _bdb_ARGS Then it looks like this: databases/py-bsddb # make DEFAULT_VERSIONS=bdb=48 debug-bdb | & grep '^BDB_VER' BDB_VER=48 databases/py-bsddb # make DEFAULT_VERSIONS=bdb=5 debug-bdb | & grep '^BDB_VER' BDB_VER=5 databases/py-bsddb # make DEFAULT_VERSIONS=bdb=48 USE_BDB=5 debug-bdb | & grep '^BDB_VER' BDB_VER=5 databases/py-bsddb # make DEFAULT_VERSIONS=bdb=48 USE_BDB=yes debug-bdb | & grep '^BDB_VER' BDB_VER=48 databases/py-bsddb # make DEFAULT_VERSIONS=bdb=5 USE_BDB=yes debug-bdb | & grep '^BDB_VER'BDB_VER=5 databases/py-bsddb # make USE_BDB=yes debug-bdb | & grep '^BDB_VER' BDB_VER=5 databases/py-bsddb # make DEFAULT_VERSIONS=bdb=48 USE_BDB=5+ debug-bdb | & grep '^BDB_VER' BDB_VER=5 databases/py-bsddb # make DEFAULT_VERSIONS=bdb=48 USE_BDB=6 WITH_BDB6_PERMITTED=yes debug-bdb | & grep '^BDB_VER' BDB_VER=6 This trivial change gives the least astonishing outcome that I can imagine. Pretending that the default version is already installed avoids duplicating and messing with an awful lot of ugly make logic. After all, it will be installed at the end of the run.
Even if I'm the one who touched most of that aweful code most lately, I won't say I understand it. I'm wondering if the code was not supposed to not let you have 4.8 as the default, so that everything but the *coin ports use 5 or more.
A commit references this bug: Author: mat Date: Mon Aug 8 13:46:57 UTC 2016 New revision: 419843 URL: https://svnweb.freebsd.org/changeset/ports/419843 Log: USE_BDB cleanup. - USE_BDB=4x+ -> USES=bdb. - USE_BDB=yes -> USES=bdb. - USE_BDB=xx -> USES=bdb:xx. Other modernisations when I see them. PR: 209183 Sponsored by: Absolight Changes: head/archivers/rpm4/Makefile head/audio/jack/Makefile head/audio/moc/Makefile head/chinese/libpinyin/Makefile head/chinese/libtabe/Makefile head/chinese/p5-Lingua-ZH-TaBE/Makefile head/comms/trustedqsl/Makefile head/comms/xastir/Makefile head/databases/dbtool/Makefile head/databases/evolution-data-server/Makefile head/databases/fortytwo-bdb/Makefile head/databases/libgda4/Makefile head/databases/libgda5/Makefile head/databases/memcacheq/Makefile head/databases/p5-BDB/Makefile head/databases/p5-BerkeleyDB/Makefile head/databases/py-bsddb/Makefile head/databases/rdfdb/Makefile head/databases/ruby-bdb/Makefile head/devel/ice/Makefile head/devel/librcc/Makefile head/devel/subversion/Makefile.common head/devel/subversion18/Makefile.common head/dns/dnshistory/Makefile head/dns/fastresolve/Makefile head/editors/nvi-devel/Makefile head/editors/poedit/Makefile head/graphics/fortytwo/Makefile head/japanese/mutt-devel/Makefile head/lang/gnu-cobol/Makefile head/lang/php55/Makefile.ext head/lang/php56/Makefile.ext head/lang/php70/Makefile.ext head/mail/avenger/Makefile head/mail/bmf/Makefile head/mail/bogofilter/Makefile head/mail/dk-milter/Makefile head/mail/dovecot/Makefile head/mail/drac/Makefile head/mail/evolution-ews/Makefile head/mail/exim/Makefile head/mail/greyfix/Makefile head/mail/isync/Makefile head/mail/meta1/Makefile head/mail/opendkim/Makefile head/mail/perdition/Makefile head/mail/postfix/Makefile head/mail/postfix-current/Makefile head/mail/postfix211/Makefile head/mail/prayer/Makefile head/mail/sendmail/Makefile head/mail/spamprobe/Makefile head/mail/spmfilter/Makefile head/mail/vacation/Makefile head/misc/hotkeys/Makefile head/net/netatalk/Makefile head/net/openldap24-server/Makefile head/net-im/jabberd/Makefile head/net-p2p/bitcoin/Makefile head/net-p2p/digitalcoin/Makefile head/net-p2p/dogecoin/Makefile head/net-p2p/jigdo/Makefile head/net-p2p/litecoin/Makefile head/net-p2p/namecoin/Makefile head/net-p2p/twister/Makefile head/net-p2p/zetacoin/Makefile head/news/inn/Makefile head/print/panda/Makefile head/security/pks/Makefile head/security/sks/Makefile head/sysutils/apt/Makefile head/sysutils/cfengine22/Makefile head/sysutils/cfengine32/Makefile head/textproc/p5-RDFStore/Makefile head/textproc/redland/Makefile head/www/c-icap/Makefile head/www/crawl/Makefile head/www/squidguard/Makefile head/www/webalizer/Makefile head/www/xshttpd-devel/Makefile
Peter, is this still a problem, or did r419843 fix it ?
I'm waiting for builds to run to confirm, but I believe r419843 didn't fix the original problem.
ping