Scenario: - FreeBSD stable/14 - ports latest - Upgrading ports, amongst them devel/dbus Result: - Very little starts anymore because the dbus shlib has changed. Expected result: - Dependent ports should have been bumped so that everything works. -- Martin
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=a347a92e6ea1376b0004ef39e52cad58eddd6ae7 commit a347a92e6ea1376b0004ef39e52cad58eddd6ae7 Author: Gleb Popov <arrowd@FreeBSD.org> AuthorDate: 2025-03-08 14:05:10 +0000 Commit: Gleb Popov <arrowd@FreeBSD.org> CommitDate: 2025-03-08 14:05:10 +0000 *: Bump revisions after dbus update PR: 285253 Reported by: Martin Birgmeier <d8zNeCFG@aon.at> accessibility/at-spi2-core/Makefile | 1 + accessibility/caribou/Makefile | 2 +- archivers/rpm4/Makefile | 2 +- astro/gpsd/Makefile | 2 +- audio/a2jmidid/Makefile | 2 +- audio/amberol/Makefile | 2 +- audio/ario/Makefile | 2 +- audio/cardinal/Makefile | 1 + audio/dragonfly-reverb-lv2/Makefile | 2 +- audio/fluidsynth/Makefile | 1 + audio/gogglesmm/Makefile | 2 +- audio/jack/Makefile | 2 +- audio/ladish/Makefile | 1 + audio/ncspot/Makefile | 2 +- audio/openal-soft/Makefile | 2 +- audio/paprefs/Makefile | 2 +- audio/patchage/Makefile | 2 +- audio/pulseaudio/Makefile | 2 +- audio/rhythmbox/Makefile | 1 + audio/shortwave/Makefile | 2 +- audio/songrec/Makefile | 2 +- audio/spotify-player/Makefile | 2 +- audio/spotifyd/Makefile | 2 +- audio/vgmplay/Makefile | 2 +- audio/zam-plugins-lv2/Makefile | 2 +- benchmarks/mangohud/Makefile | 2 +- cad/PrusaSlicer/Makefile | 2 +- chinese/fcitx/Makefile | 1 + chinese/libcangjie/Makefile | 2 +- comms/anyremote/Makefile | 2 +- deskutils/cairo-dock-plugins/Makefile | 1 + deskutils/cairo-dock/Makefile | 2 +- deskutils/caja-extensions/Makefile | 1 + deskutils/easystroke/Makefile | 2 +- deskutils/fyi/Makefile | 1 + deskutils/gnome-characters/Makefile | 1 + deskutils/gnotime/Makefile | 2 +- deskutils/growl-for-linux/Makefile | 2 +- deskutils/kdeconnect-kde/Makefile | 1 + deskutils/mate-notification-daemon/Makefile | 1 + deskutils/notification-daemon/Makefile | 2 +- deskutils/pinot/Makefile | 1 + devel/dbus-c++/Makefile | 2 +- devel/dbus-glib/Makefile | 2 +- devel/dbus-tcl/Makefile | 1 + devel/dconf/Makefile | 2 +- devel/efl/Makefile | 1 + devel/electron31/Makefile | 2 +- devel/electron32/Makefile | 1 + devel/electron33/Makefile | 2 +- devel/p5-Net-DBus/Makefile | 2 +- devel/py-qt5-pyqt/Makefile | 1 + devel/py-qt6-pyqt/Makefile | 1 + devel/qt5-dbus/Makefile | 2 +- devel/qt6-base/Makefile | 2 +- devel/sdl20/Makefile | 1 + devel/sdl3/Makefile | 1 + devel/tinysparql/Makefile | 1 + dns/dnsmasq-devel/Makefile | 2 +- dns/dnsmasq/Makefile | 2 +- editors/emacs-devel/Makefile | 1 + editors/emacs/Makefile | 1 + editors/libreoffice/Makefile | 2 +- editors/openoffice-4/Makefile | 2 +- editors/openoffice-devel/Makefile | 2 +- emulators/fbsd-duckstation/Makefile | 2 +- emulators/pcsx2/Makefile | 2 +- emulators/tic-80/Makefile | 2 +- emulators/virtualbox-ose-70/Makefile | 2 +- emulators/virtualbox-ose-additions-legacy/Makefile | 2 +- emulators/virtualbox-ose-additions/Makefile | 2 +- emulators/virtualbox-ose-legacy/Makefile | 2 +- emulators/virtualbox-ose/Makefile | 2 +- filesystems/gvfs/Makefile | 1 + ftp/filezilla/Makefile | 1 + ftp/termscp/Makefile | 2 +- ftp/wzdftpd/Makefile | 2 +- games/flightgear/Makefile | 2 +- games/lwjgl3/Makefile | 1 + games/wesnoth/Makefile | 2 +- games/xcowsay/Makefile | 2 +- graphics/colord-gtk/Makefile | 1 + graphics/colord/Makefile | 2 +- graphics/evince/Makefile | 1 + graphics/rawstudio/Makefile | 2 +- graphics/zbar/Makefile | 2 +- irc/hexchat/Makefile | 2 +- lang/clisp/Makefile | 1 + lang/squeak/Makefile | 2 +- mail/evolution/Makefile | 2 +- mail/sylpheed/Makefile | 2 +- misc/xiphos/Makefile | 2 +- multimedia/audacious-plugins/Makefile | 2 +- multimedia/butt/Makefile | 2 +- multimedia/dvdstyler/Makefile | 2 +- multimedia/gnome-mplayer/Makefile | 2 +- multimedia/handbrake/Makefile | 1 + multimedia/kmplayer/Makefile | 2 +- multimedia/kodi/Makefile | 2 +- multimedia/obs-studio/Makefile | 1 + multimedia/pipewire/Makefile | 1 + multimedia/vlc/Makefile | 2 +- multimedia/xfce4-parole/Makefile | 1 + net-im/folks/Makefile | 2 +- net-im/fractal/Makefile | 2 +- net-im/neochat/Makefile | 1 + net-im/pidgin-sipe/Makefile | 2 +- net-im/telegram-desktop/Makefile | 2 +- net-im/telepathy-gabble/Makefile | 2 +- net-im/telepathy-logger-qt5/Makefile | 2 +- net-im/telepathy-mission-control/Makefile | 2 +- net-im/uTox/Makefile | 2 +- net-p2p/gtk-gnutella/Makefile | 2 +- net/avahi-app/Makefile | 2 +- net/netatalk3/Makefile | 2 +- net/netatalk4/Makefile | 1 + net/samba416/Makefile | 2 +- net/samba419/Makefile | 2 +- net/samba420/Makefile | 2 +- net/vinagre/Makefile | 2 +- net/vino/Makefile | 2 +- print/cups-filters/Makefile | 2 +- print/cups/Makefile | 1 + print/foomatic-filters/Makefile | 2 +- print/hplip/Makefile | 2 +- science/zotero/Makefile | 1 + security/gcr3/Makefile | 2 +- security/gnome-keyring/Makefile | 2 +- security/libgnome-keyring/Makefile | 2 +- security/sssd2/Makefile | 2 +- security/wpa_supplicant-devel/Makefile | 2 +- security/wpa_supplicant/Makefile | 2 +- security/wpa_supplicant210/Makefile | 2 +- security/wpa_supplicant29/Makefile | 2 +- sysutils/cinnamon-control-center/Makefile | 2 +- sysutils/consolekit2/Makefile | 2 +- sysutils/dunst/Makefile | 1 + sysutils/fastfetch/Makefile | 1 + sysutils/mate-control-center/Makefile | 2 +- sysutils/mate-power-manager/Makefile | 1 + sysutils/mate-settings-daemon/Makefile | 2 +- sysutils/polkit/Makefile | 1 + sysutils/scanbd/Makefile | 2 +- sysutils/slurm-wlm/Makefile | 2 +- textproc/enchant/Makefile | 2 +- textproc/enchant2/Makefile | 2 +- textproc/fcitx5/Makefile | 1 + textproc/ibus/Makefile | 1 + www/chromium/Makefile | 1 + www/iridium/Makefile | 2 +- www/qt5-webengine/Makefile | 2 +- www/qt6-webengine/Makefile | 2 +- www/ungoogled-chromium/Makefile | 2 +- x11-fm/worker/Makefile | 1 + x11-themes/lxappearance/Makefile | 2 +- x11-toolkits/libdecor/Makefile | 2 +- x11-toolkits/qt5-gui/Makefile | 1 + x11-wm/afterstep-stable/Makefile | 2 +- x11-wm/awesome/Makefile | 2 +- x11-wm/compton/Makefile | 2 +- x11-wm/magpie/Makefile | 2 +- x11-wm/muffin/Makefile | 2 +- x11-wm/picom/Makefile | 1 + x11/budgie-screensaver/Makefile | 2 +- x11/cinnamon-session/Makefile | 2 +- x11/cinnamon/Makefile | 2 +- x11/deforaos-integration/Makefile | 2 +- x11/fnott/Makefile | 1 + x11/kitty/Makefile | 2 +- x11/mate-applets/Makefile | 1 + x11/mate-panel/Makefile | 1 + x11/mate-screensaver/Makefile | 2 +- x11/mate-session-manager/Makefile | 1 + x11/roxterm/Makefile | 1 + x11/slim/Makefile | 2 +- x11/xfce4-screensaver/Makefile | 1 + 176 files changed, 176 insertions(+), 123 deletions(-)
Note that bumps aren't needed with Poudriere, which is the only supported build tool for ports. Build-on-host tools like synth or portmaster should be taught to consult pkg provides/requires to properly rebuild dependencies.
(In reply to Gleb Popov from comment #2) Wondering how building in poudriere will help to _reinstall_ dependent packages without PORTREVISION bump: $ pkg info -l dbus | grep libdbus /usr/local/lib/libdbus-1.a /usr/local/lib/libdbus-1.so /usr/local/lib/libdbus-1.so.3 /usr/local/lib/libdbus-1.so.3.32.4 $ pkg info -b dbus dbus-1.14.10_5,1: libdbus-1.so.3 $ (As we can see, pkg displays 'libdbus-1.so.3' which haven't been changed in dbus-1.16.2.1. Instead, 'libdbus-1.so.3.32.4' has been updated to 'libdbus-1.so.3.38.3', but this is something pkg doesn't see, thus won't offer reinstalling.)
Oh, then I was confused. There was no shlib version bump and no need for PORTREVISION bumps either. Martin, what exactly broke for you after you upgraded dbus?
(In reply to Gleb Popov from comment #2) has this finally become the project's official position? i admit i do not stay well read on the mailing lists but i do read everything put in CHANGES. i havent seen anything to that effect and it looks like the handbook section 5.2.3.1 reads the same as i remember it. actually come to think of it, i didnt realize it said "that includes changes that only affect a package built with non-default options." is our make and our shell no longer a "supported" ports build tool? i realize for many years it hasn't been the "cool" way to do it anymore, but time and time again i've sought assurance and been assured that poudriere will not become a mandatory requirement to utilize the ports tree effectively. my site has yet another such tool, developed internally and maintained since the FreeBSD 6 pkg_install days. it has always relied on a change of PKGNAME (of which PORTREVISION is a component) as signal it's necessary to rebuild a port. save for the occasionally overlooked bump or a non-default optional dependency here and there, it's always reliably kept our rollout up to date with dynamic links intact and it ties into our workflow in a way that is entirely out of scope for poudriere, synth or portmaster. if it "should be taught to consult pkg provides/requires" was there a notice issued that might point me in the right direction? what relationship(s) necessitate i should cascade rebuilds? (whenever LIB_, RUN_, or BUILD_DEPENDS changes, just to be safe?) believe me, i'm not resistant to change in this direction. i would love it if we're ready to stop all the obligatory PORTREVISION bumping commits. i'm not opposed to enhancing our tooling here to detect staleness of same-named packages by other conditional logic. the only reason we haven't is since 5.2.3.1 still reads as it reads. i'll be glad when we usher in this better way of doing things, i just got to say i was unprepared. without commits like a347a92e i too would've had a bunch of broken packages and before reading your comment #2, arrowd@, i probably would have been at a loss to quickly realize why
(In reply to Gleb Popov from comment #4) ya know, i didnt check the shlib vers but i had the same feeling, cuz my stuff worked after dbus update but before the 100+ bumps. i only had to `service dbus restart` to unbreak a great many things. that may have been what Martin was snagged by. i took me about a frustrating hour with an oddly misbehaving system before it dawned on me lol
(In reply to Chad Jacob Milios from comment #5) Ok, I'm now probably spreading FUD, so let me expand. The FreeBSD package building cluster runs Poudriere, so all ports being committed should be buildable by it. Committers are expected to verify that any change they're pushing is buildable in Poudriere. This makes Poudriere a "blessed" build tool, because it is used by portmgr@ and pkgmgr@ The distinctive feature of Poudriere is that it builds each port in a cleanest environment possible - a pristine FreeBSD base system and only needed dependencies. All other build tools around do not guarantee this, which might lead to build failures that do not happen in Poudriere. By "not supported" I mean that committers are not obliged to check whether a port can be compiled outside of Poudriere. Such a requirement doesn't even make sense since there might be myriads of different environment configurations and it is impossible to test them all. The "not supported" claim doesn't mean that portmaster or synth shouldn't be used, however. It doesn't even mean that patches fixing builds under these tools will be rejected. It just means that you're more "out of warranty" if something goes wrong and that you are expected to try and help yourself first. Going non-poudriere path requires you to be a "power user". I often find myself ignoring problem reports like "foo/bar does not build for me with all my custom setup", because it is extremely hard to reproduce the same environment that leads to a build failure. > is our make and our shell no longer a "supported" ports build tool? For a single-shot build - yes, it is the intended way. make is our engine, a lower-level tool that is utilized by a higher-level machinery. The thing is, the "building a port" goal isn't the same as "maintain an up-to-date package set". You can't reduce the latter problem to a series of a former ones. > my site has yet another such tool Just why you're reimplementing Poudriere? Poudriere serves as a golden standard on how to do all these seemingly easy but complex in reality tasks, like computing what has changed and what ports should be rebuilt. Synth, portmaster, portupgrade and your tool are essentially reimplement parts of Poudriere logic, but miss many corner cases and complexity and then ask the framework to adapt to your simple tools, like bumping PORTREVISIONs. Even you don't like something in Poudriere, then the way forward should be to extract common logic from it into a library that all other tools may use. > i only had to `service dbus restart` to unbreak a great many things. that may have been what Martin was snagged by. i took me about a frustrating hour with an oddly misbehaving system before it dawned on me lol First, bumps wouldn't help this issue. The service wouldn't be restarted anyways. Second, and most important - the pkg tool itself is also somewhat low-level. And using it again requires a user to be the "power user". Generally, upgrading some program is not safe while this program is running. A user should understand what he's updating and how to do that safely. Or call "pkg upgrade" from the single user mode to be on the safe side. One day we'll bring PackageKit support up to state where updating packages would happen during system's shutdown or startup sequence, like it is done in many other contemporary OSes.
(In reply to Gleb Popov from comment #4) I am using portmaster because I sometimes want different OPTION settings and poudriere is too heavy for me (also, when I tried it, it did not fit well with my zfs setup, if I remember correctly). What exactly broke: I restarted the system (where only dbus had been upgrade using pkg upgrade from my local repo), and then basically nothing worked anymore. Even firefox was affected (not in the list of changed ports of comment #1), I just got to it working again by another recompile. Do the remarks in comment #2 mean that packages of exactly the same name may still be different because they would be recompiled in poudriere, published to the package site, and pkg would then do a reinstall because it detects the changed libraries? - Of course it is nice not to have to do a portrevision bump for such things, but that packages of the same name can still be different is something I must get used to. Unfortunately, it also means that portmaster gets even more unsuitable. I am not exactly sure how to proceed... -- Martin
(In reply to Martin Birgmeier from comment #8) > What exactly broke: I restarted the system (where only dbus had been upgrade using pkg upgrade from my local repo), and then basically nothing worked anymore. This is still an unclear description. Did you try running a broken app from a terminal to see its stderr? What DE or WM are you using? If you're using something heavyweight and managed to get to the point when you're running Firefox, then there is actually a lot of stuff that is working. > Do the remarks in comment #2 mean that packages of exactly the same name may still be different because they would be recompiled in poudriere, published to the package site, and pkg would then do a reinstall because it detects the changed libraries? Yup, exactly. > Of course it is nice not to have to do a portrevision bump for such things, but that packages of the same name can still be different is something I must get used to. Well, shared libs is not the only problem there. Depending on a software it may or may not be possible to get the same package from two builds in the same env. A simple example - the software hardcodes the compilation date and time in its source. This only will make a build non-reproducible and every time there would be a different package. Fortunately, we don't have to reinstall the package each time it changes in the repository. pkg knows well when it really has to be reinstalled. But when using build-on-host tools you're actually bypassing this logic, because in this scenario pkg is used only to register the final package after the build finishes.
(In reply to Gleb Popov from comment #9) I am running kde, starting it with sddm. A more complete sequence of events: - I synced my ports tree from git just a few commits before the dbus upgrade, ran portmaster, and upgraded my laptop (the only machine actually running a UI; builds take place on a build machine in a terminal only). - I restarted the laptop, everything worked smoothly (not a smoothly as with qt5, bus smoothly enough) - I decided to do another git pull; this happened to be a few commits after the dbus upgrade - I reran portmaster (on the build machine), it only rebuilt dbus. - I reran pkg repo to publish the new package. - On my laptop, I ran pkg upgrade, which duly installed the new dbus. - I restarted the laptop, which proceeded to the terminal login. - I looked at the sddm startup log (in ~/.local/share/sddm) and saw lots of dbus library errors. - On a Linux laptop (a different machine), I filed the bug report. - Once the fix got committed, I started rebuilding. - ... portmaster ... pkg repo ... pkg upgrade ... - Some things would now start. Notable exceptions: plasma6-plasma-workspace, firefox; again with dbus loader errors - Rebuilt these as well, now the situation is better. The next bug report will be that py-qt5-pyqt does not compile anymore (at least when using portmaster). I guess I am now forced to use poudriere. -- Martin
(In reply to Gleb Popov from comment #7) thank you for elaborating. that aligns with my current understanding :phew: lol > First, bumps wouldn't help this issue. The service wouldn't be restarted anyways. .... A user should understand what he's updating and how to do that safely. i *now* understand *this* particular occurrence (the PR) as per your comment #4 turned out to not even be a valid case of the bumps lol > the pkg tool itself is also somewhat low-level. And using it again requires a user to be the "power user". yeah i definitely don't want our base tools to start doing all the "magic" that Ubuntu et al are doing to a live system to "assist" the sysadmin with their upgrades (and general work). i actually would prefer to shoot my own foot occasionally. higher level tools and/or more commands/functionality being added to pkg/ports are great. i enjoy freebsd because fewer interfaces break in a decade than seem to break every year on the linux platforms i work with > Just why you're reimplementing Poudriere? we've been using and updating this system since long before Poudriere was conceived and for our workflow it's better suited. anything we'd want of Poudriere it already did and then some. it'd be trivial to implement a change like the functionality we initially spoke of; in fact it's there already i'd simply have a policy decision to make regarding how cautious (DSO abi major+minor on just LIB_DEPENDS) and how deep (1 level). i, like many im sure, am just made complacent by the current modality of counting on PKGNAME to change but if i see a memo that's really stopping i'll gladly adjust my routine. sounds like thatll be further away than i initially interpreted your comment #2 to mean
(In reply to Martin Birgmeier from comment #10) > - I reran portmaster (on the build machine), it only rebuilt dbus. > - I reran pkg repo to publish the new package. Ooof, what an outrageos way to use portmaster as a Poudriere replacement. You should've mentioned that from the beginning. To be able to create a package built on one machine and install it on another, both these machine has to have **identical environment**. This can be easily achieved with Poudriere but almost impossible if you have two physical running systems. The dbus package you compiled has probably pulled in something from the build machine environment, which was not present on the laptop. And this is a real example of what I was talking about - Poudriere saves you from such mistakes. > I guess I am now forced to use poudriere. Yes, please. It will make your life easier (and consequently, life of other committers too). > in fact it's there already i'd simply have a policy decision to make regarding how cautious (DSO abi major+minor on just LIB_DEPENDS) and how deep (1 level). Just after my explanations that all this stuff is complex and you shouldn't reinvent the wheel, you just go ahead and create another one. Really, it brings me to tears. LIB_DEPENDS can't be source of truth, because the software may decide to link to whatever is really available in the system. And this uncertainty gets multiplied by building in an unclean environment. The correct solution is to scan STAGEDIR and parse ELF metadata to get a list of used libraries. I strongly advise you to reuse machinery we have in the framework (make stage-qa and scripts in Mk/) rather adding rolling your own logic.
(In reply to Gleb Popov from comment #12) > The correct solution is to scan STAGEDIR and parse ELF metadata to get a list of used libraries. I strongly advise you to reuse machinery we have in the framework (make stage-qa and scripts in Mk/) rather adding rolling your own logic. yes our tool leverages those very bits from pkgng and Mk, which are very composable. poudriere hasnt seemed as easy to hack on for our liking; i've evaluated switching every couple years. i tried synth as well the *_DEPENDS bunch like you said are not enforced and one off builds in random environments often have a mind of their own. i for one prefer to track such behaviors down and improve our ports' adherence to section 5.14.2 of the Porter's Book. the tool im using automates a great deal of that process with an unattended binary search for such culprit(s) but thats hardly why i use it. (In reply to Martin Birgmeier from comment #10) i'll try to figure out what's going on with py-qt5-pyqt, Martin. i've been waiting for you to post that PR :) did you solve that issue?
(In reply to Martin Birgmeier from comment #10) rebuilding devel/py-dbus allowed me to then build devel/py-qt[56]-pyqt successfully again. i was getting breakage 98% the way thru the build devel/py-dbus might be worth the bump. i'm no python expert but iirc this wouldnt be the first time a python binding broke regardless of SemVer suggesting it ought to go smoothly
(In reply to Gleb Popov from comment #12) You are citing me with words I did not say. Also, it is better to stay factual than to be sarcastic, because otherwise the reader might be led to think there is scant rationale for what the author is arguing for, and furthermore one might get a wrong impression about the latter's character. -- Martin
(In reply to Martin Birgmeier from comment #15) I just collapsed two replies into one, but I agree it turned out to be confusing, sorry about that.