Bug 209515 - ports-mgmt/portupgrade: portsdb fails bacause of uninitialized constant
Summary: ports-mgmt/portupgrade: portsdb fails bacause of uninitialized constant
Status: New
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Bryan Drewery
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2016-05-15 07:44 UTC by tschweikle
Modified: 2021-10-08 15:41 UTC (History)
4 users (show)

See Also:
bugzilla: maintainer-feedback? (bdrewery)


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description tschweikle 2016-05-15 07:44:05 UTC
+ portsdb -u
portsdb: uninitialized constant PkgDBTools::DBError
++ exit 1
++ exit 1
Comment 1 tschweikle 2016-05-29 10:13:09 UTC
Fixed as of newer portupgrade and tools portupgrade depends on.
Comment 2 Chris Collins 2017-12-11 22:04:00 UTC
not fixed, getting myself on a 11.1 system.

Portupgrade crashed 80% through a rebuild all ports task, didnt show what was done so I got no idea what progress was made, and now generates this when rerunning it, even with just -h argument.

/usr/local/lib/ruby/site_ruby/2.4/pkgtools/pkgdbtools.rb:104:in `rescue in db_driver=': uninitialized constant PkgDBTools::DBError (NameError)
        from /usr/local/lib/ruby/site_ruby/2.4/pkgtools/pkgdbtools.rb:63:in `db_driver='
        from /usr/local/lib/ruby/site_ruby/2.4/pkgtools/portsdb.rb:168:in `setup'
        from /usr/local/lib/ruby/site_ruby/2.4/pkgtools/pkgtools.rb:242:in `init_pkgtools_global'
        from /usr/local/sbin/portupgrade:531:in `block in main'
        from /usr/local/lib/ruby/2.4/optparse.rb:1062:in `initialize'
        from /usr/local/sbin/portupgrade:238:in `new'
        from /usr/local/sbin/portupgrade:238:in `main'
        from /usr/local/sbin/portupgrade:2380:in `<main>'

Sadly portupgrade seems to be problematic with its db unresolved issues for years, with the main dev not active responding to issue reports, but portmaster is poor feature wise.  So I keep using it.

Any idea how to fix this one?

I already recompiled ruby, ruby-dbd and portupgrade.
Comment 3 Bryan Drewery freebsd_committer freebsd_triage 2018-01-24 18:22:21 UTC
I'll take another look sometime.
Comment 4 Chris Collins 2018-10-23 09:45:55 UTC
Still broken, on 26 servers currently.

But working on another 42 servers, I dont know what triggers it as all the servers are configured equally, just different hardware, some of the servers it originally worked but then breaks after a portupgrade crash.

Doesnt matter what ruby version is used or if portupgrade vs portupgrade-devel port.

Only flag set in make.conf is cputype.
Comment 5 Mikhail Teterin freebsd_committer freebsd_triage 2019-04-25 13:39:02 UTC
I'm having this now as well -- the problem started appearing in the middle of a portupgrade run. Using portupgrade-devel-20180309 thus:

         portupgrade -v -a -rR --config

I tie this to the upgrade of ruby24-bdb to ruby25-bdb, which portugprade itself has executed:

[...]
Apr 24 21:05:52 kachka pkg-static: ruby-2.5.5_1,1 installed
Apr 24 21:08:21 kachka pkg: ruby24-bdb-0.6.6_5 deinstalled
Apr 24 21:08:23 kachka pkg-static: ruby25-bdb-0.6.6_6 installed

After reinstalling portupgrade -- which switched the Ruby scripts from $PREFIX/bin/ruby to $PREFIX/bin/ruby25 -- things began to work again...

I'd say, portupgrade should implement special handling for when it upgrades itself OR any of the components it uses (ruby, ruby-bdb, etc.)
Comment 6 Chris Collins 2019-04-28 18:10:24 UTC
I finally did resolve it but since there was no interest shown developer side  I didnt post the followup.

In my case it was caused by something ruby side not liking hardening compile flags so ssp-buffer-size=4 and stack protector.

I had to remove from both ruby and ruby-bdb then it works as expected.

Really there needs to be some kind of built in diagnosis which would indicate the problem.  Nothing was actually visibly crashing so all seemed fine, but clearly something internally must have failed as it silently fell back to the other database type.

As to the flags I used I know they non default, but I would expect these days the vast majority of software to handle them, as is standard on many OS to use those flags now, FreeBSD is one of the few remaining hold outs.

I do feel I remember testing this on a empty make.conf, or at least with only cputype set, but I guess I am wrong given what I found to be the solution or maybe when I tested it, it was never with both ruby and ruby-bdb at the same time because they are both required to not have those flags not just one or the other.
Comment 7 Garance A Drosehn freebsd_committer freebsd_triage 2020-04-29 19:54:11 UTC
For what it's worth, this same problem hit me when upgrading a system running 13.0-CURRENT.  Here's the point in my session where things when haywire:

====> Compressing man pages (compress-man)
--->  Backing up the old version
--->  Uninstalling the old version
[Reading data from pkg(8) ... - 124 packages found - done]
--->  Deinstalling 'ruby-2.6.5,1'
--->  Preserving /usr/local/lib/libruby26.so.26 as /usr/local/lib/compat/pkg/libruby26.so.26
Updating database digests format: 100%
Checking integrity... done (0 conflicting)
Deinstallation has been requested for the following 1 packages (of 0 packages in the universe):

Installed packages to be REMOVED:
	ruby: 2.6.5,1

Number of packages to be removed: 1

The operation will free 38 MiB.
[1/1] Deinstalling ruby-2.6.5,1...
[1/1] Deleting files for ruby-2.6.5,1: 100%
[Reading data from pkg(8) ... - 123 packages found - done]
--->  Installing the new version via the port
===>  Deinstalling for ruby
===>   ruby not installed, skipping
===>  Installing for ruby-2.6.6,1
===>  Checking if ruby is already installed
===>   Registering installation for ruby-2.6.6,1 as automatic
Installing ruby-2.6.6,1...
Some of the standard commands are provided as separate ports for ease
of upgrading:

	devel/ruby-gems:	gem - RubyGems package manager
	devel/rubygem-irb:	irb - Interactive Ruby
	devel/rubygem-rake:	rake - Ruby Make
	devel/rubygem-rdoc:	rdoc - Ruby Documentation System

And some of the standard libraries are provided as separate ports
since they require extra dependencies:

	databases/rubygem-dbm:	DBM module
	databases/rubygem-gdbm:	GDBM module

Install them as occasion demands.

===> SECURITY REPORT: 
      This port has installed the following files which may act as network
      servers and may therefore pose a remote security risk to the system.
/usr/local/lib/ruby/2.6/amd64-freebsd13/socket.so

      If there are vulnerabilities in these programs there may be a security
      risk to the system. FreeBSD makes no guarantee about the security of
      ports included in the Ports Collection. Please type 'make deinstall'
      to deinstall the port if this is a concern.

      For more information, and contact details about the security
      status of this software, see the following webpage: 
https://www.ruby-lang.org/en/
===>  Cleaning for ruby-2.6.6,1
--->  Cleaning out obsolete shared libraries
Traceback (most recent call last):
	9: from /usr/local/sbin/portsclean:738:in `<main>'
	8: from /usr/local/sbin/portsclean:70:in `main'
	7: from /usr/local/sbin/portsclean:70:in `new'
	6: from /usr/local/lib/ruby/2.6/optparse.rb:1089:in `initialize'
	5: from /usr/local/sbin/portsclean:134:in `block in main'
	4: from /usr/local/lib/ruby/site_ruby/2.6/pkgtools/pkgtools.rb:242:in `init_pkgtools_global'
	3: from /usr/local/lib/ruby/site_ruby/2.6/pkgtools/portsdb.rb:168:in `setup'
	2: from /usr/local/lib/ruby/site_ruby/2.6/pkgtools/pkgdbtools.rb:100:in `db_driver='
	1: from /usr/local/lib/ruby/site_ruby/2.6/rubygems/core_ext/kernel_require.rb:54:in `require'
/usr/local/lib/ruby/site_ruby/2.6/rubygems/core_ext/kernel_require.rb:54:in `require': cannot load such file -- dbm (LoadError)
	8: from /usr/local/sbin/portsclean:738:in `<main>'
	7: from /usr/local/sbin/portsclean:70:in `main'
	6: from /usr/local/sbin/portsclean:70:in `new'
	5: from /usr/local/lib/ruby/2.6/optparse.rb:1089:in `initialize'
	4: from /usr/local/sbin/portsclean:134:in `block in main'
	3: from /usr/local/lib/ruby/site_ruby/2.6/pkgtools/pkgtools.rb:242:in `init_pkgtools_global'
	2: from /usr/local/lib/ruby/site_ruby/2.6/pkgtools/portsdb.rb:168:in `setup'
	1: from /usr/local/lib/ruby/site_ruby/2.6/pkgtools/pkgdbtools.rb:63:in `db_driver='
/usr/local/lib/ruby/site_ruby/2.6/pkgtools/pkgdbtools.rb:104:in `rescue in db_driver=': uninitialized constant PkgDBTools::DBError (NameError)
Comment 8 Garance A Drosehn freebsd_committer freebsd_triage 2020-04-29 20:19:40 UTC
#   That happened back on April 17th.  After that, any
#   attempt to run portupgrade ran into the trouble with:

/usr/local/lib/ruby/site_ruby/2.6/rubygems/core_ext/kernel_require.rb:54:in `require': cannot load such file -- dbm (LoadError)

#   or

/usr/local/lib/ruby/site_ruby/2.6/pkgtools/pkgdbtools.rb:104:in `rescue in db_driver=': uninitialized constant PkgDBTools::DBError (NameError)

#  (maybe not those pathnames, but I'd get one or both of
#  those errors msgs).  Today I did my standard:

  portsnap fetch && portsnap update && date && pkg version -vL =

#   I noticed there was a new verison of ruby26-bdb,
#   so I wanted to see what *would* happen if I tried
#   to update that (note: '-n'):
  portupgrade -Rn ruby26-bdb

#   That command died, at which point I started
#   googling around and found this bug-report.
#   So I then did:
  cd /usr/ports/ports-mgmt/portupgrade
  make && make install
  make deinstall && make clean

  cd /usr/ports/databases/ruby-bdb
  make && make install
  make deinstall && make clean
  pkg version -vL =

#   After the above 'portsnap' there was also a
#   new version of 'pkg'.  So having rebuilt both
#   portupgrade and ruby-bdb I tried to upgrade
#   that:
  portupgrade -R -n pkg

#   That update worked fine, so this machine
#   seems to be back in working order.  Thanks
#   to work-at-home orders, I may have a
#   VM snapshot of my system before the
#   update which triggered problems with
#   portupgrade, so I might be able to reproduce
#   the problem at will.  But I can't do that
#   right now, because that snapshot is at work
#   and everyone at work is ordered to stay home
#   for now...
Comment 9 Mikhail Teterin freebsd_committer freebsd_triage 2020-04-29 20:38:39 UTC
(In reply to Garance A Drosehn from comment #8)
> so this machine seems to be back in working order

It was back in working order right away. The problem in this ticket manifests itself, when portupgrade upgrades the Ruby itself -- while /the same/ ruby-interpreter continues to interpret the portupgrade-code.
Comment 10 Garance A Drosehn freebsd_committer freebsd_triage 2020-04-29 20:59:20 UTC
(In reply to Mikhail Teterin from comment #9)
> It was back in working order right away. The problem in
> this ticket manifests itself, when portupgrade upgrades
> the Ruby itself -- while /the same/ ruby-interpreter
> continues to interpret the portupgrade-code.

Not sure what you mean here.  Portupgrade remained broken on my system from April 17th until today.  I even tried making a few changes (by hand) to the scripts while trying to understand what the problem was.  The script kept failing, so I kept upgrading packages (when I needed to) via the trusty old "make && make install && make clean".

It wasn't until I rebuilt portupgrade and ruby-bdb that things worked again.

I should note that I didn't try to run portupgrade after rebuilding it.  I tried it only after rebuilding both it and ruby-bdb.  It might be that I didn't need to rebuild ruby-bdb to fix the problem.
Comment 11 Mikhail Teterin freebsd_committer freebsd_triage 2020-04-29 21:07:12 UTC
(In reply to Garance A Drosehn from comment #10)
> It wasn't until I rebuilt portupgrade and ruby-bdb that things worked again.

Sorry, I missed the ruby-bdb part -- yes, that is, what you had to rebuild after upgrading the ruby itself...

And portupgrade left it "hanging" after upgrading ruby, because portupgrade itself crashed.
Comment 12 Chris Collins 2021-08-19 00:30:17 UTC
An update since ruby26 has a unresolved security flag on it I tried to see if portupgrade would work on ruby27.

portsdb -u yields

portsdb: uninitialized constant PkgDBTools::DBError

portupgrade itself yields.  I did recompile ruby-bdb and portupgrade as well.

Traceback (most recent call last):
        9: from /usr/local/sbin/portupgrade:2380:in `<main>'
        8: from /usr/local/sbin/portupgrade:238:in `main'
        7: from /usr/local/sbin/portupgrade:238:in `new'
        6: from /usr/local/lib/ruby/2.7/optparse.rb:1093:in `initialize'
        5: from /usr/local/sbin/portupgrade:531:in `block in main'
        4: from /usr/local/lib/ruby/site_ruby/2.7/pkgtools/pkgtools.rb:242:in `init_pkgtools_global'
        3: from /usr/local/lib/ruby/site_ruby/2.7/pkgtools/portsdb.rb:168:in `setup'
        2: from /usr/local/lib/ruby/site_ruby/2.7/pkgtools/pkgdbtools.rb:100:in `db_driver='
        1: from /usr/local/lib/ruby/site_ruby/2.7/rubygems/core_ext/kernel_require.rb:54:in `require'
/usr/local/lib/ruby/site_ruby/2.7/rubygems/core_ext/kernel_require.rb:54:in `require': cannot load such file -- dbm (LoadError)
        8: from /usr/local/sbin/portupgrade:2380:in `<main>'
        7: from /usr/local/sbin/portupgrade:238:in `main'
        6: from /usr/local/sbin/portupgrade:238:in `new'
        5: from /usr/local/lib/ruby/2.7/optparse.rb:1093:in `initialize'
        4: from /usr/local/sbin/portupgrade:531:in `block in main'
        3: from /usr/local/lib/ruby/site_ruby/2.7/pkgtools/pkgtools.rb:242:in `init_pkgtools_global'
        2: from /usr/local/lib/ruby/site_ruby/2.7/pkgtools/portsdb.rb:168:in `setup'
        1: from /usr/local/lib/ruby/site_ruby/2.7/pkgtools/pkgdbtools.rb:63:in `db_driver='
/usr/local/lib/ruby/site_ruby/2.7/pkgtools/pkgdbtools.rb:104:in `rescue in db_driver=': uninitialized constant PkgDBTools::DBError (NameError)
Did you mean?  LoadError
Comment 13 Ted Milker 2021-10-08 15:41:57 UTC
I'm trying to upgrade a FreeBSD 11.4 system to 12.2 and I ran into this issue when rebuilding all ports near the end of the upgrade.  Here's the hoops I had to jump through to get pkg/portupgrade to a good state:

portupgrade --batch -f -R ruby*
# this will die with the uninitialized constant PkgDBTools::DBError (NameError) when it gets to the stage to re-install ruby27-bdb

pkg delete ruby27-bdb portupgrade
make -C /usr/ports/ports-mgmt/portupgrade clean install clean

# if ruby27-bdb gets rebuilt, everything breaks again, skip it
portupgrade --batch -f -R ruby* -x ruby27-bdb

Any other order of operations I tried results in a missing bdb.so error and then it's just broken, I can't reinstall portupgrade or ruby27-bdb.  I didn't bother trying too hard to figure it out from there because I could just revert to the prior VM snapshot.