Bug 259599 - devel/py-buildbot: Fails to run after sqlalchemy 1.4 update: AttributeError: module 'sqlalchemy.engine.strategies' has no attribute 'PlainEngineStrategy'
Summary: devel/py-buildbot: Fails to run after sqlalchemy 1.4 update: AttributeError: ...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: Normal Affects Many People
Assignee: Kai Knoblich
URL: https://cgit.freebsd.org/ports/commit...
Keywords: regression
: 260164 (view as bug list)
Depends on:
Blocks:
 
Reported: 2021-11-01 22:47 UTC by ari
Modified: 2022-02-27 23:44 UTC (History)
5 users (show)

See Also:
koobs: maintainer-feedback+
kai: merge-quarterly+


Attachments
buildbot 3.4.1 (9.81 KB, patch)
2022-02-17 06:14 UTC, Tomoyuki Sakurai
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description ari 2021-11-01 22:47:44 UTC
https://github.com/buildbot/buildbot/issues/5910



# /usr/local/etc/rc.d/buildbot check
Checking BuildBot config
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/buildbot/scripts/base.py", line 154, in getConfigFromTac
    exec(f.read(), tacGlobals)
  File "<string>", line 4, in <module>
  File "/usr/local/lib/python3.8/site-packages/buildbot/master.py", line 35, in <module>
    from buildbot.db import connector as dbconnector
  File "/usr/local/lib/python3.8/site-packages/buildbot/db/connector.py", line 31, in <module>
    from buildbot.db import enginestrategy
  File "/usr/local/lib/python3.8/site-packages/buildbot/db/enginestrategy.py", line 154, in <module>
    class BuildbotEngineStrategy(strategies.PlainEngineStrategy):
AttributeError: module 'sqlalchemy.engine.strategies' has no attribute 'PlainEngineStrategy'
Unable to load 'buildbot.tac' from '/var/buildbot':
Comment 1 Kubilay Kocak freebsd_committer freebsd_triage 2021-11-02 00:59:30 UTC
Thank you for the report Ari. Can you confirm the version update window this occurred with?
Comment 2 ari 2021-11-02 02:25:12 UTC
Sometime in the last few weeks. I see the sqlalchemy port went through some big changes just recently. So probably that.

From buildbot's issue page, looks like we need to pin buildbot to version 1.3.
Comment 3 Po-Chuan Hsieh freebsd_committer freebsd_triage 2021-11-02 14:28:30 UTC
Since py-sqlalchemy* conflicts with each other, I made the change from py-sqlalchemy12 to py-sqlalchemy14 following py-sqlalchemy-migrate change.

Without that, you won't be able to build buildbot due to the conflicts of dependencies.

After that, you get a package and there's a chance to work fine. I cannot do runtime check but it is better than nothing.

Anyway, the quick solution is to update buildbot to 3.*.
Comment 4 ari 2021-11-02 22:32:42 UTC
This change is not better than nothing. If buildbot had failed to build, I would not have wasted four hours working out what went wrong, missed a release deadline since buildbot died and everyone wouldn't now be scrambling to sort this out.

If you can't try a basic smoketest of the software, please don't change things.

A build failure would have properly alerted people who use this port to do something about it.

buildbot 3 is an option, but some notes in UPDATING would be appreciated for all the people who'll end up with broken configurations. Quite a few deprecated things were removed.
Comment 5 László Károlyi 2021-11-05 02:09:06 UTC
subscribing here, I bumped into the same problem.

upon investigating I found that buildbot has already 3.x available that would be compatible with the newer sqlalchemy module, but I haven't dived into how many breaking changes there are between 2.x and 3.x.

looking forward for a solution here.
Comment 6 ari 2021-11-05 02:35:36 UTC
https://docs.buildbot.net/latest/manual/upgrading/3.0-upgrade.html is quite useful. I've not tried it yet.

Either someone needs to create a sqlalchemy13 port or we all plunge forward into buildbot 3.
Comment 7 László Károlyi 2021-11-08 21:08:03 UTC
I've solved the problem with installing buildbot 3 from pip into a jail, within a user directory. Completed some scripts for it, stole and adjusted the pkg-originated startup script, works like a charm now.
Comment 8 ari 2021-11-16 01:06:32 UTC
Is this port abandoned? It seems like a long time to not roll back the changes to a completely broken port.
Comment 9 László Károlyi 2021-11-16 09:14:00 UTC
(In reply to ari from comment #8)
quite honestly, nothing holds you back from becoming the maintainer of this package.
Comment 10 ari 2021-11-17 00:08:13 UTC
(In reply to László Károlyi from comment #9)

Sure. Would you like to contribute your buildbot3 scripts you developed so we can upgrade this port?
Comment 11 ari 2021-11-17 02:48:05 UTC
I've had a go at it: https://gist.github.com/ari/628005d03bfdefb82e31bf80bd79eeb9

Stuck at this error when using testport in poudriere...


Installing buildbot script to /wrkdirs/usr/ports/devel/py-buildbot/work-py38/stage/usr/local/bin
writing list of installed files to '/wrkdirs/usr/ports/devel/py-buildbot/work-py38/.PLIST.pymodtmp'
===> Creating unique files: Move MAN files needing SUFFIX
===> Creating unique files: Move files needing SUFFIX
Move: bin/buildbot --> bin/buildbot-3.8
Link: @bin/buildbot --> bin/buildbot-3.8
Move: bin/buildbot --> bin/buildbot-3.8
Link: @bin/buildbot --> bin/buildbot-3.8
install: /wrkdirs/usr/ports/devel/py-buildbot/work-py38/stage/usr/local/bin/buildbot-3.8: realpath: Too many levels of symbolic links
*** Error code 71



Any thoughts?
Comment 12 Kubilay Kocak freebsd_committer freebsd_triage 2021-11-17 03:34:35 UTC
(In reply to ari from comment #8)

Not abandoned, and apologies for the issues the regression caused. I've requested resolution via an sqlalchemy13 port from sunpoet offline while we get a 3.x update sorted.
Comment 13 László Károlyi 2021-11-17 09:36:14 UTC
(In reply to ari from comment #10)
I can, but you wouldn't be able to go far with it. I use ansible with custom made playbooks and a jailhost playbook that I created: https://gitea.ksol.io/karolyi/ansible-freebsd-jailhost-tools

This unfortunately can not be used to build packages.
Comment 14 ari 2021-11-22 21:33:01 UTC
This is causing me a lot of grief. Perhaps I'm the only person here actively using buildbot so no one is actively working on it, but if there is anyone else hitting this issue I'd appreciate some assistance with my first cut of the buildbot3 port.

What is going on with

install: /wrkdirs/usr/ports/devel/py-buildbot/work-py38/stage/usr/local/bin/buildbot-3.8: realpath: Too many levels of symbolic links
Comment 15 ari 2021-11-29 23:00:18 UTC
Should this port just be marked BROKEN and removed from FreeBSD ports? If there is no interest in reverting the breakage or upgrading to version 3 based on the gist I already created, perhaps we should accept that its just broken now.
Comment 16 Kubilay Kocak freebsd_committer freebsd_triage 2021-11-30 01:20:14 UTC
@Sunpoet. Could you please:

 - fork sqlalchemy-migrate into a new sqlalchemy12-migrate port that depends on sqlalchemy12, effectively reverting ports e3f7785e1794, and
 - have the buildbot port/ports depend on that new port.
Comment 17 ari 2022-01-06 03:16:41 UTC
Perhaps it is time to just remove this port from FreeBSD ports?
Comment 18 Tomoyuki Sakurai 2022-02-17 06:14:52 UTC
Created attachment 231882 [details]
buildbot 3.4.1

update to 3.4.1. build succeeded on 14-CURRENT amd64 and 13.0-RELEASE amd64. poudriere did not complain with "-t" flag. portlint is happy. runs fine on 13.0.

my private ports overlay is at:
https://github.com/trombik/freebsd-ports-buildbot

please test. thanks.
Comment 19 ari 2022-02-19 04:54:51 UTC
(In reply to Tomoyuki Sakurai from comment #18)

That's brilliant. I've tried your ports, built in poudriere on Freebsd 13 and everything worked perfectly.
Comment 20 Tomoyuki Sakurai 2022-02-19 05:28:29 UTC
the next step is to find someone with commit bit. the PR has been stacked long enough to justify maintainer timeout. there might be bugs in my efforts, but the ports have been broken.

hope someone would triage the PR.
Comment 21 ari 2022-02-22 23:58:20 UTC
*** Bug 260164 has been marked as a duplicate of this bug. ***
Comment 22 Kubilay Kocak freebsd_committer freebsd_triage 2022-02-23 01:44:01 UTC
Tomoyuki, thank you for the well commented/test patch, was hoping sunpoet could take care of the regression, but it may have fell of the radar. Will try to get this committed/merged as soon as possible.
Comment 23 ari 2022-02-25 03:06:37 UTC
Kubilay, given that this port has been completely broken since last October please commit the patch and lets make further improvements later. Nothing could be worse than it is today.
Comment 24 commit-hook freebsd_committer freebsd_triage 2022-02-25 06:43:04 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=81cb513d26abb0060b971366db8f3ee0ac9bd3eb

commit 81cb513d26abb0060b971366db8f3ee0ac9bd3eb
Author:     Tomoyuki Sakurai <trombik1973@gmail.com>
AuthorDate: 2022-02-25 06:36:21 +0000
Commit:     Kai Knoblich <kai@FreeBSD.org>
CommitDate: 2022-02-25 06:36:21 +0000

    devel/py-buildbot*: Update to 3.4.1

    * This also should fix the regression caused after the switch from
      py-sqlalchemy12 to py-sqlalchemy14 in e3f7785e1794.

    Changelog since 2.10.5:

    https://github.com/buildbot/buildbot/blob/v3.4.1/master/docs/relnotes/index.rst

    PR:             259599
    Reviewed by:    koobs (maintainer)
    Approved by:    koobs (maintainer, implicit: MAINTAINER_POLICY)
    MFH:            2022Q1 (fix regression with py-sqlalchemy14)

 devel/py-buildbot-console-view/Makefile       |  2 +-
 devel/py-buildbot-console-view/distinfo       |  6 +++---
 devel/py-buildbot-grid-view/Makefile          |  2 +-
 devel/py-buildbot-grid-view/distinfo          |  6 +++---
 devel/py-buildbot-pkg/Makefile                |  2 +-
 devel/py-buildbot-pkg/distinfo                |  6 +++---
 devel/py-buildbot-waterfall-view/Makefile     |  2 +-
 devel/py-buildbot-waterfall-view/distinfo     |  6 +++---
 devel/py-buildbot-worker/Makefile             |  9 ++++++++-
 devel/py-buildbot-worker/distinfo             |  6 +++---
 devel/py-buildbot-www/Makefile                |  2 +-
 devel/py-buildbot-www/distinfo                |  6 +++---
 devel/py-buildbot/Makefile                    | 20 +++++++++++++++-----
 devel/py-buildbot/distinfo                    |  6 +++---
 devel/py-buildbot/files/patch-setup.py (gone) | 11 -----------
 15 files changed, 49 insertions(+), 43 deletions(-)
Comment 25 commit-hook freebsd_committer freebsd_triage 2022-02-25 06:49:07 UTC
A commit in branch 2022Q1 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=fb9e7bde0ae442f8d4bf369dad8a3ed557b89c84

commit fb9e7bde0ae442f8d4bf369dad8a3ed557b89c84
Author:     Tomoyuki Sakurai <trombik1973@gmail.com>
AuthorDate: 2022-02-25 06:36:21 +0000
Commit:     Kai Knoblich <kai@FreeBSD.org>
CommitDate: 2022-02-25 06:46:03 +0000

    devel/py-buildbot*: Update to 3.4.1

    * This also should fix the regression caused after the switch from
      py-sqlalchemy12 to py-sqlalchemy14 in e3f7785e1794.

    Changelog since 2.10.5:

    https://github.com/buildbot/buildbot/blob/v3.4.1/master/docs/relnotes/index.rst

    PR:             259599
    Reviewed by:    koobs (maintainer)
    Approved by:    koobs (maintainer, implicit: MAINTAINER_POLICY)
    MFH:            2022Q1 (fix regression with py-sqlalchemy14)

    (cherry picked from commit 81cb513d26abb0060b971366db8f3ee0ac9bd3eb)

 devel/py-buildbot-console-view/Makefile       |  2 +-
 devel/py-buildbot-console-view/distinfo       |  6 +++---
 devel/py-buildbot-grid-view/Makefile          |  2 +-
 devel/py-buildbot-grid-view/distinfo          |  6 +++---
 devel/py-buildbot-pkg/Makefile                |  2 +-
 devel/py-buildbot-pkg/distinfo                |  6 +++---
 devel/py-buildbot-waterfall-view/Makefile     |  2 +-
 devel/py-buildbot-waterfall-view/distinfo     |  6 +++---
 devel/py-buildbot-worker/Makefile             |  9 ++++++++-
 devel/py-buildbot-worker/distinfo             |  6 +++---
 devel/py-buildbot-www/Makefile                |  2 +-
 devel/py-buildbot-www/distinfo                |  6 +++---
 devel/py-buildbot/Makefile                    | 20 +++++++++++++++-----
 devel/py-buildbot/distinfo                    |  6 +++---
 devel/py-buildbot/files/patch-setup.py (gone) | 11 -----------
 15 files changed, 49 insertions(+), 43 deletions(-)
Comment 26 Kai Knoblich freebsd_committer freebsd_triage 2022-02-25 06:53:41 UTC
Committed to the main branch and MFH'ed to 2022Q1 as well!  Thank you, Tomoyuki, for the patch!

I'll leave this bug a few days open for reference.
Comment 27 Kubilay Kocak freebsd_committer freebsd_triage 2022-02-25 23:20:51 UTC
@Ari Let us know if everything is good after this commit/merge
Comment 28 ari 2022-02-27 23:44:23 UTC
Thanks for merging this. Since I reported it, I'll close this task as complete.