Bug 252716 - databases/pgloader3: port now builds again + new maintainer
Summary: databases/pgloader3: port now builds again + new maintainer
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Christoph Moench-Tegeder
Keywords: patch-ready
Depends on:
Reported: 2021-01-15 17:43 UTC by Mario Duhanic
Modified: 2021-02-20 20:49 UTC (History)
3 users (show)

See Also:

shar archive of the port (2.05 KB, application/x-shellscript)
2021-01-15 17:43 UTC, Mario Duhanic
no flags Details
patch to 3.6.2 using diff -Nur (6.92 KB, patch)
2021-01-16 10:48 UTC, Mario Duhanic
no flags Details | Diff
new diff with cleared deps (8.23 KB, patch)
2021-02-15 14:04 UTC, Mario Duhanic
no flags Details | Diff
output of last stage of poudriere testport on 12.2 R amd64 (3.67 KB, text/plain)
2021-02-15 14:06 UTC, Mario Duhanic
no flags Details
proposed working patch to 3.6.2 with poudriere=ok, portlint=ok (29.95 KB, patch)
2021-02-20 16:20 UTC, Mario Duhanic
no flags Details | Diff
poudriere testport on 12.2 R amd64 (60.12 KB, text/plain)
2021-02-20 16:24 UTC, Mario Duhanic
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Mario Duhanic 2021-01-15 17:43:09 UTC
Created attachment 221604 [details]
shar archive of the port

pgloader builds again, using SBCL (also the original author's default), this time with one job only (-j > 1 breaks the build), there is no man page included. the curl dependency may be patched or replaced using a binary_alias with fetch(1) in a later version. there is no wrapper necessary. the port was without a maintainer so I am asking to adopt this port by putting in an email address of mine. shar is included.
Comment 1 Mario Duhanic 2021-01-16 10:48:17 UTC
Created attachment 221625 [details]
patch to 3.6.2 using diff -Nur

proposed patch using diff, works with patch -p0 from /usr/ports

now SBCL instead of CCL,
new maintainer (me, was none),
no wrapper anymore necessary

portlint: ok
make install, deinstall, reinstall, package: ok

Comment 2 Fernando Apesteguía freebsd_committer 2021-02-09 09:01:37 UTC
Hi Mario,

Thanks for the patch. A few notes to take into account:

Q/A:  /tmp/252716/databases/pgloader3/pkg-plist: There are only 1 items in the plist.  Consider using PLIST_FILES instead of pkg-plist when installing less than 6 items.
 /tmp/252716/databases/pgloader3/pkg-descr: includes lines that exceed 80 characters.
 Makefile: [23]: You may remove pkg-plist if you use PLIST_FILES and/or PLIST_DIRS.
 Makefile: COMMENT should begin with a capital, and end without a period
 files: empty directory should be removed.

^Triage: Please confirm this change passes QA (portlint, poudriere at least).

Comment 3 Christoph Moench-Tegeder freebsd_committer 2021-02-10 11:43:44 UTC
(In reply to Fernando Apesteguía from comment #2)
What's worse is the "curl dependency" thing in itself - fetching additional files during build is a no-go, everything we need should be in DISTFILES. Basically, poudriere (and by extension the package builders) do not have real network connectivity.
Comment 4 Mario Duhanic 2021-02-10 12:46:57 UTC
I will review the build process removing the curl dep and set up a new build host with poudriere.

thanks for your comments.
Comment 5 Mario Duhanic 2021-02-15 14:04:14 UTC
Created attachment 222468 [details]
new diff with cleared deps

pulling quicklisp from GH (could be put into MASTER_SITES pulling via https, too, but so more control over the used version),
all previously curl-pulled deps inside GH_TUPLE

diff -Nur /usr/ports/databases/pgloader3 databases/pgloader3
Comment 6 Mario Duhanic 2021-02-15 14:06:33 UTC
Created attachment 222469 [details]
output of last stage of poudriere testport on 12.2 R amd64

poudriere testport output (last stage),
build works in fresh poudriere jail 12.2 amd64,
binary runs as expected
Comment 7 Mario Duhanic 2021-02-15 14:11:29 UTC
So if/when this gets greenlighted (please note I've asked for becoming the maintainer) I will be looking forward to patch away the dynamic quicklisp system here.

Does this need a warning until then (quicklisp is a library maintenance system, maybe similar to CPAN and pulls a lot of stuff) or is this an absolute no-go?
Quicklisp is only used during the build time here.

Thanks for your comments!
Comment 8 Christoph Moench-Tegeder freebsd_committer 2021-02-15 16:53:13 UTC
(In reply to Mario Duhanic from comment #6)

How did you get the build that far? On my side, the build fails in quicklisp with 

:  debugger invoked on a SB-BSD-SOCKETS:HOST-NOT-FOUND-ERROR in thread
:    #<THREAD "main thread" RUNNING {10048E0673}>:
:  Name service error in "getaddrinfo": 8 (Name does not resolve)

which suggests that it still tries network access (which it just cannot do, if this should ever package).

And perhaps you could have a look into the portlint errors? ("poudriere testport" runs portlint for you). Just putting one blank line after the header goes a long way there; and the "issue" whith that one-line-plist still persists (using PLIST_FILES inside the Makefile is the preferred approach in that case - it was totally ok before you changed it, and @fernape already mentioned it).
Comment 9 Mario Duhanic 2021-02-18 09:57:56 UTC
You're right.

Initially I thought we could "just continue were the port broke", but I am also convinced to do it right and to do it more secure (I guess having deps in the MASTER_SITES is a safer way so security issues in one of the deps can be better tracked and resolved).

So I will wrap my head around the quicklisp system to replace it here with something static.

I could do so immediately as all deps of QL are easily retrievable with HTTP GETs over TLS but some tests showed that there might be problems in the future with the versions used as QL may not archive all versions for future retrieval. If that's true I will gather all sources from their original or mirrored usual repos (like GH or SF) and modify the GNU dist Makefile/ASDF to invoke all libs correctly. I am not a CL guru, though, but I am looking forward to do it and to become the maintainer of this port.

So can we leave it open here (or should it be closed?) and I'll upload a working diff with a no-network poudriere build (also I'll address what you've commented)?

Thanks for your time and sorry to waste it if I came too unprepared, as an excuse I thought it would be more straight-forward. But I've learned a lot.
Comment 10 Mario Duhanic 2021-02-20 16:20:24 UTC
Created attachment 222680 [details]
proposed working patch to 3.6.2 with poudriere=ok, portlint=ok

diff -Nur /usr/ports/databases/pgloader3 databases/pgloader3

Portlint: PASS
Poudriere: PASS, builds/installs/deinstalls in jail without network access (poudriere testport -j 122Ramd64 -o databases/pgloader3)
Binary: PASS
Comment 11 Mario Duhanic 2021-02-20 16:24:30 UTC
Created attachment 222681 [details]
poudriere testport on 12.2 R amd64
Comment 12 Mario Duhanic 2021-02-20 17:00:15 UTC
Some comments:

I started with having one or two MASTER_SITES with %SUBDIR% and lot of MASTER_SITE_SUBDIRs. But in the docs this is marked "deprecated" and this clashed with the GH macros.

Now all Quicklisp-dependencies are in the MASTER_SITES and the rest in GH_TUPLES, this in my opinions nicely reflects how the dist GNU Makefile's structure.

Digging around in this build showed that I needed to pull a lot of dependencies, as mentioned mostly staying with the QL system. Zach @ Quicklisp confirmed that "all lib versions stay there forever" so "freezing" this port in this state should work. That's why I chose staying with Quicklisp and staying a hacker and letting ql think it had pulled everything it needed. I'm a Lisp noob so I stopped doing that by patching QL, instead I discovered I could simply "touch" some files.

For that I wrote a small sh-shell script with bit of awk and sed here and tr and basename there to have all those RLN and MASTER_SITES etc.

Hopefully now everything is fine. As mentioned I learned some things here :)

Thanks for committing in case this is okay and for setting me as the new maintainer.
Comment 13 commit-hook freebsd_committer 2021-02-20 20:46:53 UTC
A commit references this bug:

Author: cmt
Date: Sat Feb 20 20:46:09 UTC 2021
New revision: 566189
URL: https://svnweb.freebsd.org/changeset/ports/566189

  pgloader3: unbreak and update to 3.6.2

  submitter becomes maintainer

  PR:		252716
  Submitted by:	Mario Duhanic

Comment 14 Christoph Moench-Tegeder freebsd_committer 2021-02-20 20:48:48 UTC
Looks good, "works for me" (with what simple test case I could come up with on a whim). I've taken the liberty to remove the CONFLICTS, as the "old" pgloader version 2 does not exist anymore in ports.
Comitted as ports r566189 - thanks for your work on this.