Bug 265105

Summary: textproc/refdb: update to 1.0.3
Product: Ports & Packages Reporter: O. Hartmann <ohartmann>
Component: Individual Port(s)Assignee: Michael Gmelin <grembo>
Status: Closed Overcome By Events    
Severity: Affects Many People CC: fernape, grembo
Priority: --- Keywords: needs-qa
Version: Latest   
Hardware: Any   
OS: Any   
Bug Depends on: 265740    
Bug Blocks: 253506    
Attachments:
Description Flags
refdb v 1.0.3
none
refdb patch v 1.0.3
none
Patch refd 1.0.3
none
Patch refdb 1.0.3
fernape: maintainer-approval+
Refdb update to 1.0.3, fix setting of OPTIONS
none
Patch refDB v1.0.3
none
refdb 1.0.3 patch
none
refdb 1.0.3 patch
none
refdb 1.0.3 patchset
none
refdb 1.0.3 patchset
none
Patch example
fernape: maintainer-approval?
refdb 1.0.3 patchset
none
textproc/refdb 1.0.3 patch
none
textproc/refdb 1.0.3 patch
none
textproc/refdb 1.0.3 patch
none
Working patch
none
textproc/refdb - for your considerations
none
textproc/refdb-1.0.3 Patchset for considerations
none
Patch refdb-1.0.3, revision
none
Patchset refdb-1.0.3
none
Update port refdb to 1.0.3
none
Update port textproc/refdb 1.0.3
none
Upgrade refdb-1.0.3
none
Upgrade refdb-1.0.3
none
correct run dependencies none

Description O. Hartmann 2022-07-09 06:47:30 UTC
Created attachment 235143 [details]
refdb v 1.0.3

Attached is a patch to provide textproc/refdb version 1.0.3.

Tested with PostgreSQL 13 and 14 on 14-CURRENT and 13-STABLE as well a s13.1-RELENG. Also slight sqilite tests. MySQL needs to be tested. MariaDB untested.
Comment 1 Bugzilla Automation freebsd_committer freebsd_triage 2022-07-09 06:47:30 UTC
Maintainer informed via mail
Comment 2 O. Hartmann 2022-07-09 08:12:23 UTC
Created attachment 235144 [details]
refdb patch v 1.0.3

Building docs is a kind of complicated and exhausting, so disable building docs by default until this issue is fixed.
Comment 3 Fernando Apesteguía freebsd_committer freebsd_triage 2022-07-12 06:07:38 UTC
Hummm... the patch updates a bunch of ports... Would you mind checking it?

Thanks!
Comment 4 O. Hartmann 2022-07-12 06:16:26 UTC
(In reply to Fernando Apesteguía from comment #3)

Hello. 
Can you please be more specific?
Comment 5 Fernando Apesteguía freebsd_committer freebsd_triage 2022-07-12 06:19:51 UTC
(In reply to O. Hartmann from comment #4)
Look at the attached patch. It contains updates for multiple ports. The patch starts like this:

diff --git a/devel/opencl/Makefile b/devel/opencl/Makefile
index 506049f58925..a7f18a0f85e9 100644
--- a/devel/opencl/Makefile
+++ b/devel/opencl/Makefile
@@ -1,5 +1,5 @@
 PORTNAME=	opencl
-PORTVERSION=	3.0.8
+PORTVERSION=	3.0.11
...
...

that doesn't seem to have anything to do with textproc/refdb.
Comment 6 O. Hartmann 2022-07-12 07:44:51 UTC
(In reply to Fernando Apesteguía from comment #5)

Oh, very sorry ... my bad ... :-(
Comment 7 O. Hartmann 2022-07-12 07:55:15 UTC
Created attachment 235207 [details]
Patch refd 1.0.3

The attached patch corrects some mistakes made while updating the development repository.
Comment 8 Fernando Apesteguía freebsd_committer freebsd_triage 2022-07-12 11:15:47 UTC
Q/A: sqlite, mysql and pgsql are specified in both USES+= and also in MYSQL_USES, SQLITE_USES, PGSQL_USES. The former is not necessary.

Q/A: It would be nice to have some patches regenerated so they conform to the expected format:

WARN: textproc/refdb/files/patch-doc__Makefile.am: patch was not generated using ``make makepatch''.  It is recommended to use ``make makepatch'' when you need to [re-]generate a patch to ensure proper patch format.
WARN: textproc/refdb/files/patch-phpweb-index.php.in: patch was not generated using ``make makepatch''.  It is recommended to use ``make makepatch'' when you need to [re-]generate a patch to ensure proper patch format.
WARN: textproc/refdb/files/patch-scripts-refdb.in: patch was not generated using ``make makepatch''.  It is recommended to use ``make makepatch'' when you need to [re-]generate a patch to ensure proper patch format.
WARN: textproc/refdb/files/patch-scripts-refdb_latex2utf8txt: patch was not generated using ``make makepatch''.  It is recommended to use ``make makepatch'' when you need to [re-]generate a patch to ensure proper patch format.

Thanks!
Comment 9 O. Hartmann 2022-07-12 11:55:13 UTC
Created attachment 235213 [details]
Patch refdb 1.0.3

Attached, the corrected patchfile.
Comment 10 Michael Gmelin freebsd_committer freebsd_triage 2022-07-15 17:16:00 UTC
Created attachment 235276 [details]
Refdb update to 1.0.3, fix setting of OPTIONS

Please see the attached fix to solve the ports qa issue (as discussed on the ML).
Didn't obsolete the other patch yet.
Comment 11 Fernando Apesteguía freebsd_committer freebsd_triage 2022-07-15 18:56:42 UTC
Thanks Michael,

That passes the DEVELOPER=yes sanity check. Apart from that, I had to:

* Add USES=gettext, otherwise, we get:
  macro 'AM_ICONV_LINK' not found in library
* Rename SQLITE option helper (the option is SQLITE, but the helpers where SQLITE3_...)

And now I get:

Warning: Possible REINPLACE_CMD issues:
- - REINPLACE_CMD ran, but did not modify file contents: doc/Makefile.am
====> Checking for pkg-plist issues (check-plist)
===> Parsing plist
===> Checking for items in STAGEDIR missing from pkg-plist
===> Checking for items in pkg-plist which are not in STAGEDIR
Error: Missing: etc/rc.d/refdbd
===> Error: Plist issues found.
*** Error code 1

So I had to remove etc/rc.d/refdbd from pkg-plist.

So please, can the OP run some Q/A with poudriere and/or portlint && portclippy?

Thanks!
Comment 12 O. Hartmann 2022-07-18 08:14:58 UTC
(In reply to Michael Gmelin from comment #10)

Hello

sorry for the delay. I will have a look at this. Thank you for taken your time to support me.
Comment 13 Fernando Apesteguía freebsd_committer freebsd_triage 2022-07-18 08:37:25 UTC
(In reply to O. Hartmann from comment #12)
No problem, just make sure to use any of the tools I mentioned. They are great tools!
Comment 14 O. Hartmann 2022-08-06 09:27:37 UTC
Created attachment 235716 [details]
Patch refDB v1.0.3

Attached you'll find another attempt of the patch, this time I took the liberty to incorporate all suggestions. I had to make some minor changes compared to my initial patch, those revealed themselfes after I was able to check via poudriere (I first had to overcome the undocumented "overlay" problemacy with poudriere-devel, sorry that it took so long). One point is that a specific XML binary is not automatically installed when it is NOT reflected in the BUILD_DEPEND list. 

I also was able to overcome somehow the problems with building the documentation, see for yourselfs.

I followed the steps suggested at the wesite for porters (handbook). "make stage-qa" still drops out with an error:

[...]

 root@thor:/pool/ports.dev/textproc/refdb # make stage-qa
====> Running Q/A tests (stage-qa)
Error: 'share/sgml/catalog.ports' is referring to /pool/ports.dev/textproc/refdb/work/stage
Warning: port uses /usr/local/var instead of /var
Notice: You have some Perl modules as dependencies but you do not have devel/p5-Module-CoreList installed, the perlcore QA check gets better results when using it, especially with older Perl versions.
*** Error code 1

Stop.
make: stopped in /pool/ports.dev/textproc/refd
[...]

but building the package as non-root also works fine as installing and deinstalling the packages - as long as the option for building docs is enabled (did not test the controversal setting).

Poudriere built the port from a fresh start on 13.1-RELENG-p0, and with a preexisting poudriere ports tree on CURRENT and 13-STABLE.
Comment 15 O. Hartmann 2022-08-06 09:32:19 UTC
(In reply to Fernando Apesteguía from comment #11)

Where to find portclippy? FreeBSD's ports tree does not have such a named tool ...
Comment 16 O. Hartmann 2022-08-06 16:48:52 UTC
There is still a minor problem with --localstatedir, which stage-qa is reporting to be set to /usr/local/var.
For now, I "hardcoded" /var with CONFIGURE_ARGS+= ... --localstatedir=/var, could't find the proper PREFIX-LOCALSTATEDIR variable used by the FreeBSD ports system (missing a kind of Index / Glossary for such things).
Comment 17 Fernando Apesteguía freebsd_committer freebsd_triage 2022-08-06 18:07:36 UTC
(In reply to O. Hartmann from comment #15)
It is part of ports-mgmt/portfmt
Comment 18 O. Hartmann 2022-08-06 19:14:09 UTC
Created attachment 235725 [details]
refdb 1.0.3 patch

Some minor corrections were necessary.

Insted of applying --localstatedir=/var to Makefile's CONFIGURE_ARGS I decided to patch work/refdb-1.0.3/configure and provide a patchfile. It seems to handle make makeplist easier (in my case).

Further, I followed the handbook and added OPTIONS_SUB=yes which should then provide commenting out all stuff introduced when building with OPTIONS DOCS set to yes - when understood correctly, one has to prefix those additional lines in pkg-plist with %%DOCS%% - I'm not sure, the ports system gets more confusing every time something is added ...

Next round of avaluation  ...
Comment 19 O. Hartmann 2022-08-06 19:16:24 UTC
Created attachment 235726 [details]
refdb 1.0.3 patch

Delete a remnant patch file
Comment 20 Fernando Apesteguía freebsd_committer freebsd_triage 2022-08-08 15:14:05 UTC
Hi again,

Would you mind integrating the changes from grembo@? The last patch is broken in the same way we discussed in comment #8

Thanks!
Comment 21 O. Hartmann 2022-08-08 17:14:15 UTC
??

: make makepatch
Generated patch-configure (ONLY THIS ONE?)
Generated patch-doc_Makefile.am >> patch-doc__Makefile.am (legacy)
Generated patch-phpweb_index.php.in >> patch-phpweb-index.php.in (legacy)
Generated patch-scripts_refdb.in >> patch-scripts-refdb.in (legacy)
Generated patch-scripts_refdb__latex2utf8txt >> patch-scripts-refdb_latex2utf8txt (legacy)
patch-configure only contains metadata changes; not replacing
patch-doc__Makefile.am only contains metadata changes; not replacing
patch-phpweb-index.php.in only contains metadata changes; not replacing
patch-scripts-refdb.in only contains metadata changes; not replacing
patch-scripts-refdb_latex2utf8txt only contains metadata changes; not replacing
Comment 22 O. Hartmann 2022-08-08 18:46:23 UTC
Created attachment 235782 [details]
refdb 1.0.3 patchset

Last attempt.
Comment 23 Michael Gmelin freebsd_committer freebsd_triage 2022-08-08 19:12:38 UTC
(In reply to O. Hartmann from comment #22)

> Last attempt

Why? You're so close now :)

There's at least one change from comment #11 missing:

> Rename SQLITE option helper (the option is SQLITE, but the helpers
> where[sic] SQLITE3_...)

So basically you either rename the option to "SQLITE3", or, probably better, change those two lines:

SQLITE3_USES=       sqlite
SQLITE3_CONFIGURE_WITH= sqlite3

to

SQLITE_USES=       sqlite
SQLITE_CONFIGURE_WITH= sqlite3
Comment 24 O. Hartmann 2022-08-08 19:32:08 UTC
Created attachment 235784 [details]
refdb 1.0.3 patchset

...
Comment 25 Fernando Apesteguía freebsd_committer freebsd_triage 2022-08-08 20:58:22 UTC
(In reply to O. Hartmann from comment #24)
Thanks for the work. Sorry I did not explain myself better before. Things that need to be done with current patch (" refdb 1.0.3 patchset").

Pass portlint:
FATAL: Makefile: [29]: use a tab (not space) after a variable name
FATAL: Makefile: [30]: use a tab (not space) after a variable name
FATAL: Makefile: [52]: use a tab (not space) after a variable name
FATAL: Makefile: [62]: use a tab (not space) after a variable name
FATAL: Makefile: [63]: use a tab (not space) after a variable name
FATAL: Makefile: [72]: use a tab (not space) after a variable name
FATAL: Makefile: [73]: use a tab (not space) after a variable name
FATAL: Makefile: [82]: use a tab (not space) after a variable name
FATAL: Makefile: [83]: use a tab (not space) after a variable name
WARN: Makefile: CATALOG appears in PORT_OPTIONS:M, but is not listed in OPTIONS_DEFINE.
WARN: Makefile: Consider adding support for a NLS knob to conditionally disable gettext support.
WARN: Makefile: the port uses Java but is not part of the ``java'' category
WARN: Makefile: "IGNORE" has to appear earlier.
WARN: Makefile: "LIB_DEPENDS" has to appear earlier.
WARN: Makefile: "BUILD_DEPENDS" has to appear earlier.
WARN: Makefile: "RUN_DEPENDS" has to appear earlier.
WARN: Makefile: wrong dependency value for BUILD_DEPENDS. BUILD_DEPENDS requires 2 or 3 colon-separated tuples.
9 fatal errors and 8 warnings found.

Pass portclippy:
 portclippy Makefile 
# PORTNAME block
PORTNAME
PORTVERSION
CATEGORIES
MASTER_SITES

# Maintainer block
MAINTAINER
COMMENT

# License block
LICENSE
LICENSE_FILE

-USES

# Dependencies
+BUILD_DEPENDS
LIB_DEPENDS
RUN_DEPENDS
-BUILD_DEPENDS

# USES block
+USES
+USE_RC_SUBR

# USES=shebangfix related variables
+SHEBANG_FILES

# Configure block
HAS_CONFIGURE
+CONFIGURE_ARGS

# Make block
INSTALL_TARGET

-PORTDOCS

# CFLAGS/CXXFLAGS/LDFLAGS block
CFLAGS
LIBS

-CONFIGURE_ARGS
# Packaging list block
+PORTDOCS

# Options definitions
OPTIONS_DEFINE
-OPTIONS_SUB
+OPTIONS_DEFAULT
OPTIONS_SINGLE
OPTIONS_SINGLE_DB
-OPTIONS_DEFAULT
+OPTIONS_SUB

# Options helpers
DOCS_BUILD_DEPENDS

-SHEBANG_FILES

-USE_RC_SUBR

Apply suggestion by grembo@ by which we move MYSQL_USES, SQLITE_USES, etc *outside* the .if block and put them in the options block along with the MYSQL_CONFIGURE_WITH, PGSQL_CONFIGURE_WITH, etc...

And then, we can test the whole thing in poudriere :-)
Comment 26 Fernando Apesteguía freebsd_committer freebsd_triage 2022-08-08 21:09:51 UTC
Created attachment 235785 [details]
Patch example

This is a rework of the patch. It is untested yet so I did not deprecate the previous one. I'll try to build test it tomorrow.
Comment 27 O. Hartmann 2022-08-09 13:55:19 UTC
Created attachment 235800 [details]
refdb 1.0.3 patchset

Next "last attempt".

Mid air collision. I was already changing the patchset towards the expectations, when the example showed up. I didn't check whether I met all changes made in that provided example, but I hope.

Additionaly, following the suggestions of the porter's handbook, the post-install message has been moved into a pkg-message file.

The patchset compiles with poudriere on a 13.1-RELEASE-p0 jail.
Comment 28 Fernando Apesteguía freebsd_committer freebsd_triage 2022-08-09 14:59:22 UTC
(In reply to O. Hartmann from comment #27)
Hi there,

Thanks for the patch. I'm a bit puzzled though. Did you actually use portlint before submitting?

$ portlint -AC
FATAL: Makefile: [30]: use a tab (not space) after a variable name
FATAL: Makefile: [34]: use a tab (not space) after a variable name
FATAL: Makefile: [38]: use a tab (not space) after a variable name
FATAL: Makefile: [52]: use a tab (not space) after a variable name
FATAL: Makefile: [53]: use a tab (not space) after a variable name
FATAL: Makefile: [54]: use a tab (not space) after a variable name
FATAL: Makefile: [55]: use a tab (not space) after a variable name
FATAL: Makefile: [56]: use a tab (not space) after a variable name
FATAL: Makefile: [57]: use a tab (not space) after a variable name
FATAL: Makefile: [58]: contiguous blank lines (> 1 lines) found.
Comment 29 Michael Gmelin freebsd_committer freebsd_triage 2022-08-09 16:42:17 UTC
How is this overcome by events? The update is still necessary and there is a ready to test patch here.

@Oliver Would it make sense for committers to finish the patch completely and you test to make sure that it works for you as a user and maintainer of the port? I think once it has been cleaned up, maintaining it for smaller updates will be much easier for you.

Also, for future work, would it make sense to talk a bit about port workflows, tools (linting, checking etc)?

I'm changing this back to in-progress and claim it, as I think what the PR is asking for is still valid and required and we will sort it out/steer it home somehow.
Comment 30 Michael Gmelin freebsd_committer freebsd_triage 2022-08-09 16:45:12 UTC
p.s. I think this is one of those things where having a review in Phabricator (reviews.freebsd.org) would have been a lot more productive to clean up things in an iterative and more interactive way (uploading new patches to bugzilla is a time-consuming and sometimes frustrating workflow).
Comment 31 Michael Gmelin freebsd_committer freebsd_triage 2022-08-09 16:58:15 UTC
(In reply to Michael Gmelin from comment #29)

@O Hartmann:
> Depends on: 265740

Well, that's one way to answer :(
Comment 32 O. Hartmann 2022-08-09 17:29:50 UTC
(In reply to Michael Gmelin from comment #31)

My apology, I forgot to refer to that PR in the first place and it is NOT considered to be an answer.

My resources are very limited, I really appreciated your help and assistance; but in the past years maintaining a port/ports have become a kind of "artwork" where the arrangement of spaces obviously became more important than having a working port at hand. 

I can not exactly recall when I introduced this port, I used to use it in my time as a scientist and with another great piece of software vanished from ports not long ago, jabref, refdb is a formidable piece of software maintainig larger TeX bibliographies. Personally I use refdb as a private port since 2012/14 on the basis of a crude port, a couple of months ago I convinced Markus Hoenicka, the upstream developer and maintainer of the port to push another tarball with all the good stuff in it since release 0.9 - so you have 1.0.3 now.

Despite this, the change of the LLVM basis also introduced a certain linker error also hitting a simple declaration in the 0.9 release of the port - which I tried to fix via patch, but nobody took notice of that. Somehow this is a contrast to that what is going on right now, where the arrangement of whitespaces becomes an issue.
Comment 33 O. Hartmann 2022-08-09 17:31:54 UTC
And I'm sorry for the inconvenience.
Comment 34 Fernando Apesteguía freebsd_committer freebsd_triage 2022-08-09 20:58:23 UTC
Sorry for this bad experience, Oliver.

Spaces are not more important than having a working port. Having a working port means the spaces issue should be non-existent. The ports framework is very complex and based on Makefiles. That is a double challenge. The only way we can ensure that we grow sustainably is by taking the utmost care with the ports. We have over 30K different ports. It helps a lot if things are where they are expected to be in the format they are expected to have.

Using tools like portlint and portclippy is not to make maintainers suffer a miserable life, quite the contrary. Ports sometimes need a big update or a complete rework that forces us to change and reorder many things, but it is for the sake of consistency and avoiding bugs that we use these tools.

The use of poudriere is _a must_. The patch that is attached to this PR fails in poudriere with:

====> Running Q/A tests (stage-qa)
Error: 'share/sgml/catalog.ports' is referring to /wrkdirs/usr/ports/textproc/refdb/work/stage

The catalog.ports file has two lines referencing absolute paths pointing to directories in the stage directory. That will work in local, but we can not create packages with that. Setting up poudriere is not that difficult, but I acknowledge that it can be a bit resource hungry. But it is worth the effort. It catches many bugs by building in a clean environment.

I hope you would reconsider dropping maintainership of this port.

P.S: I'll try to find where the issue with catalog.ports is and finish this update.
Comment 35 O. Hartmann 2022-08-10 06:07:41 UTC
(In reply to Fernando Apesteguía from comment #28)


This is not portclippy ...
Comment 36 O. Hartmann 2022-08-10 06:19:35 UTC
Created attachment 235819 [details]
textproc/refdb 1.0.3 patch

I was working on eliminating that .if block for DOCS when I went nuclear.

Attached, you'll find the attempt to eliminate this block according the new port's conformity rules. 
Issues: 

I do not find adequate descriptions how to solve JAVA_BUILD=YES and JAVA_VERSION=8+.
The JAVA_BUILD portion is translated towards

DOCS_USES=              java:build

I did this "intuitive" if someone objects. Please feel free to correct.

As I mentioned earlier, there was an overhaul of the Makefile years ago and I honestly have no clue what it realy supposed to do. There always was a trmendous problem building the DOC prior to refdb 1.0.1 (never showed up in the FreeBSD public). Obviously, docs do build now.

As I reported, the port does build, install and deinstall on my poudriere system I have at hand (just now running a clean run to ensure the failures reported in here).

I even do not know the purpose of that obscure SGML file created in the Makefile, I have to take a much deeper look into it - I by myself consider such problems far more serious than shuffling tabs and spaces around.

The code base itself is also a bit outdated, my bigger concern is that it doesn't meet modern security aspects and probalbly endanger the integrity of FreeBSD. So, if there is any advice hinting me how to simply check, you're welcome.
Comment 37 O. Hartmann 2022-08-10 06:23:28 UTC
... sorry for the noise, obviously USES=java drops the builder now.
Comment 38 O. Hartmann 2022-08-10 07:21:36 UTC
Created attachment 235820 [details]
textproc/refdb 1.0.3 patch

DOCS_USE=               java:8+,build

is not the correct syntax of defining this dependency, but I did not find in the hurry the correct description in the handbook. 
Despite this, port 

textproc/fop

reels in the JAVA SDK, so the question is here: do we need this extra dependency? textproc/fop was considered an option in an very early attempt only used when building the doc, but it has revealed itself as a necessity when using transformations in a day-to-day work with RefDB.

The latest patchfile is considered for a build evaluation. I observe on my builder a strange behaviour that the build several times fails and restarting it it runs passed through then - not building any adjavent port. Maybe its a flaw in the "poudriere in a jail" setup.
Comment 39 Michael Gmelin freebsd_committer freebsd_triage 2022-08-10 09:14:39 UTC
Hi Oliver,

Independently of your open question about java/dependencies (which we will look into), I did a short shell recording of using portlint, portclippy, and portfmt, maybe it's interesting to you:

https://people.freebsd.org/~grembo/portsqa.html

Cheers
Comment 40 O. Hartmann 2022-08-10 09:21:28 UTC
Created attachment 235821 [details]
textproc/refdb 1.0.3 patch

Fixed a stupid "copy and blindly paste" mistake in Makefile regarding the content of the catalog file, located at

/usr/local/share/sgml/catalog.ports

Its content is now fixed also in the poudriere environment (post-install issue, it never popped up in a traditional make build)
Comment 41 Fernando Apesteguía freebsd_committer freebsd_triage 2022-08-11 13:48:54 UTC
===> Checking for items in pkg-plist which are not in STAGEDIR                                                      
Error: Missing: etc/rc.d/refdbd                                                                                     
Error: Missing: %%DOCSDIR%%/ch01.html                                                                               
Error: Missing: %%DOCSDIR%%/ch01s02.html                                                                            
Error: Missing: %%DOCSDIR%%/ch01s03.html                                                                            
Error: Missing: %%DOCSDIR%%/ch01s04.html                                                                            
Error: Missing: %%DOCSDIR%%/ch01s05.html                                                                            
Error: Missing: %%DOCSDIR%%/ch01s06.html                                                                            
Error: Missing: %%DOCSDIR%%/ch01s07.html                                                                            
Error: Missing: %%DOCSDIR%%/ch02.html                                                                               
Error: Missing: %%DOCSDIR%%/ch02s02.html                                                                            
Error: Missing: %%DOCSDIR%%/ch02s03.html                                                                            
Error: Missing: %%DOCSDIR%%/ch02s04.html                                                                            
Error: Missing: %%DOCSDIR%%/ch02s05.html                                                                            
Error: Missing: %%DOCSDIR%%/ch03.html                                                                               
Error: Missing: %%DOCSDIR%%/ch03s02.html                                                                            
Error: Missing: %%DOCSDIR%%/ch04.html                                                                               
Error: Missing: %%DOCSDIR%%/ch04s02.html                                                                            
Error: Missing: %%DOCSDIR%%/ch04s03.html                                                                            
Error: Missing: %%DOCSDIR%%/ch04s04.html           
...
Comment 42 Fernando Apesteguía freebsd_committer freebsd_triage 2022-08-11 13:52:31 UTC
Created attachment 235847 [details]
Working patch

This is the first working patch I could get (notice the reported errors from the comment above).

In this patch:

    * Add USES=localbase
    * Remove pkg-plist entries for automatically added files (USE_RC_SUBR and
      PORTDOCS)
    * Remove noop REINPLACE_CMD
    * Remove catalog generation.

It is unclear to me what the catalog generation does. What is it for?
Comment 43 O. Hartmann 2022-08-11 19:39:06 UTC
Created attachment 235851 [details]
textproc/refdb - for your considerations

Attached, a patch based upon the latest lecture. I regret the inconvenience. In some aspects I wasn't able to reproduce the problems and errors as the reporter did not not offer any hint in which stage the problem occured.

Hopefully, this patch will do
Comment 44 O. Hartmann 2022-08-12 10:08:52 UTC
Created attachment 235858 [details]
textproc/refdb-1.0.3 Patchset for considerations

In the /doc folder, Makefile provided a workaround concerning a bug in fop <= 1.0 regarding hyphenation (I assume so). The workaround triggered a miscompilation of the PDF manual. Eliminating this workaround makes doc compilation work again and ending as expected. The PDF has still style issue not met in this patch.

A still open issue is how to treat and deal with paperize, setting --stringparam paper.type A4 in doc/Makefile.am (xsltproc ...) seems a bit too static. I do not know whether xsltproc respects any papersize environemnatl settings.

Eliminating JAVA dependency (DOC_USE = ...). JAVA is a dependency of textproc/fop, so let textproc/fop deal with JAVA.
Comment 45 O. Hartmann 2022-08-31 12:55:16 UTC
Created attachment 236264 [details]
Patch refdb-1.0.3, revision

Diggin deeper into the sources, I found some problems when DOCS are build and another problem within the sources regarding the default database path. Latter refers to a linuxism, as far as I know, FreeBSD's default db base is /var/db and for the database folder it is considered to be /var/db/refdb. I also adjusted some default paths and I hope I've met the FreeBSD standard so far.

I also put all the %%PORTDOCS%% tagged stuff back into pkg-plist - following the porters handbook.

By fixing the doc build, I also figured that some dependencies are not quite so docs bound as I thought (i.e. dtdparse), since some ports are required converting from RIS to other bibliographic representations. Building the docs requires also convert from the ImageMagick suite. Also the port www/tidy-lib and textproc/sgrep is required. Fixed also some inconvenient configure issues by hard-configuring them (i.e. tidy isn't resolved on FreeBSD to be existent, although residing at /usr/local/bin/tidy). I think this might be a minor issue.

I also tried to make use of DB_DIR in src/refdbd.h, it is defined there as a C preprocessor macro, but src/refdbd.c doesn't refer to it (although including refdbd.h). I'll report this upstream, maybe there is a more elegant solution to handle this with configure (it seems that some options refering to db-dir are without effect so far).

I tried to incorporate prior considerations (but lost then somehow track), ran all supposed to be run test (portclippy, portlint -AC) and finally I tried to build on a poudriere VM. So far, poudriere does great, but the porters handbook also referes to a final test to run building the port as non-root - that fails always in the first attempt due to a weird fop error:

[...]
INFO: Rendered page #3.
Aug 31, 2022 9:57:44 AM org.apache.fop.events.LoggingEventListener processEvent
INFO: Rendered page #4.
Authorization required, but no authorization protocol specified
 
Exception in thread "main" java.awt.AWTError: Can't connect to X11 window server using ':0' as the value of the DISPLAY variable.
        at java.desktop/sun.awt.X11GraphicsEnvironment.initDisplay(Native Method)
        at java.desktop/sun.awt.X11GraphicsEnvironment$1.run(X11GraphicsEnvironment.java:102)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at java.desktop/sun.awt.X11GraphicsEnvironment.<clinit>(X11GraphicsEnvironment.java:61)
        at java.base/java.lang.Class.forName0(Native Method)
[...]

Start building the port again without "make clean" finish. Maybe I'm skipping that part.
Comment 46 O. Hartmann 2022-09-01 11:51:17 UTC
The port seems to have a serious issue when building documents.

I'll upload the latest patchset soon. Whatever I do, 8 out of 10 compile runs fail, 2 out of 10 work. This is some kind of weird.

I can not see exactly where the problem occurs, I assume it's 

(disabled parallel jobs ... I hope so)
[...]
MONTHSmkdir -p refdb-manual && cp ../doc/manual.css ../doc/refdbmanualfig1.png ../doc/refdbmanualfig2.png ../doc/refdbmanualfig3.png ../doc/refdbmanualfig4.png ../doc/refdbmanualfig5.png ../doc/refdbmanualfig6.png refdb-manual/ && xsltproc -o refdb-manual/ --nonet --xinclude include/manual-xhtml.xsl refdb-manual.xml
rm -rf refdb-manual/*
mkdir -p refdb-manual && cp ../doc/manual.css ../doc/refdbmanualfig1.png ../doc/refdbmanualfig2.png ../doc/refdbmanualfig3.png ../doc/refdbmanualfig4.png ../doc/refdbmanualfig5.png ../doc/refdbmanualfig6.png refdb-manual/ && xsltproc -o refdb-manual/ --nonet --xinclude include/manual-xhtml.xsl refdb-manual.xml
gmake[2]: *** [Makefile:1006: refdb-manual/ch01s02.html] Error 1
gmake[2]: *** Waiting for unfinished jobs....
Comment 47 O. Hartmann 2022-09-01 13:33:57 UTC
Created attachment 236287 [details]
Patchset refdb-1.0.3
Comment 48 O. Hartmann 2022-09-01 13:59:48 UTC
I've deleted some of the Makefile statements, which are remnants from a "friendly committer" who had refurbished the port a while ago. Since then, lot of things happened.

I realized that 8 out of 10 build-runs (building docs) failed, while 2 out of 10 finished -  a disturbing fact.

Checking the build process of the port by ignoring FreeBSD's port framework revealed some strange things, so enabling all those docbook-xsl stuff which has been disabled seem to mitigate several not understood errors.

By deleting the extra LDFLAGS and CPPFLAGS those strange erratic of building failures seem to have gone.

If someone may have the courtosy of testing on poudriere? In my case, 5 of 5 test builds finished successfuly.
Comment 49 O. Hartmann 2022-09-02 06:31:49 UTC
Created attachment 236302 [details]
Update port refdb to 1.0.3

Building DOCS results in most build runs in a failure, other runs succeed. The problem has not been identified, but is only present when building the docuemntation (extra docs apart from manpages). The cause seems to be a race condition, not sure which part of the tools used trigger this. I checked on smaller poudriere hosts (4C/8T) which seemingly perform allright and some older larger boxes ((2x8C)/32T and 32C/64T) which trigger the problem rapidly.

By declaring the whole ./doc build subfolder to be processed nonparallel (setting .NOTPARALLEL: in toplevel Makefile.am of ./doc) DOCS seems to build without any problems.

I think this is not of interest so far and hopefully by now, the new port frame can be considered ready ... or not ...
Comment 50 O. Hartmann 2022-09-03 11:57:53 UTC
Created attachment 236331 [details]
Update port textproc/refdb 1.0.3

Meet %%DOCS%% prependices in the pkg-plist file.
Comment 51 O. Hartmann 2022-09-05 06:27:30 UTC
Created attachment 236362 [details]
Upgrade refdb-1.0.3

Eliminate an illadvised logic regdading the dependency of databases/libdbi-drivers:

- make port databases/libdbi-drivers a RUN_DEPENDENCY instead of BUILD_DEPENDENCY (autoremove would otherwise kill the driver package, which is prerequisit to refdb's proper function, the former logic was stupid, shame on me)
- do not make the install of databases/libdbi-drivers depending on if exists(), it is a requirement if not.

At the moment I do not know whether binary packages and traditional make behaves the same when checking the dependency databases/libdbi-drivers: 

The Makefile checks for the existence of port/package databases/libdbi-drivers as a RUN_DEPENDENCY, depending on what database option has been selected in port textproc/refdb. We need database support anyway, so install databases/libdbi-drivers. Questioon is: does the ports framework then allow checks for the existence of the propper database access library in ${LOCALBASE}/lib/dbd according to the necessities defined by the selected database?
The former logic seemed illadvised to me, so I deleted the extra .elif statement.
Comment 52 O. Hartmann 2022-09-05 06:30:09 UTC
Created attachment 236363 [details]
Upgrade refdb-1.0.3

Correct patch, sorry.
Comment 53 O. Hartmann 2022-10-09 09:05:06 UTC
Created attachment 237162 [details]
correct run dependencies

In some marginal cases, dependend on port databases/libdbi-drivers does not get installed. Tried to correct this issue by adding appropriate "_RUN_DEPENDS=" tags per DB option. Since _LIB_DEPENDS=" does not allow any deviation from ${LOCALBASE}/lib I went with _RUN_DEPENDS= (I caught always a pattern violation, the apprpriate libs are stored in ${LOCALBASE}/lib/dbd/...).