Bug 236845 - sysutils/syslog-ng320: Don't drag in python when PYTHON isn't selected, other fixes
Summary: sysutils/syslog-ng320: Don't drag in python when PYTHON isn't selected, other...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Cy Schubert
URL:
Keywords: patch
Depends on:
Blocks:
 
Reported: 2019-03-28 02:24 UTC by Craig Leres
Modified: 2019-03-31 20:41 UTC (History)
0 users

See Also:
bugzilla: maintainer-feedback? (cy)


Attachments
patch (9.88 KB, patch)
2019-03-28 02:24 UTC, Craig Leres
leres: maintainer-approval?
Details | Diff
pet portlint + deprecete some ports (59.68 KB, patch)
2019-03-29 06:12 UTC, Cy Schubert
no flags Details | Diff
Functional diff. (5.18 KB, patch)
2019-03-30 17:33 UTC, Cy Schubert
no flags Details | Diff
patch against r497322 (6.32 KB, patch)
2019-03-30 21:20 UTC, Craig Leres
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Craig Leres freebsd_committer 2019-03-28 02:24:04 UTC
I have DEFAULT_VERSIONS=python3=3.7 and noticed poudriere was also
building python36. When I investigated I found it was being built due to
syslog-ng (syslog-ng320) which had several issues:

 - unconditional USES=python:3.6,build
 - PYTHON_CONFIGURE_OFF=--disable-python --with-python=3.6
 - other general hardcoding to python36

In my testing python I found isn't used at all with --disable-python.

When PYTHON is enabled there are many byacc errors:

    /bin/sh ./ylwrap modules/afsmtp/afsmtp-grammar.y y.tab.c modules/afsmtp/afsmtp-grammar.c y.tab.h modules/afsmtp/afsmtp-grammar.h y.output modules/afsmtp/afsmtp-gramm
    ar.output -- byacc -Wno-yacc -Wno-other -d 2>&1 | (/usr/bin/egrep -v "(warning: ([0-9]+ )?(nonterminal|rule)s? useless in grammar)|(^updating|unchanged)" || true)
    /bin/sh ./ylwrap modules/afsmtp/afsmtp-grammar.y y.tab.c modules/afsmtp/afsmtp-grammar.h y.tab.h modules/afsmtp/afsmtp-grammar.h y.output modules/afsmtp/afsmtp-gramm
    ar.output -- byacc -Wno-yacc -Wno-other -d 2>&1 | (/usr/bin/egrep -v "(warning: ([0-9]+ )?(nonterminal|rule)s? useless in grammar)|(^updating|unchanged)" || true)
    Usage: byacc [options] filename

    Options:
      -b file_prefix        set filename prefix (default "y.")
      -B                    create a backtracking parser
    [...]

A configure message seems to indicate that it would prefer bison:

    configure: WARNING: No proper bison found, you'll not be able to change lib/cfg-grammar.y

I added PYTHON_USES=bison:build and this solved the errors.

I removed USES=python:3.6,build and added PYTHON_USES=python:3.5+,build
to all use of any of the python3 versions and changed other python version references to use the PYTHON_VER and PYTHON_SUFFIX variables.

I also removed SHEBANG_LANG=python3 because there are no dependencies
that would cause the python3 meta port to be installed and thus no python3 on the path.

While here fix some portlint nits:

 - use tab (not space) to make indentation
 - whitespace before end of line
 - order (e.g. DISTFILES and USES)
 - unnecessary ${INSTALL} -o/-g
Comment 1 Craig Leres freebsd_committer 2019-03-28 02:24:42 UTC
Created attachment 203203 [details]
patch
Comment 2 Cy Schubert freebsd_committer 2019-03-28 03:00:23 UTC
Thanks. I'll pet portlint separately from the functional commits. May as well do all the syslog-ng* ports while I'm at it.
Comment 3 Cy Schubert freebsd_committer 2019-03-28 03:17:14 UTC
Backporting python fix as well.

I've asked Peter @ Balabit as well. He provides as is.
Comment 4 Cy Schubert freebsd_committer 2019-03-29 06:12:25 UTC
Created attachment 203233 [details]
pet portlint + deprecete some ports

This patch, two commits already committed to my git repo, pets portlint (first git commit) and deprecates the older syslog-ng ports and the eventlog port, which was superceeded by syslog-ng311 (IIRC).

Testing in a clean jail $PYTHON_VER does nothing. However python 3.6 appears to be the prereq. It's getting late here. I'll analyze the results of the ongoing build tomorrow.
Comment 5 Cy Schubert freebsd_committer 2019-03-30 17:33:16 UTC
Created attachment 203261 [details]
Functional diff.

In addition to the portlint and deprecate patches already attached and already committed to my git tree, this functional diff implements a slightly adjusted form of what this PR is trying to accomplish. I'm hoping to commit this before the end of the month (maybe today) to get it into 2019Q2. Currently testing in a clean jail.

This only affects syslog-ng318/319/320, as prior versions need python27. The question is, when will python27 be removed? I want to deprecated 317 and earlier at about the same time.

Give the portlint patch + this patch a spin.
Comment 6 Cy Schubert freebsd_committer 2019-03-30 18:36:18 UTC
Fixed up the patch. Committed soon.

My commit message will explain why python is always installed, even with this or my adjusted patch based on this. Blame glib20 for that.
Comment 7 commit-hook freebsd_committer 2019-03-30 18:37:41 UTC
A commit references this bug:

Author: cy
Date: Sat Mar 30 18:36:52 UTC 2019
New revision: 497322
URL: https://svnweb.freebsd.org/changeset/ports/497322

Log:
  Don't depend on python 3.6 when the python option is deselected.
  However syslog-ng* does depend on glib20. glib20 does depend on
  python and if python is deslected the default python27 is installed
  by glib20. If OTOH the python option is selected, python36 is installed
  and glib20 will use it instead. Installing syslog-ng without python
  support and subsequently changing one's mind to later depend on
  python will result in both python27 and python36 to be installed.

  PR:		236845
  Inspired by:	leres@

Changes:
  head/sysutils/syslog-ng318/Makefile
  head/sysutils/syslog-ng319/Makefile
  head/sysutils/syslog-ng320/Makefile
Comment 8 Cy Schubert freebsd_committer 2019-03-30 18:38:11 UTC
Fixed.
Comment 9 Cy Schubert freebsd_committer 2019-03-30 19:56:25 UTC
Unfortunately this has caused additional breakage, when python is not selected. It's still required to build.

===>  Building for syslog-ng-3.20.1
gmake[2]: Entering directory '/export/wrkdir/amd64/usr/ports/sysutils/syslog-ng/work/syslog-ng-3.20.1'
Makefile:22291: *** "Python interpreter is required to generate grammar files, but it was not detected during configure".  Stop.
gmake[2]: Leaving directory '/export/wrkdir/amd64/usr/ports/sysutils/syslog-ng/work/syslog-ng-3.20.1'
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/sysutils/syslog-ng
*** Error code 1

Stop.
make: stopped in /usr/ports/sysutils/syslog-ng

There still needs to be a build requirement but no install requirement.
Comment 10 Craig Leres freebsd_committer 2019-03-30 21:20:18 UTC
Created attachment 203268 [details]
patch against r497322

(In reply to Cy Schubert from comment #9)
Interesting. I'm sure this was building for me.

One problem I see is that:

    PYTHON_CONFIGURE_WITH=python=${PYTHON_VER}

does not seem to be the same as:

   CONFIGURE_ARGS=--with-python=${PYTHON_VER}

I think the problem is PYTHON_CONFIGURE_WITH only happens when PYTHON is selected but we need it either way. (It might also be an ordering problem, I think we have to have --disable-python before --with-python=${PYTHON_VER})

Another issue is that PYTHON_PLIST_FILES is still hardcoded for 36.

Finally I still don't think SHEBANG_LANG=python3 works, it will patch merge-grammar.py to use /usr/local/bin/python3 which won't exist. I suspect USES=shebang is smarter than it was when this was first written.

Here's a new patch against r497322.
Comment 11 Cy Schubert freebsd_committer 2019-03-30 21:40:06 UTC
(In reply to Craig Leres from comment #10)
I've already committed what you have in the attachment.

Peter Czanik <peter.czanik@balabit.com>, our upline's representative to FreeBSD (Balabit employee), specifically emailed me quite a while ago that they intend to only support python 3.6 and greater, which is surprising why you managed to get it to work under 3.5.

I asked Peter, when the PR was originally filed, to take a look at this. He basically emails the ports to me and I commit them. He's the real maintainer.

The port is working now. Let's wait for Peter to weigh in.

As to not installing python for people who want to keep $LOCALBASE lean, glib20 drags in python27. It makes more sense for those who what less installed to select PYTHON as some version of python will be installed anyway and it may as well be a python that is supported by our python upline. In the worst case scenario if a person deselects PYTHON and selects it later the result will be python27 and python36 installed. If PYTHON is selected from the get-go glib20 will use python36 that was previously installed by syslog-ng. I'm seriously considering removing the PYTHON option altogether and installing it regardless. 

Also note that even if PYTHON is deslected and PYTHON is used during the build only, glib20 requires it to run so even pkg users must have some version of python installed. Working to remove the python dependency in syslog-ng is a waste of time.
Comment 12 Cy Schubert freebsd_committer 2019-03-31 17:11:42 UTC
Sorry, I'll test the patch by specifically installing another version of python.

Either way python will be installed. We should be able to reduce the number of pythons installed to only one, satisfying glib20 and syslog-ng.
Comment 13 commit-hook freebsd_committer 2019-03-31 18:51:28 UTC
A commit references this bug:

Author: cy
Date: Sun Mar 31 18:51:09 UTC 2019
New revision: 497443
URL: https://svnweb.freebsd.org/changeset/ports/497443

Log:
  Correctly support python35+.

  PR:		236845
  Submitted by:	leres@

Changes:
  head/sysutils/syslog-ng318/Makefile
  head/sysutils/syslog-ng319/Makefile
  head/sysutils/syslog-ng320/Makefile
Comment 14 Cy Schubert freebsd_committer 2019-03-31 18:54:34 UTC
The parts of the patch that weren't previously committed work. I've commited them as well.
Comment 15 Craig Leres freebsd_committer 2019-03-31 20:31:34 UTC
(In reply to Cy Schubert from comment #14)
With r497443 the committed version functionally identical to what I last tested (my version sorted USES).

Regarding USES=3.5+, I used that because it seems to be the standard way of selecting the DEFAULT_VERSIONS=python3 version of python. If it turns out syslog-ng doesn't work with 3.5, then it could change to be USES=3.6+. Upstream is likely the best one to comment on this.

Thanks for slogging through on this.
Comment 16 Cy Schubert freebsd_committer 2019-03-31 20:41:26 UTC
Thanks for your perseverance too.