Bug 213691 - net/openntpd: unblock package build via static libressl
Summary: net/openntpd: unblock package build via static libressl
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Only Me
Assignee: Christian Weisgerber
URL:
Keywords: patch, patch-ready
Depends on: 213889
Blocks:
  Show dependency treegraph
 
Reported: 2016-10-21 22:37 UTC by Jan Beich
Modified: 2016-10-30 16:30 UTC (History)
3 users (show)

See Also:
naddy: maintainer-feedback+
jbeich: merge-quarterly?


Attachments
v1 (1.65 KB, patch)
2016-10-21 22:37 UTC, Jan Beich
jbeich: maintainer-approval? (naddy)
Details | Diff
9.3R i386 |poudriere bulk -t| log (XZ compressed) (59.64 KB, application/x-xz)
2016-10-21 22:39 UTC, Jan Beich
no flags Details
svn diff for net/openntpd (960 bytes, patch)
2016-10-30 15:13 UTC, Bernard Spil
brnrd: maintainer-approval?
naddy: maintainer-approval+
Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Jan Beich freebsd_committer freebsd_triage 2016-10-21 22:37:38 UTC
Created attachment 176044 [details]
v1

LibreSSL is currently ABI-incompatible with OpenSSL. One way to bypass ABI is to not rely on it by using a static library. This is an example how to bundle such a library.
Comment 1 Jan Beich freebsd_committer freebsd_triage 2016-10-21 22:39:18 UTC
Created attachment 176045 [details]
9.3R i386 |poudriere bulk -t| log (XZ compressed)
Comment 2 Jan Beich freebsd_committer freebsd_triage 2016-10-21 22:45:07 UTC
Actually, PORTREVISION bump is unnecessary because:
- package(s) for this port are not currently built on the cluster
- statu quo is preserved for DEFAULT_VERSIONS = ssl=libressl
Comment 3 Jan Beich freebsd_committer freebsd_triage 2016-10-21 22:47:35 UTC
$ fgrep -H 'checking for tls_config_set_ca_mem' /poudriere/data/logs/bulk/*-default/latest/logs/openntpd-6.0p1_2,2.log
/poudriere/data/logs/bulk/101i386-default/latest/logs/openntpd-6.0p1_2,2.log:checking for tls_config_set_ca_mem... yes
/poudriere/data/logs/bulk/103amd64-default/latest/logs/openntpd-6.0p1_2,2.log:checking for tls_config_set_ca_mem... yes
/poudriere/data/logs/bulk/110i386-default/latest/logs/openntpd-6.0p1_2,2.log:checking for tls_config_set_ca_mem... yes
/poudriere/data/logs/bulk/93amd64-default/latest/logs/openntpd-6.0p1_2,2.log:checking for tls_config_set_ca_mem... yes
/poudriere/data/logs/bulk/93i386-default/latest/logs/openntpd-6.0p1_2,2.log:checking for tls_config_set_ca_mem... yes
Comment 4 Christian Weisgerber freebsd_committer freebsd_triage 2016-10-22 20:43:06 UTC
I already considered this approach, but linking statically means that when there is an update to libressl, the installed openntpd will not pick up the fixed library and continue to use the old code.

For this to work, the libressl maintainer would have to check the tree for such constructs and bump the PORTREVISION of all affected ports whenever there is an update to libressl.
Comment 5 Jan Beich freebsd_committer freebsd_triage 2016-10-24 10:16:17 UTC
security/acme-client already has a hack that requires just as much care on security/libressl updates. Also, the following isn't complicated:

  $ fgrep -lr --include='Makefile*' :security/libressl ${PORTSDIR=/usr/ports} |
    sed 's,/Makefile,,' | xargs ${PORTSDIR}/Tools/scripts/bump-revision.sh

If security/libressl provided a non-conflicting shared library the maintainer would still have to do the same every time ABI (or .so.X+1) changes. Looking at the history of updates this happens often (e.g. ports r414560, ports r420102) but not every time. devel/abi-compliance-checker can be used if trust in upstream drops.
Comment 6 commit-hook freebsd_committer freebsd_triage 2016-10-28 20:38:37 UTC
A commit references this bug:

Author: naddy
Date: Fri Oct 28 20:38:17 UTC 2016
New revision: 424884
URL: https://svnweb.freebsd.org/changeset/ports/424884

Log:
  Unblock the package build via static libressl.  Adapted from [1].

  OpenNTPD requires libtls, which is only available from LibreSSL.
  If LibreSSL is not the default SSL library, link with a static copy
  of the LibreSSL libraries.  This approach has the drawback that
  whenever there is an update to libressl, the openntpd port will
  require a PORTREVISION bump, otherwise it will not pick up the fixed
  library and continue to use the old code.

  Drop the RESSL option as LibreSSL is now always available.

  PR:		213691 [1]
  Submitted by:	jbeich

Changes:
  head/net/openntpd/Makefile
Comment 7 Christian Weisgerber freebsd_committer freebsd_triage 2016-10-28 20:40:43 UTC
Thank you, I committed a tweaked version of this.
We'll see how it goes.
Comment 8 Simeon Simeonov 2016-10-29 14:45:33 UTC
How does one build openntpd without SSL support now?
I have an openssl port and will still like to continue using openntpd as before with (RSSL disabled).
Comment 9 Christian Weisgerber freebsd_committer freebsd_triage 2016-10-29 15:10:44 UTC
(In reply to blackmore from comment #8)

One doesn't.
What problem are you trying to solve? There will be no conflict with the openssl port you have installed.
Comment 10 Simeon Simeonov 2016-10-29 15:19:00 UTC
The problem is that openntpd-6.0p1_1,2 is forcing the installation of libressl and that is in conflict with the installed security/openssl port.

Previously I had selected to disable RSSL (SSL support). That option is now removed.

These changes make things wotk for me:

--- a/net/openntpd/Makefile
+++ b/net/openntpd/Makefile
@@ -2,7 +2,7 @@
 
 PORTNAME=	openntpd
 PORTVERSION=	6.0p1
-PORTREVISION=	2
+PORTREVISION=	3
 PORTEPOCH=	2
 CATEGORIES=	net
 MASTER_SITES=	OPENBSD/OpenNTPD
@@ -21,6 +21,12 @@ GROUPS=		_ntp
 USES=		ssl
 
 GNU_CONFIGURE=	yes
+
+OPTIONS_DEFINE=                RESSL
+RESSL_DESC=            SSL/TLS support via LibreSSL
+
+OPTIONS_DEFAULT=       RESSL
+
 CONFIGURE_ARGS=	--with-cacert=${LOCALBASE}/etc/ssl/cert.pem
 
 pre-build:
@@ -33,6 +39,7 @@ post-stage:
 .include <bsd.port.pre.mk>
 
 # Requires libtls from LibreSSL
+.if ${PORT_OPTIONS:MRESSL}
 .if ${SSL_DEFAULT:Mlibressl*}
 CPPFLAGS+=		-I${OPENSSLINC}
 LDFLAGS+=		-L${OPENSSLLIB}
@@ -40,6 +47,8 @@ LDFLAGS+=		-L${OPENSSLLIB}
 BUILD_DEPENDS+=		${NONEXISTENT}:security/libressl:stage
 CPPFLAGS+=		-I${WRKDIR}/libressl/include
 LDFLAGS+=		-L${WRKDIR}/libressl/lib
+.endif # RESSL
+
 # security/libressl ignores implicit SSP_UNSAFE, see Mk/bsd.ssp.mk
 . if ${OPSYS} == FreeBSD && ${OSVERSION} < 1000036 && ${ARCH} == i386
 SSP_CFLAGS?=	-fstack-protector
Comment 11 Jan Beich freebsd_committer freebsd_triage 2016-10-29 15:45:46 UTC
Can you show complete build log? security/libressl is built as part of net/openntpd but is not actually installed. In a few days binary packages should become available where you can confirm lack of dependency[1] on libressl package.

  $ pkg info
  pkg-1.9.2                      Package manager

  $ pkg install openntpd
  Updating FreeBSD repository catalogue...
  FreeBSD repository is up-to-date.
  All repositories are up-to-date.
  Checking integrity... done (0 conflicting)
  The following 2 package(s) will be affected (of 0 checked):

  New packages to be INSTALLED:
          openntpd: 6.0p1_2,2
          ca_root_nss: 3.27.1

  Number of packages to be installed: 2

  The process will require 2 MiB more space.

  Proceed with this action? [y/N]:

[1] Caveat: openntpd package currently always depends on system SSL library (whatever that may be) even if unused. USES=ssl is only needed for DEFAULT_VERSIONS=ssl=libressl , but USES facility doesn't work after bsd.port.pre.mk. This may be fixed by moving the workaround here into USES=ssl.
Comment 12 Simeon Simeonov 2016-10-29 15:52:13 UTC
Sure. Make /etc/make.conf contains among other things:

## world ##
MALLOC_PRODUCTION=yes
SSL_OP_NO_SSLv2=yes
SSL_OP_NO_SSLv3=yes
security_openssl_UNSET=SSL2 SSL3

## ports ##
WITH_OPENSSL_PORT=yes
OPTIONS_UNSET=DEBUG DOCS X11
OPTIONS_SET=OPTIMIZED_CFLAGS
DEFAULT_VERSIONS=python=2.7 python2=2.7 python3=3.5 php=5.6 ssl=openssl


The build goes as follows:

root@SagaBOX ~]# portmaster -adi

===>>> The following actions will be taken if you choose to proceed:
	Upgrade openntpd-6.0p1_1,2 to openntpd-6.0p1_2,2

===>>> Proceed? y/n [y]

===>>> Starting build for ports that need updating <<<===

===>>> Launching child to install net/openntpd

===>>> All >> net/openntpd (1/1)

===>>> Currently installed version: openntpd-6.0p1_1,2
===>>> Port directory: /usr/ports/net/openntpd

===>>> Starting check for build dependencies
===>>> Gathering dependency list for net/openntpd from ports

===>>> The dependency for security/libressl
       seems to be handled by openssl-1.0.2j_1,1

===>>> Dependency check complete for net/openntpd

===>>> All >> openntpd-6.0p1_1,2 (1/1)

===>  Cleaning for openntpd-6.0p1_2,2
===>  License ISCL accepted by the user
===>  Found saved configuration for openntpd-5.7p4_2,2
===>   openntpd-6.0p1_2,2 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by openntpd-6.0p1_2,2 for building

===>  libressl-2.4.3 conflicts with installed package(s): 
      openssl-1.0.2j_1,1

      They install files into the same place.
      You may want to stop build with Ctrl + C.
===>  License BSD4CLAUSE accepted by the user
===>  Found saved configuration for libressl-2.4.3
===>   libressl-2.4.3 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by libressl-2.4.3 for building
===>  Extracting for openntpd-6.0p1_2,2
=> SHA256 Checksum OK for openntpd-6.0p1.tar.gz.
===>  Patching for openntpd-6.0p1_2,2
===>  Applying FreeBSD patches for openntpd-6.0p1_2,2
===>   openntpd-6.0p1_2,2 depends on file: /usr/local/lib/libcrypto.so.9 - found

===>  libressl-2.4.3 conflicts with installed package(s): 
      openssl-1.0.2j_1,1

      They install files into the same place.
      You may want to stop build with Ctrl + C.
===>  License BSD4CLAUSE accepted by the user
===>  Found saved configuration for libressl-2.4.3
===>   libressl-2.4.3 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by libressl-2.4.3 for building

===>  libressl-2.4.3 conflicts with installed package(s): 
      openssl-1.0.2j_1,1

      They will not build together.
      Please remove them first with pkg delete.
*** Error code 1

Stop.
make[2]: stopped in /usr/ports/security/libressl
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/security/libressl
*** Error code 1

Stop.
make: stopped in /usr/ports/net/openntpd

===>>> make build failed for net/openntpd
===>>> Aborting update

===>>> Update for net/openntpd failed
===>>> Aborting update


===>>> You can restart from the point of failure with this command line:
       portmaster <flags> net/openntpd
Comment 13 Bernard Spil freebsd_committer freebsd_triage 2016-10-29 18:42:15 UTC
(In reply to Simeon Simeonov from comment #12)

> WITH_OPENSSL_PORT= yes
is deprecated, please use 
> DEFAULT_VERSIONS+= ssl=openssl
in stead.

The port doesn't bring in LibreSSL as far as I can see. It only builds libressl up to staging target and uses it to statically link libtls.
Comment 14 Simeon Simeonov 2016-10-29 18:50:49 UTC
The same result when WITH_OPENSSL_PORT= yes is removed (obviously).

The port pulls inn LibreSSL as a dependency. As a result the port-build (of openntpd) fails.
This should not happen, no matter the difference in definitions.
Comment 15 Bernard Spil freebsd_committer freebsd_triage 2016-10-30 13:20:28 UTC
(In reply to Simeon Simeonov from comment #14)
Hi Simeon,

Your build-system looks out-of-sync. If anything your output should show many warnings regarding the use of WITH_OPENSSL_PORT but apparently the warnings are hidden by portmaster.

If anything this looks like an issue with portmaster, not with this port.

> ===>>> The following actions will be taken if you choose to proceed:
>         Re-install openntpd-6.0p1_2,2
>         Install security/libressl

With ports framework

> make -C /usr/ports/net/openntpd
> /!\ WARNING /!\
> Using WITH_OPENSSL_PORT in make.conf is deprecated, replace it with DEFAULT_VERSIONS+=ssl=openssl in your make.conf

Just verified that it DOES NOT install LibreSSL when using the ports-framework.
portmaster uses run-, build, all-depends-list to check for dependencies

Please open a PR for ports-mgmt/portmaster.

Thanks, Bernard.
Comment 16 Simeon Simeonov 2016-10-30 13:29:10 UTC
Well... not exactly...
OK. After removing openntpd I do the following steps (NO portmaster involved, no outdated make.conf either)


[root@SagaBOX ~]# portsnap fetch update && pkg version -v -L = ; pkg updating -d 20161030
Looking up portsnap.FreeBSD.org mirrors... 6 mirrors found.
Fetching snapshot tag from ec2-eu-west-1.portsnap.freebsd.org... done.
Ports tree hasn't changed since last snapshot.
No updates needed.
Ports tree is already up to date.

[root@SagaBOX ~]# grep -i ssl /etc/make.conf 
SSL_OP_NO_SSLv2=yes
SSL_OP_NO_SSLv3=yes
security_openssl_UNSET=SSL2 SSL3
##WITH_OPENSSL_PORT=yes
DEFAULT_VERSIONS=python=2.7 python2=2.7 python3=3.5 php=5.6 ssl=openssl

root@SagaBOX /usr/ports/net/openntpd]# make install clean
===>  License ISCL accepted by the user
===>  Found saved configuration for openntpd-5.7p4_2,2
===>   openntpd-6.0p1_2,2 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by openntpd-6.0p1_2,2 for building

===>  libressl-2.4.3 conflicts with installed package(s): 
      openssl-1.0.2j_1,1

      They install files into the same place.
      You may want to stop build with Ctrl + C.
===>  License BSD4CLAUSE accepted by the user
===>  Found saved configuration for libressl-2.4.3
===>   libressl-2.4.3 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by libressl-2.4.3 for building
===>  Extracting for openntpd-6.0p1_2,2
=> SHA256 Checksum OK for openntpd-6.0p1.tar.gz.
===>  Patching for openntpd-6.0p1_2,2
===>  Applying FreeBSD patches for openntpd-6.0p1_2,2
===>   openntpd-6.0p1_2,2 depends on file: /usr/local/lib/libcrypto.so.9 - found

===>  libressl-2.4.3 conflicts with installed package(s): 
      openssl-1.0.2j_1,1

      They install files into the same place.
      You may want to stop build with Ctrl + C.
===>  License BSD4CLAUSE accepted by the user
===>  Found saved configuration for libressl-2.4.3
===>   libressl-2.4.3 depends on file: /usr/local/sbin/pkg - found
===> Fetching all distfiles required by libressl-2.4.3 for building

===>  libressl-2.4.3 conflicts with installed package(s): 
      openssl-1.0.2j_1,1

      They will not build together.
      Please remove them first with pkg delete.
*** Error code 1

Stop.
make[2]: stopped in /usr/ports/security/libressl
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/security/libressl
*** Error code 1

Stop.
make: stopped in /usr/ports/net/openntpd
Comment 17 commit-hook freebsd_committer freebsd_triage 2016-10-30 14:57:01 UTC
A commit references this bug:

Author: brnrd
Date: Sun Oct 30 14:56:58 UTC 2016
New revision: 424947
URL: https://svnweb.freebsd.org/changeset/ports/424947

Log:
  security/libressl: Switch to CONFLICTS_INSTALL

    - There's no conflict at build time

  PR:		213691

Changes:
  head/security/libressl/Makefile
  head/security/libressl-devel/Makefile
Comment 18 Bernard Spil freebsd_committer freebsd_triage 2016-10-30 15:13:47 UTC
Created attachment 176310 [details]
svn diff for net/openntpd

(In reply to Bernard Spil from comment #15)
Hi Simeon,

Aha!

Sorry, hadn't checked what happens when OpenSSL is actually installed when you build openntpd, guess what...

I think I've solved this with this patch.

The issue is with setting USES= ssl which forces the ports framework to build/install libressl. This is now only set when LibreSSL is indeed installed.
In other cases it will build/stage LibreSSL but not install or depend on it.
Comment 19 commit-hook freebsd_committer freebsd_triage 2016-10-30 15:17:04 UTC
A commit references this bug:

Author: brnrd
Date: Sun Oct 30 15:16:02 UTC 2016
New revision: 424949
URL: https://svnweb.freebsd.org/changeset/ports/424949

Log:
  security/acme-client: Fix CONFLICT errors when OpenSSL is installed

    - As reported for OpenNTPd

  PR:		213691
  Reported by:	Simeon Simeonov <blackmore@pichove.org>

Changes:
  head/security/acme-client/Makefile
Comment 20 Simeon Simeonov 2016-10-30 15:44:00 UTC
That will work, but do we want to eliminate the option to build openntpd without (any) SSL support?
I feel that this option may be useful on some systems like tiny Raspberry Pi installations.

In any case this should fix it for most people, including me. Thank you for your time and patience!
Comment 21 Jan Beich freebsd_committer freebsd_triage 2016-10-30 16:08:45 UTC
Unless a back out is in question, please, discuss any regressions in separate bugs. This bug became a mess of different issues.

Also, ports 424949 landed despite caveat described in comment 11.
Comment 22 Bernard Spil freebsd_committer freebsd_triage 2016-10-30 16:15:50 UTC
(In reply to Jan Beich (mail not working) from comment #21)
Hi Jan,

A fix for the issue is also attached to this PR. Small change but at least allows everyone to build.

Cheers, Bernard.
Comment 23 Jan Beich freebsd_committer freebsd_triage 2016-10-30 16:24:31 UTC
Do you mean ports 424947 *after* you've ignored my patch in bug 213889? ;)
Comment 24 Christian Weisgerber freebsd_committer freebsd_triage 2016-10-30 16:26:33 UTC
(In reply to Bernard Spil from comment #22)

Yes, please commit.
Comment 25 Jan Beich freebsd_committer freebsd_triage 2016-10-30 16:30:34 UTC
Comment on attachment 176310 [details]
svn diff for net/openntpd

After applying DEFAULT_VERSIONS=ssl=libressl build fails: http://sprunge.us/ACdA