Bug 230538 - devel/apr1 fails with mysql80-client
Summary: devel/apr1 fails with mysql80-client
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Some People
Assignee: Jochen Neumeister
URL: https://bz.apache.org/bugzilla/show_b...
Keywords: needs-patch
Depends on:
Blocks:
 
Reported: 2018-08-11 16:05 UTC by Please
Modified: 2023-04-01 15:49 UTC (History)
11 users (show)

See Also:
bugzilla: maintainer-feedback? (apache)


Attachments
poudniere log (186.76 KB, text/plain)
2018-08-11 16:05 UTC, Please
no flags Details
proposed patch for mysql80 (2.59 KB, patch)
2018-08-12 12:55 UTC, Please
no flags Details | Diff
working patch not coflicing with other patches (1.34 KB, patch)
2018-08-12 14:37 UTC, Please
no flags Details | Diff
Build stopped with this messages (5.69 KB, text/plain)
2022-04-08 09:12 UTC, Andrey Bushev
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Please 2018-08-11 16:05:09 UTC
Created attachment 196091 [details]
poudniere log

---Begin OPTIONS List---
===> The following configuration options are available for apr-1.6.3.1.6.1_1:
     IPV6=on: IPv6 protocol support
====> Database support
     BDB=on: Berkeley DB support
     GDBM=on: GNU dbm library support
     LDAP=on: LDAP protocol support
     MYSQL=on: MySQL database support
     NDBM=off: NDBM support
     ODBC=off: ODBC database backend
     PGSQL=off: PostgreSQL database support
     SQLITE=off: SQLite database support
====> Cryptography provider: you can only select none or one of them
     SSL=on: OpenSSL crypto driver
     NSS=off: NSS crypto driver
===> Use 'make config' to modify these settings
---End OPTIONS List---

...

=======================<phase: package        >============================
===>  Building package for apr-1.6.3.1.6.1_1
pkg-static: Unable to access file /wrkdirs/usr/ports/devel/apr1/work/stage/usr/local/lib/apr-util-1/apr_dbd_mysql-1.so:No such file or directory
pkg-static: Unable to access file /wrkdirs/usr/ports/devel/apr1/work/stage/usr/local/lib/apr-util-1/apr_dbd_mysql.a:No such file or directory
pkg-static: Unable to access file /wrkdirs/usr/ports/devel/apr1/work/stage/usr/local/lib/apr-util-1/apr_dbd_mysql.so:No such file or directory
*** Error code 1

Problem might be mentioned in : bug #220699
Comment 1 Please 2018-08-11 17:02:15 UTC
It looks like it is a APR1 conflict with Mysql80-client 

(build and install apr manually)
./configure --with-apr=../apr-1.6.3/ --with-mysql 

apr-util configure gives this with mysql80-client 

....
checking for mysql_config... /usr/local/bin/mysql_config
  adding "-I/usr/local/include/mysql" to CPPFLAGS
  setting LIBS to "-L/usr/local/lib/mysql -lmysqlclient -pthread -lz -lm -lrt -lexecinfo -lssl -lcrypto"
checking for mysql.h... no
checking for mysql/mysql.h... no
...

when mysql57-client gives:
....
checking for mysql_config... /usr/local/bin/mysql_config
  adding "-I/usr/local/include/mysql" to CPPFLAGS
  setting LIBS to "-L/usr/local/lib/mysql -lmysqlclient -pthread -lz -lm -lrt -lexecinfo -lssl -lcrypto"
checking for mysql.h... yes
checking for mysql_init in -lmysqlclient... yes
checking for my_global.h... yes
....
Comment 2 Please 2018-08-11 19:43:30 UTC
After a while, it looks like my_global.h is generating a problem 

it is mentioned:

my_global.h should not have #defines that modify behavior of system
headers; this is brittle, and better suited to CMake options,
so move it there.

It should also not try to #undef symbols from other files, which is
again brittle (especially as these symbols tend to change without
people remembering to update my_global.h -- it seems to already have
happened during the introduction of yaSSL). Instead, make sure these
symbols are simply not added during compilation in the first place.
Every plugin now gets compiled with ${SSL_DEFINES} added automatically,
so they don't have to add that manually -- except if they are recompiled
for embedded, in which case ${SSL_DEFINES} is never added in the first
place (instead of having to #undef them).
https://github.com/mysql/mysql-server/commit/abbd9d90ed19ebe8e9294309cc189f256f812f70
Comment 3 Please 2018-08-11 20:36:50 UTC
To make apr1 compile with mysql80-client I found that 2 files need to updated from apr-util:

apr-util-1.6.1/dbd/apr_dbd_mysql.c

add to after line 50:
/* MySQL 8.0 replaces my_bool with C99 bool. Earlier versions of MySQL had
* a typedef to char. Gem users reported failures on big endian systems when
* using C99 bool types with older MySQLs due to mismatched behavior. */
#ifndef HAVE_TYPE_MY_BOOL
#include <stdbool.h>
typedef bool my_bool;
#endif

second file : apr-util-1.6.1/build/dbd.m4
replace line 178:
AC_CHECK_HEADERS([mysql.h my_global.h my_sys.h],
with 
AC_CHECK_HEADERS([mysql.h],

replace line 183:
AC_CHECK_HEADERS([mysql/mysql.h mysql/my_global.h mysql/my_sys.h],
with
AC_CHECK_HEADERS([mysql/mysql.h],

the same on lines 209 and 215

then replace all occurrence of 
[#include <mysql/my_global.h>]) 
with 
[#include <mysql/mysql.h>])

and

[#include <my_global.h>]) 
with 
[#include <mysql.h>])

then commands to test it : 
autoconf
./configure
make

see if dbd/apr_dbd_mysql.o file was generated.

I am no expert at doing patches, hope someone will pick it app and apply this with some testing, ah this is only when mysql80-client is installed.
Comment 4 Jochen Neumeister freebsd_committer freebsd_triage 2018-08-12 07:02:28 UTC
(In reply to Please from comment #3)

Is there a link to this 2 files? :)
Then i will have a look
Comment 6 Please 2018-08-12 12:55:47 UTC
Created attachment 196124 [details]
proposed patch for mysql80

also makefile require USES=autoreconf for rebuilding configure
Comment 7 Please 2018-08-12 14:37:24 UTC
Created attachment 196126 [details]
working patch not coflicing with other patches

Patch works, produces apr_dbd_mysql.so, other packages build fine. if library works and it is stable, cannot say.

To make it fully work, did modification to Makefile too :
--- Makefile.orig       2018-03-21 22:24:43.000000000 +0100
+++ Makefile    2018-08-12 16:25:15.442304000 +0200
@@ -16,7 +16,7 @@

 LIB_DEPENDS=   libexpat.so:textproc/expat2

-USES=          iconv pathfix libtool cpe
+USES=          autoreconf iconv pathfix libtool cpe
 USE_LDCONFIG=  yes
 GNU_CONFIGURE= yes

@@ -136,6 +136,8 @@
                ${SETENV} ${APR_CONF_ENV} ./configure ${APR_CONF_ARGS} ${CONFIGURE_ARGS})
        @${ECHO_MSG} "# ===> ${.TARGET} apr-util-${APU_VERSION}"
        (cd ${APU_WRKDIR} && \
+               ${SETENV} ${APU_CONF_ENV} autoconf)
+       (cd ${APU_WRKDIR} && \
                ${SETENV} ${APU_CONF_ENV} ./configure ${APU_CONF_ARGS} ${CONFIGURE_ARGS})

 do-build:
Comment 8 Jochen Neumeister freebsd_committer freebsd_triage 2018-08-12 15:15:23 UTC
(In reply to Please from comment #6)

The Patch 196124 is obsolete?
Comment 9 Please 2018-08-12 18:06:06 UTC
Yeap. The only one that works for me is 196126, keep in mind that I have no idea what will happen to people with mysql5x. So I assume Makefile would need to have condition patch. This way package will keep compatibility with m57 and will work with m80. I do not know how to do it.
Comment 10 Jochen Neumeister freebsd_committer freebsd_triage 2018-08-13 20:06:20 UTC
There was an update for mysql80 today.
It will take a few days until all tests are done.
I'll take care of it :-)
Comment 12 Please 2018-08-14 17:09:53 UTC
you are using mysql56-client-5.6.41, update make.conf to have: 
DEFAULT_VERSIONS+=mysql=8.0
Comment 14 Please 2018-08-18 11:08:47 UTC
Library compile with patch and produce valid .so that does not help to the build process of other packages. 

However, apr1-util for apache24 with mod_dbd do not like it, and gives error: 
  Can't load driver file apr_dbd_mysql-1.so

Did some code tracing and this is kind of general error without specific information. The same error is when driver name is changed in configuration file for non-existing dso. So apr gives always the same error. Is apr faulty?
Reference to 'apr_dbd_get_driver' with error 'APR_STATUS_IS_EDSOOPEN' and then to 'apr_dso_load' to dso.c

I would mark port as conflict with mysql80 if mysql option is selected unless someone have idea how to fix it.
Comment 15 Bernard Spil freebsd_committer freebsd_triage 2018-10-01 09:16:40 UTC
Adding Mahdi to the cc list.

Mahdi: Any changes in MySQL 8 that may cause this problem? For MariaDB I've been having issues with the changed libmariadb and mysql_config...
Comment 16 Mahdi Mokhtari freebsd_committer freebsd_triage 2018-10-06 17:26:10 UTC
Hi,
First sorry for my a bit delayed answer on this.
today I looked into it a bit:
it seems the apr1's configure script is unable to find the mysql.h as you already pointed, it's right there in /usr/local/include/mysql/mysql.h as it is in all mysqls.
I wonder about the line `--with-mysql=${LOCALBASE}` in apr's Makefile I'm gonna play with it a bit more... (not sure if it's right thing to do ``:D but seems like a path ;D)

@brnrd, what kind of change do you mean for mysql_config? paths? output-formats?
Comment 17 Bernard Spil freebsd_committer freebsd_triage 2018-10-24 16:47:40 UTC
Added in the upstream bugzilla:

I'm trying to fix this as team-member of the apache@ team in FreeBSD which is also the maintainer of the APR packages.

Looks like this issue is NOT FreeBSD specific, MySQL 8 ships different headers than 5.x. The autoconf needs some 

Looking at the config.log I see
> configure:20376: checking for mysql.h
> configure:20376: cc -c -O2 -fno-strict-aliasing -pipe -march=native  -DLIBICONV_PLUG -fstack-protector  -I/usr/ports/devel/apr1/work/apr-1.6.5/include -I/usr/local/include -I/usr/local/include/mysql -DHAVE_MYSQL_H -I/usr/local/include -DLIBICONV_PLUG conftest.c >&5
> conftest.c:21:10: fatal error: 'my_global.h' file not found
> #include <my_global.h>
>          ^~~~~~~~~~~~~
> 1 error generated.
> configure:20376: $? = 1
> configure: failed program was:

MySQL 8.0 does not ship a my_global.h, and APR decides that there's no usable MySQL.
There's a my_global.h in 
   https://github.com/mysql/mysql-server/tree/5.7/include 
but not in 
   https://github.com/mysql/mysql-server/tree/8.0/include

The requirement for my_global.h was introduced in 2008
https://svn.apache.org/viewvc?view=revision&revision=747625
Comment 18 commit-hook freebsd_committer freebsd_triage 2018-10-27 21:47:50 UTC
A commit references this bug:

Author: brnrd
Date: Sat Oct 27 21:46:52 UTC 2018
New revision: 483173
URL: https://svnweb.freebsd.org/changeset/ports/483173

Log:
  devel/apr1: Update APR to 1.6.5

   - MySQL 8.0 (port marked "devel") not compatible [1]

  PR:		230538 [1]
  Reported by:	<please forget me.uk>
  Differential Revision:	https://reviews.freebsd.org/D17688

Changes:
  head/devel/apr1/Makefile
  head/devel/apr1/distinfo
  head/devel/apr1/files/patch-apr__configure
  head/devel/apr1/files/patch-apr_poll_unix_kqueue.c
Comment 19 Bernard Spil freebsd_committer freebsd_triage 2018-10-27 22:08:22 UTC
Please follow up upstream.
I did my part, please assist.
Comment 20 Arnaud de Prelle 2018-10-30 22:02:21 UTC
I just upgraded from MariaDB 10.1 to MariaDB 10.3 (mariadb103-server-10.3.10_1) and I'm encountering the same issue.

===>  Installing for apr-1.6.5.1.6.1
===>  Checking if apr already installed
===>   Registering installation for apr-1.6.5.1.6.1
pkg-static: Unable to access file /usr/ports/devel/apr1/work/stage/usr/local/lib/apr-util-1/apr_dbd_mysql-1.so:No such file or directory
pkg-static: Unable to access file /usr/ports/devel/apr1/work/stage/usr/local/lib/apr-util-1/apr_dbd_mysql.a:No such file or directory
pkg-static: Unable to access file /usr/ports/devel/apr1/work/stage/usr/local/lib/apr-util-1/apr_dbd_mysql.so:No such file or directory
*** Error code 74

Stop.
make[1]: stopped in /usr/ports/devel/apr1
*** Error code 1

Stop.
make: stopped in /usr/ports/devel/apr1
Comment 21 Jochen Neumeister freebsd_committer freebsd_triage 2018-10-31 17:11:53 UTC
(In reply to apn from comment #20)

Ports-Tree up-to-date?
You test it with delete this port and then a new install?
Comment 22 commit-hook freebsd_committer freebsd_triage 2018-10-31 17:35:35 UTC
A commit references this bug:

Author: brnrd
Date: Wed Oct 31 17:35:28 UTC 2018
New revision: 483598
URL: https://svnweb.freebsd.org/changeset/ports/483598

Log:
  databases/mariadb103-client: Fix mariadb_config output

   - port fix from mariadb102-client port

  PR:		226049, 230538
  Reported by:	flo

Changes:
  head/databases/mariadb103-client/Makefile
Comment 23 Bernard Spil freebsd_committer freebsd_triage 2018-10-31 17:43:15 UTC
(In reply to apn from comment #20)
Please check if this is fixed with mariadb103-client-10.3.10_2
Comment 24 Arnaud de Prelle 2018-10-31 18:49:26 UTC
(In reply to Bernard Spil from comment #23)

Hi Bernard, yes it's fixed ! Thanks
Comment 25 Dani I. 2020-01-02 16:19:44 UTC
(In reply to Bernard Spil from comment #23)

Bernard: Could you please take a look at bug #242156 ?
Comment 26 Eugene Grosbein freebsd_committer freebsd_triage 2020-12-26 08:39:42 UTC
The PR was closed by mistake, the problem is still here. Re-open it.
Comment 27 Jochen Neumeister freebsd_committer freebsd_triage 2021-08-26 21:25:19 UTC
does the problem still exist?
Comment 28 avkarenow 2021-09-07 20:58:40 UTC
Unfortunately yes, but the solution mentioned in comments #3 (replace my_bool, remove my_global.h and my_sys.h) and #6 (USES=autoreconf) still works.
I compiled arp1 with MySQL 8.0.26 on FreeBSD 13.0 after that modifications.
Comment 29 Andrey Bushev 2022-04-08 09:12:11 UTC
Created attachment 233047 [details]
Build stopped with this messages
Comment 30 Andrey Bushev 2022-04-08 09:14:19 UTC
Comment on attachment 233047 [details]
Build stopped with this messages

Hi,
Have same problem on FreeBSD 12.3-RELEASE-p5 GENERIC  amd64 with mysql80-client-8.0.28_2.
After replace "my_bool" in file apr_dbd_mysql.c (from solution in Comment #3) apr is build and updated from apr-1.7.0.1.6.1_1 to apr-1.7.0.1.6.1_2.
File dbd.m4 do not changed.
Comment 31 Matthias Mergenthaler 2023-02-16 14:39:38 UTC
I have the same problem building apr1 with mysql support. After changing my_bool to bool everything goes fine. Is it possible to fix this in the offical repository?
Comment 32 Jochen Neumeister freebsd_committer freebsd_triage 2023-02-16 19:48:58 UTC
(In reply to Matthias Mergenthaler from comment #31)

Hello Matthias,

since I am working on making MySQL 8.0 default, how can I reproduce the error? Are you installing MySQL 8.0 with the default settings?
Comment 33 Matthias Mergenthaler 2023-02-16 20:06:04 UTC
(In reply to Jochen Neumeister from comment #32)
Hi Jochen,
yes, I use MySql 8.0 with no changes in the config. 
If I try to make apr1 with the 'MYSQL' option I get a error stating that my_bool is unknown.
If I replace all occurrences of my_bool with just bool everything goes flawless.
Comment 34 Jochen Neumeister freebsd_committer freebsd_triage 2023-02-18 11:22:06 UTC
(In reply to Matthias Mergenthaler from comment #33)

Thank you for the explanation. Did you use the patch from here? Or can you provide your changes in a patch?
Comment 35 Enji Cooper freebsd_committer freebsd_triage 2023-02-27 19:21:40 UTC
my_bool was removed in MySQL, so it sounds like the proposed changes are the right thing to do internally until apr1* is updated with the fix: https://dev.mysql.com/doc/relnotes/mysql/8.0/en/news-8-0-1.html#mysqld-8-0-1-compiling .
Comment 36 Enji Cooper freebsd_committer freebsd_triage 2023-02-27 19:25:06 UTC
Filed upstream as https://bz.apache.org/bugzilla/show_bug.cgi?id=66498 .
Comment 37 ksuzuki 2023-03-29 03:56:26 UTC
(In reply to Jochen Neumeister from comment #34)
The patch for this is contained is comment #3.  It works fine in my poudriere.

--- apr-util-1.6.1/dbd/apr_dbd_mysql.c  2023-03-29 12:35:55.308793000 +0900
+++ apr-util-1.6.1/dbd/apr_dbd_mysql.c.orig     2023-03-29 12:41:10.572540000 +0900
@@ -50,6 +50,14 @@

 #include "apr_dbd_internal.h"

+/* MySQL 8.0 replaces my_bool with C99 bool. Earlier versions of MySQL had
+* a typedef to char. Gem users reported failures on big endian systems when
+* using C99 bool types with older MySQLs due to mismatched behavior. */
+#ifndef HAVE_TYPE_MY_BOOL
+#include <stdbool.h>
+typedef bool my_bool;
+#endif
+
 /* default maximum field size 1 MB */
 #define FIELDSIZE 1048575
Comment 38 commit-hook freebsd_committer freebsd_triage 2023-04-01 15:46:09 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=f473a726dba249ec936d340b8b1167ce5996bd30

commit f473a726dba249ec936d340b8b1167ce5996bd30
Author:     Bernard Spil <brnrd@FreeBSD.org>
AuthorDate: 2023-04-01 15:42:03 +0000
Commit:     Bernard Spil <brnrd@FreeBSD.org>
CommitDate: 2023-04-01 15:42:03 +0000

    devel/apr1: Update to 1.7.3

     * Update APR-util to 1.6.3
     * Fix build with MySQL 8 [2]
     * databases/db5 is depreacated since 2022-06-30 [3]
     * Switch default BDB to 18

    PR:             269857, 230538 [2], 261523 [3]
    Submitted by:   ngie
    With hat:       apache

 devel/apr1/Makefile                                |  15 ++-
 devel/apr1/distinfo                                |  10 +-
 devel/apr1/files/patch-PR59332 (new)               |  18 +++
 devel/apr1/files/patch-PR61517 (gone)              | 126 ---------------------
 ...r-1.7.0_configure => patch-apr-1.7.3_configure} |   4 +-
 ...kqueue.c => patch-apr-1.7.3_poll_unix_kqueue.c} |  14 +--
 devel/apr1/files/patch-bdb18                       |   4 +-
 7 files changed, 43 insertions(+), 148 deletions(-)
Comment 39 Bernard Spil freebsd_committer freebsd_triage 2023-04-01 15:49:24 UTC
Committed patch fixes MySQL 8 whilst not breaking MariaDB.