Bug 239732 - databases/mariadb103-server: Variable 'innodb_compression_algorithm' can't be set to the value of 'lz4'
Summary: databases/mariadb103-server: Variable 'innodb_compression_algorithm' can't be...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Bernard Spil
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-08-09 06:25 UTC by TAO ZHOU
Modified: 2019-12-22 10:46 UTC (History)
2 users (show)

See Also:
bugzilla: maintainer-feedback? (brnrd)


Attachments
Patch to support lz4 compression (763 bytes, patch)
2019-08-12 03:51 UTC, TAO ZHOU
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description TAO ZHOU 2019-08-09 06:25:21 UTC
lz4 support is missing event if I selected ticked it in poudriere options.
It also shows 'on' in `pkg info`.

```
mariadb103-server-10.3.16_1
Name           : mariadb103-server
Version        : 10.3.16_1
Installed on   : Fri Aug  9 15:17:15 2019 AEST
Origin         : databases/mariadb103-server
Architecture   : FreeBSD:11:amd64
Prefix         : /usr/local
Categories     : databases ipv6
Licenses       : GPLv2
Maintainer     : brnrd@FreeBSD.org
WWW            : https://mariadb.org/
Comment        : Multithreaded SQL database (server)
Options        :
	CONNECT_EXTRA  : on
	DOCS           : on
	GSSAPI_BASE    : off
	GSSAPI_HEIMDAL : on
	GSSAPI_MIT     : off
	GSSAPI_NONE    : off
	INNOBASE       : on
	LZ4            : on
	LZO            : off
	MROONGA        : off
	MSGPACK        : off
	OQGRAPH        : off
	ROCKSDB        : off
	SNAPPY         : off
	SPHINX         : on
	SPIDER         : on
	TOKUDB         : off
	WSREP          : on
	ZMQ            : off
	ZSTD           : off

```

I could see 'liblz4' from ldd output.
```
~ # ldd /usr/local/libexec/mysqld
/usr/local/libexec/mysqld:
	liblzma.so.5 => /usr/lib/liblzma.so.5 (0x80181b000)
	libbz2.so.4 => /usr/lib/libbz2.so.4 (0x801a44000)
	libz.so.6 => /lib/libz.so.6 (0x801c58000)
	libm.so.5 => /lib/libm.so.5 (0x801e70000)
	libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x80209d000)
	libwrap.so.6 => /usr/lib/libwrap.so.6 (0x8022a0000)
	libcrypt.so.5 => /lib/libcrypt.so.5 (0x8024a9000)
	libssl.so.11 => /usr/local/lib/libssl.so.11 (0x8026c8000)
	libcrypto.so.11 => /usr/local/lib/libcrypto.so.11 (0x802a00000)
	libc++.so.1 => /usr/lib/libc++.so.1 (0x802eec000)
	libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x8031ba000)
	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x8033d9000)
	libthr.so.3 => /lib/libthr.so.3 (0x8035e8000)
	libc.so.7 => /lib/libc.so.7 (0x803810000)
	libelf.so.2 => /lib/libelf.so.2 (0x803bcc000)
```

According to this link https://mariadb.com/kb/en/library/compression/+comments/2002, the CMakeCache.txt should contain

```
WITH_INNODB_LZ4:STRING=AUTO
HAVE_LZ4_H:INTERNAL=1
HAVE_LZ4_SHARED_LIB:INTERNAL=1
```
But when I looked at CMakeCache.txt in poudriere, it only had
```
WITH_INNODB_LZ4:BOOL=true
```
I think this might be the problem.
Comment 1 TAO ZHOU 2019-08-12 03:51:29 UTC
Created attachment 206463 [details]
Patch to support lz4 compression
Comment 2 TAO ZHOU 2019-08-12 03:52:26 UTC
I have successfully compiled mariadb with lz4 support with the above patch.
Comment 3 ari 2019-09-09 04:00:38 UTC
Is there anything we can do to help get this merged?
Comment 4 Bernard Spil freebsd_committer 2019-12-11 16:30:45 UTC
LZ4 hooks into multiple things. In your case, apparently the innobase (or xtradb?!) engine is lacking LZ4 support

> % grep LZ4 work/mariadb-10.4.11/CMakeCache.txt 
> //use bundled LZ4
> GRN_WITH_BUNDLED_LZ4:BOOL=OFF
> GRN_WITH_LZ4:BOOL=true
> LZ4_INCLUDE_DIR:PATH=/usr/local/include
> LZ4_LIBRARY:FILEPATH=/usr/local/lib/liblz4.so
> WITH_INNODB_LZ4:BOOL=true
> WITH_ROCKSDB_LZ4:BOOL=true
> //ADVANCED property for variable: LZ4_INCLUDE_DIR
> LZ4_INCLUDE_DIR-ADVANCED:INTERNAL=1
> //ADVANCED property for variable: LZ4_LIBRARY
> LZ4_LIBRARY-ADVANCED:INTERNAL=1

From innobase lz4.cmake:
>  IF (WITH_INNODB_LZ4 STREQUAL "ON" OR WITH_INNODB_LZ4 STREQUAL "AUTO")

So probably BOOL doesn't work here. Will adapt, probably more things that don't work as expected!
Comment 5 commit-hook freebsd_committer 2019-12-14 14:34:17 UTC
A commit references this bug:

Author: brnrd
Date: Sat Dec 14 14:34:11 UTC 2019
New revision: 520119
URL: https://svnweb.freebsd.org/changeset/ports/520119

Log:
  databases/mariadb104-server: Update to 10.4.11

   * Fix compression [1]
   * Fix mariabackup-based replication [2]

  PR:		239732 [1], 240692 [2]
  Submitted by:	Tao Zhou <zhoutao laocius org> [1], Mike Andrews <mandrews bit0 com> [2]

Changes:
  head/databases/mariadb104-server/Makefile
  head/databases/mariadb104-server/distinfo
  head/databases/mariadb104-server/files/patch-scripts_wsrep__sst__mariabackup.sh
Comment 6 TAO ZHOU 2019-12-16 23:20:49 UTC
Just built mariadb104-server. LZ4 is still not supported.


~ # ldd /usr/local/libexec/mysqld
/usr/local/libexec/mysqld:
	liblzma.so.5 => /usr/lib/liblzma.so.5 (0x801380000)
	libbz2.so.4 => /usr/lib/libbz2.so.4 (0x8013ac000)
	libz.so.6 => /lib/libz.so.6 (0x8013c0000)
	libm.so.5 => /lib/libm.so.5 (0x8013da000)
	libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x80140c000)
	libwrap.so.6 => /usr/lib/libwrap.so.6 (0x801411000)
	libcrypt.so.5 => /lib/libcrypt.so.5 (0x80141d000)
	libssl.so.11 => /usr/local/lib/libssl.so.11 (0x80143e000)
	libcrypto.so.11 => /usr/local/lib/libcrypto.so.11 (0x8014d1000)
	libthr.so.3 => /lib/libthr.so.3 (0x8017b8000)
	libdl.so.1 => /usr/lib/libdl.so.1 (0x8017e5000)
	libc++.so.1 => /usr/lib/libc++.so.1 (0x8017e9000)
	libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x8018b9000)
	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x8018db000)
	libc.so.7 => /lib/libc.so.7 (0x8018f5000)
	libelf.so.2 => /lib/libelf.so.2 (0x801cee000)
Comment 7 ari 2019-12-21 12:36:12 UTC
Connfirmed. brnrd you did not apply the patch Tao provided and LZ4 still doesn't work.
Comment 8 ari 2019-12-21 12:53:14 UTC
I take that back. Its a bit confusing since this bug is for the 103 version, but I see you patched the 104 version. I tested 103.
Comment 9 Bernard Spil freebsd_committer 2019-12-21 13:18:45 UTC
The patch (In reply to ari from comment #7)
Correct, I did not apply the patch as that patch doesn't make sense to me, there is no _CMAKE_STRING as far as I can see.

https://svnweb.freebsd.org/ports/head/Mk/Uses/cmake.mk?revision=488341&view=markup

If you'd checked the commit for 10.4 you an see that it uses _CMAKE_ON and _CMAKE_OFF to work around this in MariaDB.
Comment 10 Bernard Spil freebsd_committer 2019-12-21 13:22:09 UTC
From CMakeCache after the changes:

> GRN_WITH_BUNDLED_LZ4:BOOL=OFF
> GRN_WITH_LZ4:STRING=ON
> LZ4_INCLUDE_DIR:PATH=/usr/local/include
> LZ4_LIBRARY:FILEPATH=/usr/local/lib/liblz4.so
> LZ4_LIBS:FILEPATH=/usr/local/lib/liblz4.so
> WITH_INNODB_LZ4:STRING=ON
> WITH_ROCKSDB_LZ4:STRING=ON
Comment 11 commit-hook freebsd_committer 2019-12-21 16:22:07 UTC
A commit references this bug:

Author: brnrd
Date: Sat Dec 21 16:21:40 UTC 2019
New revision: 520571
URL: https://svnweb.freebsd.org/changeset/ports/520571

Log:
  databases/mariadb103-server: Update to 10.3.21

   - Fix table compression [1]
   - Fix link error in mariabackup [2]
   - Fix GNUism [3]

  PR:		239732 [1], 236101 [2], 240692 [3]
  Submitted by:	<iron udjin gmail com> [2], Mike Andrews <mandrews bit0 com> [3]
  Reported by:	Tao Zhou <zhoutao laocius org> [1]

Changes:
  head/databases/mariadb103-server/Makefile
  head/databases/mariadb103-server/distinfo
  head/databases/mariadb103-server/files/patch-extra_mariabackup_CMakeLists.txt
  head/databases/mariadb103-server/files/patch-scripts_wsrep__sst__mariabackup.sh
Comment 12 commit-hook freebsd_committer 2019-12-21 17:46:19 UTC
A commit references this bug:

Author: brnrd
Date: Sat Dec 21 17:45:18 UTC 2019
New revision: 520572
URL: https://svnweb.freebsd.org/changeset/ports/520572

Log:
  databases/mariadb102-server: Update to 10.2.30

     - Fix table compression [1]
     - Fix link error in mariabackup [2]
     - Fix GNUism [3]

  PR:		239732 [1], 236101 [2], 240692 [3]
  Submitted by:	<iron udjin gmail com> [2], Mike Andrews <mandrews bit0 com> [3]
  Reported by:	Tao Zhou <zhoutao laocius org> [1]

Changes:
  head/databases/mariadb102-server/Makefile
  head/databases/mariadb102-server/distinfo
  head/databases/mariadb102-server/files/patch-extra_mariabackup_CMakeLists.txt
  head/databases/mariadb102-server/files/patch-scripts_wsrep__sst__mariabackup.sh
Comment 13 commit-hook freebsd_committer 2019-12-21 20:51:37 UTC
A commit references this bug:

Author: brnrd
Date: Sat Dec 21 20:51:19 UTC 2019
New revision: 520581
URL: https://svnweb.freebsd.org/changeset/ports/520581

Log:
  database/mariadb101-server: Various fixes

   - Fix table compression [1]
   - Fix GNUism [2]

  PR:		239732 [1], 240692 [2]
  Submitted by:	Mike Andrews <mandrews bit0 com> [3]
  Reported by:	Tao Zhou <zhoutao laocius org> [1]

Changes:
  head/databases/mariadb101-server/Makefile
  head/databases/mariadb101-server/files/patch-scripts_wsrep__sst__mariabackup.sh