Bug 239732 - databases/mariadb103-server: Variable 'innodb_compression_algorithm' can't be set to the value of 'lz4'
Summary: databases/mariadb103-server: Variable 'innodb_compression_algorithm' can't be...
Status: Closed FIXED
Alias: None
Product: Ports & Packages
Classification: Unclassified
Component: Individual Port(s) (show other bugs)
Version: Latest
Hardware: Any Any
: --- Affects Many People
Assignee: Bernard Spil
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-08-09 06:25 UTC by TAO ZHOU
Modified: 2019-12-22 10:46 UTC (History)
2 users (show)

See Also:
bugzilla: maintainer-feedback? (brnrd)

Attachments
Patch to support lz4 compression (763 bytes, patch)
2019-08-12 03:51 UTC, TAO ZHOU 		no flags Details | Diff
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description TAO ZHOU 2019-08-09 06:25:21 UTC 
lz4 support is missing event if I selected ticked it in poudriere options.
It also shows 'on' in `pkg info`.

```
mariadb103-server-10.3.16_1
Name           : mariadb103-server
Version        : 10.3.16_1
Installed on   : Fri Aug  9 15:17:15 2019 AEST
Origin         : databases/mariadb103-server
Architecture   : FreeBSD:11:amd64
Prefix         : /usr/local
Categories     : databases ipv6
Licenses       : GPLv2
Maintainer     : brnrd@FreeBSD.org
WWW            : https://mariadb.org/
Comment        : Multithreaded SQL database (server)
Options        :
	CONNECT_EXTRA  : on
	DOCS           : on
	GSSAPI_BASE    : off
	GSSAPI_HEIMDAL : on
	GSSAPI_MIT     : off
	GSSAPI_NONE    : off
	INNOBASE       : on
	LZ4            : on
	LZO            : off
	MROONGA        : off
	MSGPACK        : off
	OQGRAPH        : off
	ROCKSDB        : off
	SNAPPY         : off
	SPHINX         : on
	SPIDER         : on
	TOKUDB         : off
	WSREP          : on
	ZMQ            : off
	ZSTD           : off

```

I could see 'liblz4' from ldd output.
```
~ # ldd /usr/local/libexec/mysqld
/usr/local/libexec/mysqld:
	liblzma.so.5 => /usr/lib/liblzma.so.5 (0x80181b000)
	libbz2.so.4 => /usr/lib/libbz2.so.4 (0x801a44000)
	libz.so.6 => /lib/libz.so.6 (0x801c58000)
	libm.so.5 => /lib/libm.so.5 (0x801e70000)
	libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x80209d000)
	libwrap.so.6 => /usr/lib/libwrap.so.6 (0x8022a0000)
	libcrypt.so.5 => /lib/libcrypt.so.5 (0x8024a9000)
	libssl.so.11 => /usr/local/lib/libssl.so.11 (0x8026c8000)
	libcrypto.so.11 => /usr/local/lib/libcrypto.so.11 (0x802a00000)
	libc++.so.1 => /usr/lib/libc++.so.1 (0x802eec000)
	libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x8031ba000)
	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x8033d9000)
	libthr.so.3 => /lib/libthr.so.3 (0x8035e8000)
	libc.so.7 => /lib/libc.so.7 (0x803810000)
	libelf.so.2 => /lib/libelf.so.2 (0x803bcc000)
```

According to this link https://mariadb.com/kb/en/library/compression/+comments/2002, the CMakeCache.txt should contain

```
WITH_INNODB_LZ4:STRING=AUTO
HAVE_LZ4_H:INTERNAL=1
HAVE_LZ4_SHARED_LIB:INTERNAL=1
```
But when I looked at CMakeCache.txt in poudriere, it only had
```
WITH_INNODB_LZ4:BOOL=true
```
I think this might be the problem.
Comment 1 TAO ZHOU 2019-08-12 03:51:29 UTC 
Created attachment 206463 [details]
Patch to support lz4 compression
Comment 2 TAO ZHOU 2019-08-12 03:52:26 UTC 
I have successfully compiled mariadb with lz4 support with the above patch.
Comment 3 ari 2019-09-09 04:00:38 UTC 
Is there anything we can do to help get this merged?
Comment 4 Bernard Spil freebsd_committer freebsd_triage 2019-12-11 16:30:45 UTC 
LZ4 hooks into multiple things. In your case, apparently the innobase (or xtradb?!) engine is lacking LZ4 support

> % grep LZ4 work/mariadb-10.4.11/CMakeCache.txt 
> //use bundled LZ4
> GRN_WITH_BUNDLED_LZ4:BOOL=OFF
> GRN_WITH_LZ4:BOOL=true
> LZ4_INCLUDE_DIR:PATH=/usr/local/include
> LZ4_LIBRARY:FILEPATH=/usr/local/lib/liblz4.so
> WITH_INNODB_LZ4:BOOL=true
> WITH_ROCKSDB_LZ4:BOOL=true
> //ADVANCED property for variable: LZ4_INCLUDE_DIR
> LZ4_INCLUDE_DIR-ADVANCED:INTERNAL=1
> //ADVANCED property for variable: LZ4_LIBRARY
> LZ4_LIBRARY-ADVANCED:INTERNAL=1

From innobase lz4.cmake:
>  IF (WITH_INNODB_LZ4 STREQUAL "ON" OR WITH_INNODB_LZ4 STREQUAL "AUTO")

So probably BOOL doesn't work here. Will adapt, probably more things that don't work as expected!
Comment 5 commit-hook freebsd_committer freebsd_triage 2019-12-14 14:34:17 UTC 
A commit references this bug:

Author: brnrd
Date: Sat Dec 14 14:34:11 UTC 2019
New revision: 520119
URL: https://svnweb.freebsd.org/changeset/ports/520119

Log:
  databases/mariadb104-server: Update to 10.4.11

   * Fix compression [1]
   * Fix mariabackup-based replication [2]

  PR:		239732 [1], 240692 [2]
  Submitted by:	Tao Zhou <zhoutao laocius org> [1], Mike Andrews <mandrews bit0 com> [2]

Changes:
  head/databases/mariadb104-server/Makefile
  head/databases/mariadb104-server/distinfo
  head/databases/mariadb104-server/files/patch-scripts_wsrep__sst__mariabackup.sh
Comment 6 TAO ZHOU 2019-12-16 23:20:49 UTC 
Just built mariadb104-server. LZ4 is still not supported.


~ # ldd /usr/local/libexec/mysqld
/usr/local/libexec/mysqld:
	liblzma.so.5 => /usr/lib/liblzma.so.5 (0x801380000)
	libbz2.so.4 => /usr/lib/libbz2.so.4 (0x8013ac000)
	libz.so.6 => /lib/libz.so.6 (0x8013c0000)
	libm.so.5 => /lib/libm.so.5 (0x8013da000)
	libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x80140c000)
	libwrap.so.6 => /usr/lib/libwrap.so.6 (0x801411000)
	libcrypt.so.5 => /lib/libcrypt.so.5 (0x80141d000)
	libssl.so.11 => /usr/local/lib/libssl.so.11 (0x80143e000)
	libcrypto.so.11 => /usr/local/lib/libcrypto.so.11 (0x8014d1000)
	libthr.so.3 => /lib/libthr.so.3 (0x8017b8000)
	libdl.so.1 => /usr/lib/libdl.so.1 (0x8017e5000)
	libc++.so.1 => /usr/lib/libc++.so.1 (0x8017e9000)
	libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x8018b9000)
	libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x8018db000)
	libc.so.7 => /lib/libc.so.7 (0x8018f5000)
	libelf.so.2 => /lib/libelf.so.2 (0x801cee000)
Comment 7 ari 2019-12-21 12:36:12 UTC 
Connfirmed. brnrd you did not apply the patch Tao provided and LZ4 still doesn't work.
Comment 8 ari 2019-12-21 12:53:14 UTC 
I take that back. Its a bit confusing since this bug is for the 103 version, but I see you patched the 104 version. I tested 103.
Comment 9 Bernard Spil freebsd_committer freebsd_triage 2019-12-21 13:18:45 UTC 
The patch (In reply to ari from comment #7)
Correct, I did not apply the patch as that patch doesn't make sense to me, there is no _CMAKE_STRING as far as I can see.

https://svnweb.freebsd.org/ports/head/Mk/Uses/cmake.mk?revision=488341&view=markup

If you'd checked the commit for 10.4 you an see that it uses _CMAKE_ON and _CMAKE_OFF to work around this in MariaDB.
Comment 10 Bernard Spil freebsd_committer freebsd_triage 2019-12-21 13:22:09 UTC 
From CMakeCache after the changes:

> GRN_WITH_BUNDLED_LZ4:BOOL=OFF
> GRN_WITH_LZ4:STRING=ON
> LZ4_INCLUDE_DIR:PATH=/usr/local/include
> LZ4_LIBRARY:FILEPATH=/usr/local/lib/liblz4.so
> LZ4_LIBS:FILEPATH=/usr/local/lib/liblz4.so
> WITH_INNODB_LZ4:STRING=ON
> WITH_ROCKSDB_LZ4:STRING=ON
Comment 11 commit-hook freebsd_committer freebsd_triage 2019-12-21 16:22:07 UTC 
A commit references this bug:

Author: brnrd
Date: Sat Dec 21 16:21:40 UTC 2019
New revision: 520571
URL: https://svnweb.freebsd.org/changeset/ports/520571

Log:
  databases/mariadb103-server: Update to 10.3.21

   - Fix table compression [1]
   - Fix link error in mariabackup [2]
   - Fix GNUism [3]

  PR:		239732 [1], 236101 [2], 240692 [3]
  Submitted by:	<iron udjin gmail com> [2], Mike Andrews <mandrews bit0 com> [3]
  Reported by:	Tao Zhou <zhoutao laocius org> [1]

Changes:
  head/databases/mariadb103-server/Makefile
  head/databases/mariadb103-server/distinfo
  head/databases/mariadb103-server/files/patch-extra_mariabackup_CMakeLists.txt
  head/databases/mariadb103-server/files/patch-scripts_wsrep__sst__mariabackup.sh
Comment 12 commit-hook freebsd_committer freebsd_triage 2019-12-21 17:46:19 UTC 
A commit references this bug:

Author: brnrd
Date: Sat Dec 21 17:45:18 UTC 2019
New revision: 520572
URL: https://svnweb.freebsd.org/changeset/ports/520572

Log:
  databases/mariadb102-server: Update to 10.2.30

     - Fix table compression [1]
     - Fix link error in mariabackup [2]
     - Fix GNUism [3]

  PR:		239732 [1], 236101 [2], 240692 [3]
  Submitted by:	<iron udjin gmail com> [2], Mike Andrews <mandrews bit0 com> [3]
  Reported by:	Tao Zhou <zhoutao laocius org> [1]

Changes:
  head/databases/mariadb102-server/Makefile
  head/databases/mariadb102-server/distinfo
  head/databases/mariadb102-server/files/patch-extra_mariabackup_CMakeLists.txt
  head/databases/mariadb102-server/files/patch-scripts_wsrep__sst__mariabackup.sh
Comment 13 commit-hook freebsd_committer freebsd_triage 2019-12-21 20:51:37 UTC 
A commit references this bug:

Author: brnrd
Date: Sat Dec 21 20:51:19 UTC 2019
New revision: 520581
URL: https://svnweb.freebsd.org/changeset/ports/520581

Log:
  database/mariadb101-server: Various fixes

   - Fix table compression [1]
   - Fix GNUism [2]

  PR:		239732 [1], 240692 [2]
  Submitted by:	Mike Andrews <mandrews bit0 com> [3]
  Reported by:	Tao Zhou <zhoutao laocius org> [1]

Changes:
  head/databases/mariadb101-server/Makefile
  head/databases/mariadb101-server/files/patch-scripts_wsrep__sst__mariabackup.sh