Bug 255748

Summary: databases/mysql57-server: Update to 5.7.34 (fixes security vulnerabilities??)
Product: Ports & Packages Reporter: Christos Chatzaras <chris>
Component: Individual Port(s)Assignee: Jochen Neumeister <joneum>
Status: Closed FIXED    
Severity: Affects Many People CC: dev, elwood, freebsd, joneum, leres, pi, ports-secteam, shoesoft, titus, trueos
Priority: Normal Keywords: needs-patch, needs-qa, security
Version: LatestFlags: bugzilla: maintainer-feedback? (joneum)
koobs: merge-quarterly?
Hardware: Any   
OS: Any   
URL: https://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-34.html
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=255937
Attachments:
Description Flags
not working patch
none
possible working patch
none
mysql57-server
none
mysql57-client none

Description Christos Chatzaras 2021-05-10 13:18:42 UTC
Update to 5.7.34:

https://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-34.html
Comment 1 Jochen Neumeister freebsd_committer 2021-05-11 17:55:24 UTC
(In reply to Christos Chatzaras from comment #0)

It would be nice if you upload a patch too :)
Comment 2 Christos Chatzaras 2021-05-11 22:46:17 UTC
Created attachment 224862 [details]
not working patch

I tried to compile it with this patch but it failed for both mysql57-server and mysql57-client.

After I remove patch-cmake_build__configurations_compiler__options.cmake from mysql57-client it compiled successfully.

The mysql57-server failed to compile first because of patch-sql_locks_shared__spin__lock.cc which I remove as they fix it in 5.7.34: https://bugs.mysql.com/bug.php?id=102288

But it still fails to compile:

--- storage/myisammrg/CMakeFiles/myisammrg.dir/all ---
--- storage/myisammrg/CMakeFiles/myisammrg.dir/myrg_rnext.c.o ---
[ 29%] Building C object storage/myisammrg/CMakeFiles/myisammrg.dir/myrg_rnext.c.o
cd /usr/ports/databases/mysql57-server/work/.build/storage/myisammrg && /usr/local/bin/clang90 -DHAVE_CONFIG_H -DMYSQL_SERVER -I/usr/ports/databases/mysql57-server/work/.build/include -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/extra/rapidjson/include -I/usr/ports/databases/mysql57-server/work/.build/libbinlogevents/include -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/libbinlogevents/export -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/include -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/sql -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/libbinlogevents/include -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/sql/auth -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/regex -fPIC -fno-omit-frame-pointer -fno-strict-aliasing -O2 -pipe  -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -Wall -Wextra -Wformat-security -Wvla -Wwrite-strings -Wdeclaration-after-statement -ffunction-sections -fdata-sections -O2 -pipe
-fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -MD -MT storage/myisammrg/CMakeFiles/myisammrg.dir/myrg_rnext.c.o -MF CMakeFiles/myisammrg.dir/myrg_rnext.c.o.d -o CMakeFiles/myisammrg.dir/myrg_rnext.c.o -c /usr/ports/databases/mysql57-server/work/mysql-5.7.34/storage/myisammrg/myrg_rnext.c
--- storage/innobase/CMakeFiles/innobase_embedded.dir/all ---
--- storage/innobase/CMakeFiles/innobase_embedded.dir/gis/gis0geo.cc.o ---
[ 29%] Building CXX object storage/innobase/CMakeFiles/innobase_embedded.dir/gis/gis0geo.cc.o
cd /usr/ports/databases/mysql57-server/work/.build/storage/innobase && /usr/local/bin/clang++90 -DCOMPILER_HINTS -DHAVE_CONFIG_H -DHAVE_IB_ATOMIC_PTHREAD_T_GCC=1 -DHAVE_IB_GCC_ATOMIC_COMPARE_EXCHANGE=1 -DHAVE_IB_GCC_ATOMIC_THREAD_FENCE=1 -DHAVE_IB_GCC_SYNC_SYNCHRONISE=1 -DHAVE_LZ4=1 -DHAVE_NANOSLEEP=1 -DMUTEX_EVENT -I/usr/ports/databases/mysql57-server/work/.build/include -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/extra/rapidjson/include -I/usr/ports/databases/mysql57-server/work/.build/libbinlogevents/include -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/libbinlogevents/export -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/storage/innobase/include -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/storage/innobase/handler -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/libbinlogevents/include -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/include -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/sql -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/sql/auth -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/regex -fPIC -fno-omit-frame-pointer -fno-strict-aliasing -O2 -pipe  -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing   -isystem /usr/local/include -std=c++11 -Wall -Wextra -Wformat-security -Wvla -Woverloaded-virtual -Wno-unused-parameter -Wno-null-conversion -Wno-unused-private-field -ffunction-sections -fdata-sections -O2 -pipe  -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing   -isystem /usr/local/include -std=c++11 -fPIC -MD -MT storage/innobase/CMakeFiles/innobase_embedded.dir/gis/gis0geo.cc.o -MF CMakeFiles/innobase_embedded.dir/gis/gis0geo.cc.o.d -o CMakeFiles/innobase_embedded.dir/gis/gis0geo.cc.o -c /usr/ports/databases/mysql57-server/work/mysql-5.7.34/storage/innobase/gis/gis0geo.cc
--- storage/myisammrg/CMakeFiles/myisammrg.dir/all ---
--- storage/myisammrg/CMakeFiles/myisammrg.dir/myrg_rnext_same.c.o ---
[ 29%] Building C object storage/myisammrg/CMakeFiles/myisammrg.dir/myrg_rnext_same.c.o
cd /usr/ports/databases/mysql57-server/work/.build/storage/myisammrg && /usr/local/bin/clang90 -DHAVE_CONFIG_H -DMYSQL_SERVER -I/usr/ports/databases/mysql57-server/work/.build/include -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/extra/rapidjson/include -I/usr/ports/databases/mysql57-server/work/.build/libbinlogevents/include -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/libbinlogevents/export -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/include -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/sql -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/libbinlogevents/include -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/sql/auth -I/usr/ports/databases/mysql57-server/work/mysql-5.7.34/regex -fPIC -fno-omit-frame-pointer -fno-strict-aliasing -O2 -pipe  -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -Wall -Wextra -Wformat-security -Wvla -Wwrite-strings -Wdeclaration-after-statement -ffunction-sections -fdata-sections -O2 -pipe  -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -MD -MT storage/myisammrg/CMakeFiles/myisammrg.dir/myrg_rnext_same.c.o -MF CMakeFiles/myisammrg.dir/myrg_rnext_same.c.o.d -o CMakeFiles/myisammrg.dir/myrg_rnext_same.c.o -c /usr/ports/databases/mysql57-server/work/mysql-5.7.34/storage/myisammrg/myrg_rnext_same.c
--- storage/innobase/CMakeFiles/innobase_embedded.dir/all ---
--- storage/innobase/CMakeFiles/innobase_embedded.dir/fts/fts0que.cc.o ---
2 errors generated.
*** [storage/innobase/CMakeFiles/innobase_embedded.dir/fts/fts0que.cc.o] Error code 1

make[4]: stopped in /usr/ports/databases/mysql57-server/work/.build
*** [all] Error code 6

make[2]: stopped in /usr/ports/databases/mysql57-server/work/.build
1 error

make[2]: stopped in /usr/ports/databases/mysql57-server/work/.build
===> Compilation failed unexpectedly.
Try to set MAKE_JOBS_UNSAFE=yes and rebuild before reporting the failure to
the maintainer.
*** Error code 1

Stop.
make[1]: stopped in /usr/ports/databases/mysql57-server
*** Error code 1

Stop.
make: stopped in /usr/ports/databases/mysql57-server

===>>> make build failed for databases/mysql57-server
===>>> Aborting update


===>>> You can restart from the point of failure with this command line:
       portmaster <flags> databases/mysql57-server

This command has been saved to ~/portmasterfail.txt
Comment 3 titus m 2021-05-20 10:09:27 UTC
patch
work/mysql-5.7.34/sql/locks/shared_spin_lock.cc
at line 246 make both branches like #ifdef APPLE
my_atomic_store64(&this->m_exclusive_owner, reinterpret_cast<int64>(self));
change port Makefile
at CMAKE_ARGS => WITH_DEBUG=1 (from WITH_DEBUG=0)
kill all the patches in files/*
and it builds (with lots of warnings)
Comment 4 Christos Chatzaras 2021-05-20 12:10:03 UTC
(In reply to titus m from comment #3)

Thank you for the help. I will upload a patch in few hours.
Comment 5 Christos Chatzaras 2021-05-20 13:39:33 UTC
Created attachment 225122 [details]
possible working patch

I create this patch with these steps:

cd /usr/ports/databases

cp -frp mysql57-server mysql57-server.orig

cd /usr/ports/databases/mysql57-server

nano -w Makefile (I change version and enable debug)

make makesum

make extract

cp work/mysql-5.7.34/sql/locks/shared_spin_lock.cc work/mysql-5.7.34/sql/locks/shared_spin_lock.cc.orig

nano -w work/mysql-5.7.34/sql/locks/shared_spin_lock.cc (I did the change described in Comment 3)

make makepatch

cd /usr/ports/databases

diff -ruN mysql57-server.orig mysql57-server > /root/mysql57-server-5.7.34.diff

-----


Then I run these commands:

gitup ports (to return to previous version)

cd /usr/ports/databases/mysql57-server

patch < /root/mysql57-server-5.7.34.diff

portmaster --no-confirm -y mysql57-server-5.7.33 mysql57-client-5.7.33

But it fails:

https://pastebin.com/raw/hg9vKUcR

-----

If I run these commands it installs 5.7.34 successfully:

pkg del mysql57-server mysql57-client

cd /usr/ports/databases/mysql57-server

make install clean

-----

Any idea what is going wrong?
Comment 6 Christos Chatzaras 2021-05-20 14:40:43 UTC
Ignore my previous message. I forgot to remove patch-cmake_build__configurations_compiler__options.cmake from mysql57-client

The patches for both mysql57-client and mysql57-server work.

I run these commands:

cd /usr/ports/databases

patch -p0 < /root/mysql57-server-5.7.34.diff

patch -p0 -E < /root/mysql57-client-5.7.34.diff

portmaster --no-confirm -y mysql57-server-5.7.33 mysql57-client-5.7.33

and it upgrade it successfully.


-------


Patch for mysql57-server:

https://pastebin.com/raw/jzMiKhiU

Patch for mysql57-client:

https://pastebin.com/raw/dc8XGRKh
Comment 7 Kubilay Kocak freebsd_committer freebsd_triage 2021-05-24 11:08:24 UTC
(In reply to Christos Chatzaras from comment #6)

Please include any proposed patches as attachments, thanks!
Comment 8 Kubilay Kocak freebsd_committer freebsd_triage 2021-05-24 11:21:58 UTC
The mysql relnotes only stipulate a single security reference:

The linked OpenSSL library for MySQL Server has been updated to version 1.1.1k. Issues fixed in the new OpenSSL version are described at https://www.openssl.org/news/cl111.txt and https://www.openssl.org/news/vulnerabilities.html. (Bug #32680637)

The oracle MySQL vulnerability matrix references additional CVE's for versions '5.7.33 and prior' that dont reference openssl:

https://www.oracle.com/security-alerts/cpuapr2021.html#AppendixMSQL

The bundling of all cve's against all mysql versions in the vuxml entry doesnt make things easier to understand either.

What's the situation/reality with respect to 5.7.x ?
Comment 9 Christos Chatzaras 2021-05-24 11:23:21 UTC
Created attachment 225217 [details]
mysql57-server
Comment 10 Christos Chatzaras 2021-05-24 11:23:50 UTC
Created attachment 225218 [details]
mysql57-client
Comment 11 commit-hook freebsd_committer 2021-05-30 09:42:12 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=ebbb2d513387c14bbb667f88e84ffbdb87a4dda3

commit ebbb2d513387c14bbb667f88e84ffbdb87a4dda3
Author:     Dima Panov <fluffy@FreeBSD.org>
AuthorDate: 2021-05-30 09:39:09 +0000
Commit:     Dima Panov <fluffy@FreeBSD.org>
CommitDate: 2021-05-30 09:39:09 +0000

    databases/mysql57-*: update to 5.7.32 GA release

    Release Notes: https://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-34.html

    PR:     255748 (based on)
    MFH:    2021Q2
    Approved by:    joneum@ (implicit)

 ...e_build__configurations_compiler__options.cmake | 16 +++++++--------
 databases/mysql57-server/Makefile                  |  4 +++-
 databases/mysql57-server/distinfo                  |  6 +++---
 ...d__configurations_compiler__options.cmake (new) | 20 ++++++++++++++++++
 ...eplication_src_recovery_state_transfer.cc (new) | 20 ++++++++++++++++++
 .../mysql57-server/files/patch-sql_binlog.cc (new) | 24 ++++++++++++++++++++++
 .../files/patch-sql_locks_shared__spin__lock.cc    | 12 +++++------
 .../patch-storage_innobase_fsp_fsp0fsp.cc (new)    | 11 ++++++++++
 8 files changed, 95 insertions(+), 18 deletions(-)
Comment 12 commit-hook freebsd_committer 2021-05-30 09:43:14 UTC
A commit in branch 2021Q2 references this bug:

URL: https://cgit.FreeBSD.org/ports/commit/?id=f09ca5935793c8a522d926cc4c7b3fe5807e4d3b

commit f09ca5935793c8a522d926cc4c7b3fe5807e4d3b
Author:     Dima Panov <fluffy@FreeBSD.org>
AuthorDate: 2021-05-30 09:39:09 +0000
Commit:     Dima Panov <fluffy@FreeBSD.org>
CommitDate: 2021-05-30 09:42:00 +0000

    databases/mysql57-*: update to 5.7.32 GA release

    Release Notes: https://dev.mysql.com/doc/relnotes/mysql/5.7/en/news-5-7-34.html

    PR:     255748 (based on)
    MFH:    2021Q2
    Approved by:    joneum@ (implicit)

    (cherry picked from commit ebbb2d513387c14bbb667f88e84ffbdb87a4dda3)

 ...e_build__configurations_compiler__options.cmake | 16 +++++++--------
 databases/mysql57-server/Makefile                  |  4 +++-
 databases/mysql57-server/distinfo                  |  6 +++---
 ...d__configurations_compiler__options.cmake (new) | 20 ++++++++++++++++++
 ...eplication_src_recovery_state_transfer.cc (new) | 20 ++++++++++++++++++
 .../mysql57-server/files/patch-sql_binlog.cc (new) | 24 ++++++++++++++++++++++
 .../files/patch-sql_locks_shared__spin__lock.cc    | 12 +++++------
 .../patch-storage_innobase_fsp_fsp0fsp.cc (new)    | 11 ++++++++++
 8 files changed, 95 insertions(+), 18 deletions(-)
Comment 13 Christos Chatzaras 2021-05-30 10:18:11 UTC
The commit title says about 5.7.32 and not 5.7.34.
Comment 14 Stefan Ehmann 2021-05-30 17:32:38 UTC
The update seems to break kmail for me:

Console shows this while akonadi server/mysql is starting:
Assertion failed: (param->buffer_length != 0), function setup_one_fetch_function, file /wrkdirs/usr/ports/databases/mysql57-client/work/mysql-5.7.34/libmysql/libmysql.c, line 4112
Comment 15 freebsd 2021-05-31 10:56:03 UTC
This update just installs a new /usr/loal/etc/mysql/my.cnf with values that do not necessarily reflect the installed MySQL. 

In two separate instances, I had to change the MySQL path (not using /var/db/mysql, which my.cnf does; it is set corectly in /etc/rc.conf!) and the InnoDB value (which my.cnf sets as 128 MB, which may not reflect its actual size at the moment of startup - meaning it will throw an error and refuse to start - I had to change it to 76M twice).

Please have the port NOT overwrite an existing my.cnf and/or just install a .sample or .original.
Comment 16 Florian Walpen 2021-05-31 14:46:52 UTC
(In reply to Stefan Ehmann from comment #14)

> Assertion failed: (param->buffer_length != 0), function setup_one_fetch_function, file /wrkdirs/usr/ports/databases/mysql57-client/work/mysql-5.7.34/libmysql/libmysql.c, line 4112

Somehow libmysql.c is compiled without the "-DNDEBUG" define. Not sure why, some parts are built with the flag set, and CMAKE Release or RelWithDebInfo builds usually apply this flag.

Didn't investigate the build system and changes any further, but adding

CFLAGS+=       -DNDEBUG
CXXFLAGS+=     -DNDEBUG

to "databases/mysql57-client/Makefile" brings akonadi back to life.
Comment 17 Stefan Ehmann 2021-05-31 20:54:57 UTC
(In reply to Florian Walpen from comment #16)
Can conform the workaround.

Not sure if the assert is bogus or if there is a bug somewhere.
Comment 18 Florian Walpen 2021-05-31 22:05:51 UTC
(In reply to Stefan Ehmann from comment #17)

> Not sure if the assert is bogus or if there is a bug somewhere.

These are two separate issues, IMHO. It's very likely that akonadi (or qt5-sqldrivers-mysql) does something bad here. Or at least something unexpected from the MySQL developers' view. I've seen the same bug reported for gentoo linux in 2009, so it's probably a persistent issue with akonadi.

But assertions are meant for development purposes. They should not be present in release builds, I think we have to fix that anyway.
Comment 19 elwood 2021-06-16 17:50:37 UTC
(In reply to Florian Walpen from comment #16)

Thanks for your workaround, Florian!

It will run for me with digikam and "MySQL Database support"-Ports option too!!
Comment 20 Jochen Neumeister freebsd_committer 2021-06-16 19:00:10 UTC
I unfortunately destroyed my poudriere server today. I need a few days to reinstall it. After that I will take care of the problem
Comment 21 Jochen Neumeister freebsd_committer 2021-08-26 21:23:00 UTC
MySQL is now in Version 5.7.35 into the Ports