Bug 247413

Summary: databases/mariadb104-server: server won't start on clean install
Product: Ports & Packages Reporter: Henrik Hudson <ports>
Component: Individual Port(s)Assignee: Bernard Spil <brnrd>
Status: Closed FIXED    
Severity: Affects Many People CC: brnrd, george, maniactraq, nnikonov, ports, vvd
Priority: --- Flags: bugzilla: maintainer-feedback? (brnrd)
Version: Latest   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
mariadb server.cnf
none
add MARIADB_GROUP to SUB_LIST and create rundir before starting the server
none
get effective group via id
none
set permissions on /var/run/mysql/ correctly brnrd: maintainer-approval+

Description Henrik Hudson 2020-06-19 12:19:45 UTC
Created attachment 215774 [details]
mariadb server.cnf

This is weird.

FreeBSD 11.3 installs on a AWS AMI. I use ansible to deploy configurations. I have now replicated this on 2 clean installs that did not have MariaDB installed at all and no filesystems outside of a basic UFS / . DB is running on added ZFS disks. 

I have migrated my my.cnf into the new mysql/conf.d/server.cnf . My /etc/rc.conf has:
mysql_enable="YES"
mysql_dbdir="/data/db/mysqldata"

On a clean install if I run:
/usr/local/etc/rc.d/mysql-server start

it's showing that it builds my new tables, etc...and log files and the mysql DB get created. However, after about 5 seconds it exits. No MariaDB running. I can run 'start' multiple times and the same thing.

If I run:
cd '/usr/local' ; /usr/local/bin/mysqld_safe --datadir='/data/db/mysqldata/'

it starts with no issue. I kill the processes. Now, the rc.d script works fine and starts immediately.

I have attached my server.cnf. The other config files 2 are stock. It seems running mysqld_safe sets something somewhere or caches something that the rc.d script isn't picking up or setting?

Outside of adding the 'innodb_doublewrite', 'innodb_flush_method', and 'innodb_log_group_home_dir' options per the comments for ZFS I have been using the same config more or less for years.
Comment 1 Henrik Hudson 2020-06-19 19:13:10 UTC
I have replicated with a stock install and making the following changes only:

rc.conf:
mysql_enable="YES"
mysql_dbdir="/data/db/mysqldata"

server.cnf:
datadir                         = /data/db/mysqldata

Same issues. Watching 'ps aux' and 'top' output the rc.d script never tries to even start SQL it seems.

I added an echo before the 'run_rc_command' to the rc.d script to see the variables from the 'command_args=' line and they look correct.

Running:
/usr/sbin/daemon -c /usr/local/bin/mysqld_safe --defaults-extra-file=/data/db/mysqldata/my.cnf --user=mysql --datadir=/data/db/mysqldata --pid-file=/data/db/mysqldata/hostname.pid

manually works fine. Again; after doing this the rc.d script works fine.
Comment 2 Henrik Hudson 2020-06-19 19:28:54 UTC
I did a full stock install and removed the '-f' from the command_args in the rc.d script and now I think we're seeing the error:

mkdir: /var/run/mysql: Permission denied
200619 19:23:36 mysqld_safe Fatal error Can't create database directory '/var/run/mysql/mysql.sock'

So, we need to create the  /var/run/mysql directory and give "mysql:mysql" owner:group when the startup script runs it would seem as a pre-start? or as a post install step?

side note: personally, I like seeing the output and would prefer not having that '-f' in there to send output to /dev/null . Had that not been in there I think this would have been solved a lot faster :)

thanks.
Comment 3 Bernard Spil freebsd_committer freebsd_triage 2020-06-21 13:18:52 UTC
With the new config, the port should create the /var/run/mysql dir on install.

Will add that to the port, thanks for reporting!
Comment 4 Nikolay Nikonov 2020-06-25 02:39:48 UTC
I've stumbled onto related problem today when pkg updated mariadb104-server-10.4.13_1 to mariadb104-server-10.4.13_2

After the update I restarted the mysql-server and it refused to start. No records in .err file.

After reading this thread, I manually created /var/run/mysql dir with mysql:mysql owner/group and managed to start the server.
Comment 5 Nikolay Nikonov 2020-06-25 03:03:15 UTC
Sorry for another comment, but now the server starts successfully, but it can't see DBs. All permissions are fine. Everything in .err is fine. Weird.
Comment 6 Henrik Hudson 2020-06-25 04:38:13 UTC
Did you migrate your my.cnf from /usr/local/etc/ into the client and server.cnf files in /usr/local/etc/mysql/conf.d/  ???

The process owner (mysql) and permissions to the DB haven't changed.
Comment 7 Nikolay Nikonov 2020-06-25 05:44:23 UTC
Nope. I have not migrated the my.cnf from /usr/local/etc  (sleep deprivation takes tall).

I rolled back a system snapshot for now to restore services, but I'll experiment later for sure.
Comment 8 Vladimir Druzenko freebsd_committer freebsd_triage 2020-06-25 10:56:23 UTC
(In reply to Henrik Hudson from comment #2)
Check this: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246694#c22
Comment 9 Henrik Hudson 2020-06-25 14:55:28 UTC
(In reply to Nikolay Nikonov from comment #7)

Yeah, you have to do that or add the mysql_opts option to your /etc/rc.conf otherwise it will ignore your configuration as it's not looking at /usr/local/etc/my.cnf anymore.

This was posted in /usr/ports/UPDATING  , but yeah..caught me out too.
Comment 10 commit-hook freebsd_committer freebsd_triage 2020-06-27 19:42:56 UTC
A commit references this bug:

Author: brnrd
Date: Sat Jun 27 19:42:19 UTC 2020
New revision: 540633
URL: https://svnweb.freebsd.org/changeset/ports/540633

Log:
  databases/mariadb104-server: Make directories configurable

   * Create socket dir [1]
   * Take note of the changed directories

  PR:		247413 [1]
  Reported by:	Henrik Hudson <ports thelanman net>

Changes:
  head/databases/mariadb104-client/Makefile
  head/databases/mariadb104-client/pkg-plist
  head/databases/mariadb104-server/Makefile
  head/databases/mariadb104-server/files/client.cnf.sample.in
  head/databases/mariadb104-server/files/my.cnf.sample.in
  head/databases/mariadb104-server/files/pkg-message.in
  head/databases/mariadb104-server/files/server.cnf.sample.in
  head/databases/mariadb104-server/pkg-plist
Comment 11 Vladimir Druzenko freebsd_committer freebsd_triage 2020-06-27 21:50:35 UTC
(In reply to commit-hook from comment #10)
After server start with default server.cnf:
# ls -l /var/log/mysql/
-rw-rw----  1 mysql  mysql    1680 28 июня  00:46 ${hostname}.err

This line:
log_error                       = /var/log/mysql/${hostname}.err
Comment 12 Vladimir Druzenko freebsd_committer freebsd_triage 2020-07-02 15:39:54 UTC
(In reply to VVD from comment #11)
[02.07.2020][18:26:47] <VVD> how to set path for log file?
[02.07.2020][18:27:02] <VVD> log_error = /var/log/mysql/${hostname}.err 
[02.07.2020][18:27:17] <VVD> create file /var/log/mysql/${hostname}.err:
[02.07.2020][18:27:22] <VVD> # ls -l /var/log/mysql/
[02.07.2020][18:27:22] <VVD> -rw-rw----  1 mysql  mysql    1680 28 июня  00:46 ${hostname}.err
[02.07.2020][18:31:30] <grknight> VVD: use real paths, variables are not accepted
[02.07.2020][18:33:28] <VVD> grknight, thanks, but by default it set to real hostname - is it possible to set path for this log only (not all datadir)?
[02.07.2020][18:35:04] <grknight> if you set log_basename, then all logs are set to the same start name
[02.07.2020][18:35:20] <VVD> but path?
[02.07.2020][18:37:36] <grknight> it has to be a hard-coded path for log_error
[02.07.2020][18:37:53] <grknight> or else use syslog
[02.07.2020][18:38:59] <VVD> thanks

So this is impossible.
Comment 13 Vladimir Druzenko freebsd_committer freebsd_triage 2020-07-02 15:42:29 UTC
(In reply to VVD from comment #12)

This is from IRC #maria@freenode.

[02.07.2020][18:40:33] <grknight> many people and distros just use log_error=/var/log/mysql/mysqld.err    but it can be whatever you like of course
Comment 14 SBB 2020-07-03 19:43:15 UTC
(In reply to VVD from comment #11)
I'm seeing the same thing with the default server.cnf.

Can that be fixed so I don't have to go edit that config file for every server?
Comment 15 Vladimir Druzenko freebsd_committer freebsd_triage 2020-07-04 21:09:46 UTC
(In reply to SBB from comment #14)
Patch with fix is here: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247747#c7
Comment 16 Henrik Hudson 2020-07-06 21:41:42 UTC
Seems between the _2 and the _3 patch branch the 'mysql' user is no longer created in a clean install. All my ansible scripts failed since they expect that user to be there.

I have some server that I installed with the _2 build and they worked. The _3 build failed to add that user.
Comment 17 Henrik Hudson 2020-07-06 21:53:14 UTC
Also, /var/run/mysql is created, but since the user isn't created it never sets the correct owner, so MariaDB still doesn't start. I assume fixing adding the user will also fix this.
Comment 18 Vladimir Druzenko freebsd_committer freebsd_triage 2020-07-07 04:51:08 UTC
(In reply to Henrik Hudson from comment #17)
Try patch from this issue:
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247747#c8
Comment 19 Georg Altmann 2020-08-14 12:17:44 UTC
Just installed mariadb104-server-10.4.13_4 from package on 12.1-RELEASE-p8.
It is still broken, /usr/local/etc/rc.d/mysql-server contains a raw
%%MARIADB_GROUP%%
on line 123:

    if [ ! -d "${mysql_rundir}" ]; then
        install -d -o ${mysql_user} -g %%MARIADB_GROUP%% "${mysql_rundir}"

    fi

and thus /var/run/mysql is not created.
Comment 20 Georg Altmann 2020-08-14 16:35:57 UTC
Created attachment 217213 [details]
add MARIADB_GROUP to SUB_LIST and create rundir before starting the server
Comment 21 Vladimir Druzenko freebsd_committer freebsd_triage 2020-08-14 17:22:06 UTC
(In reply to Georg Altmann from comment #20)

IMHO, this way is better:
mysql_group="`/usr/bin/id -gn ${mysql_user}`"
Comment 22 Georg Altmann 2020-08-15 08:25:34 UTC
Created attachment 217225 [details]
get effective group via id

I have not seen it being done like this in any other port, but makes sense IMHO. See new patch. OK?
Comment 23 SBB 2020-10-07 02:00:10 UTC
Just wanted to mention that the latest version still doesn't set proper permissions on /var/run/mysql/ so mariadb will not start up after a clean install.
Comment 24 Georg Altmann 2020-10-18 08:06:06 UTC
Created attachment 218855 [details]
set permissions on /var/run/mysql/ correctly

Mixed up spaces and tabs in previous patch, did not apply.
Comment 25 Bernard Spil freebsd_committer freebsd_triage 2020-10-18 18:56:31 UTC
Comment on attachment 218855 [details]
set permissions on /var/run/mysql/ correctly

Makes sense
Comment 26 commit-hook freebsd_committer freebsd_triage 2020-10-24 11:50:42 UTC
A commit references this bug:

Author: brnrd
Date: Sat Oct 24 11:50:33 UTC 2020
New revision: 553187
URL: https://svnweb.freebsd.org/changeset/ports/553187

Log:
  databases/mariadb104-server: Security update to 10.4.15

   * Fix server start on clean install [1]

  PR:		247413 [1]
  MFH:		2020Q4
  Security:	a2565962-1156-11eb-9c9c-d4c9ef517024

Changes:
  head/databases/mariadb104-server/Makefile
  head/databases/mariadb104-server/distinfo
  head/databases/mariadb104-server/files/mysql-server.in
Comment 27 commit-hook freebsd_committer freebsd_triage 2020-10-24 12:01:46 UTC
A commit references this bug:

Author: brnrd
Date: Sat Oct 24 12:00:48 UTC 2020
New revision: 553188
URL: https://svnweb.freebsd.org/changeset/ports/553188

Log:
  MFH: r553187

  databases/mariadb104-server: Security update to 10.4.15

   * Fix server start on clean install [1]

  PR:		247413 [1]
  Security:	a2565962-1156-11eb-9c9c-d4c9ef517024

  Approved by:	ports-secteam (fluffy)

Changes:
_U  branches/2020Q4/
  branches/2020Q4/databases/mariadb104-server/Makefile
  branches/2020Q4/databases/mariadb104-server/distinfo
  branches/2020Q4/databases/mariadb104-server/files/mysql-server.in