Summary: | mysqld SIGILL with mysql80-server-8.0.35_1, mysql81-server-8.1.0 and mysql84-server-8.4.0 with FreeBSD 14.1 | ||||||
---|---|---|---|---|---|---|---|
Product: | Ports & Packages | Reporter: | John Lind <steinhelten> | ||||
Component: | Individual Port(s) | Assignee: | Mikael Urankar <mikael> | ||||
Status: | New --- | ||||||
Severity: | Affects Some People | CC: | chris, freebsd, grahamperrin, mikael | ||||
Priority: | --- | Flags: | linimon:
maintainer-feedback?
|
||||
Version: | Latest | ||||||
Hardware: | arm64 | ||||||
OS: | Any | ||||||
Attachments: |
|
Description
John Lind
2024-07-06 22:44:39 UTC
I created a completely new image from FreeBSD-14.1-RELEASE-arm64-aarch64-RPI.img.xz and tried mysql81-server. As before, I created a my.cnf file that was the same as the example except I reduced the buffer size: innodb_buffer_pool_size = 128M I got some errors pertaining to file ownership and permissions, which I worked through (chown mysql:mysql /var/db/mysql*). Then this happened: # service mysql-server onestart 2024-08-03T21:53:00.766292Z 0 [System] [MY-015017] [Server] MySQL Server Initialization - start. 2024-08-03T21:53:00.783242Z 0 [System] [MY-013169] [Server] /usr/local/libexec/mysqld (mysqld 8.1.0) initializing of server in progress as process 4349 mysqld: Error on delete of '/var/db/mysql/auto.cnf' (OS errno 2 - No such file or directory) 2024-08-03T21:53:00.888899Z 0 [Warning] [MY-010107] [Server] World-writable config file '/var/db/mysql/auto.cnf' has been removed. 2024-08-03T21:53:00.908502Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started. 2024-08-03T21:53:08Z UTC - mysqld got signal 4 ; Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware. Thread pointer: 0x55cb57bdc000 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... [0x23052b0] +0x0 [0x2305414] +0x0 [0x84ae6e48] +0x0 [0x84ae6594] +0x0 [0x842f51a4] +0x0 [0x30b3580] +0x0 [0x2fb4720] +0x0 [0x2fb49b4] +0x0 [0x2faf394] +0x0 [0x2fb0f54] +0x0 [0x2fbc904] +0x0 [0x30666fc] +0x0 [0x2dba484] +0x0 [0x2bbc0d4] +0x0 [0x2bc24e0] +0x0 [0x1fbe6b0] +0x0 [0x31142bc] +0x0 [0x84adc3a0] +0x0 stack_bottom = 937e0e90 thread_stack 0x100000 0x2d6db20 <_Z19my_print_stacktracePKhm+0x13c> at /usr/local/libexec/mysqld 0x23052b0 <_Z18print_fatal_signali+0x25c> at /usr/local/libexec/mysqld 0x2305414 <handle_fatal_signal+0x40> at /usr/local/libexec/mysqld 0x84ae6e48 <pthread_sigmask+0x534> at /lib/libthr.so.3 Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0): Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. /usr/local/etc/rc.d/mysql-server: WARNING: failed precmd routine for mysql # At that point, I uninstalled mysql81-server-8.1.0 and installed mysql84-server and tried again, with the same my.cnf file. This time, I still got SIGILL but with a very different looking backtrace. # service mysql-server onestart 2024-08-03T22:03:13.218920Z 0 [System] [MY-015017] [Server] MySQL Server Initialization - start. 2024-08-03T22:03:13.230469Z 0 [System] [MY-013169] [Server] /usr/local/libexec/mysqld (mysqld 8.4.0) initializing of server in progress as process 4503 mysqld: Error on delete of '/var/db/mysql/auto.cnf' (OS errno 2 - No such file or directory) 2024-08-03T22:03:13.321074Z 0 [Warning] [MY-010107] [Server] World-writable config file '/var/db/mysql/auto.cnf' has been removed. 2024-08-03T22:03:13.342227Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started. 2024-08-03T22:03:21Z UTC - mysqld got signal 4 ; Signal SIGILL (Illegal trap) at address 0x905879ee Most likely, you have hit a bug, but this error can also be caused by malfunctioning hardware. Thread pointer: 0x2c0b80581000 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 92b4de78 thread_stack 0x100000 #0 0x24ea743 <unknown> #1 0x24ea30f <unknown> #2 0x85b68e5b handle_signal at /usr/src/lib/libthr/thread/thr_sig.c:298 #3 0x85b68593 thr_sighandler at /usr/src/lib/libthr/thread/thr_sig.c:243 Trying to get some variables. Some pointers may be invalid and cause the dump to abort. Query (0): Connection ID (thread ID): 1 Status: NOT_KILLED The manual page at http://dev.mysql.com/doc/mysql/en/crashing.html contains information that should help you find out what is causing the crash. /usr/local/etc/rc.d/mysql-server: WARNING: failed precmd routine for mysql # I tried going to the web page that the error recommends, but I can't even find the error log. :( I'm trying to follow their suggestions, but this is failing so early in the process that nothing really applies. While we're at it, let me add that /var/db/mysql_secure and /var/db/mysql_tmpdir get created with the wrong ownship. After installing, you have to do # chown mysql:mysql /var/db/mysql_* or you won't get very far at all. I know I mentioned this, but it's easy to miss and probably relatively easy to fix in the package. Well, Oracle just told me to go fish. How can I help move this forward? Isn't ANYONE trying to use MySQL with FreeBSD 14.1? Upon encouragement from a post in the forums, I ran mysqld from gdb and got the following. Note that I edited out a bunch of the gdb boilerplate for brevity. I hacked the /use/local/etc/rc.c/mysql-server script to get the exact parameters it is being called with. I set up options in .gdb according to the reference manual entry for using gdb with mysqld (https://dev.mysql.com/doc/refman/8.4/en/using-gdb-on-mysqld.html). That gave me the following. # gdb /usr/local/libexec/mysqld GNU gdb (GDB) 14.1 [GDB v14.1 for FreeBSD] ... This GDB was configured as "aarch64-portbld-freebsd14.0". Type "show configuration" for configuration details. ... Reading symbols from /usr/local/libexec/mysqld... (No debugging symbols found in /usr/local/libexec/mysqld) (gdb) run --defaults-extra-file=/usr/local/etc/mysql/my.cnf --initialize-insecure --basedir=/usr/local --datadir=/var/db/mysql --user=mysql Starting program: /usr/local/libexec/mysqld --defaults-extra-file=/usr/local/etc/mysql/my.cnf --initialize-insecure --basedir=/usr/local --datadir=/var/db/mysql --user=mysql [New LWP 100236 of process 8836] 2024-08-12T21:56:45.403943Z 0 [System] [MY-015017] [Server] MySQL Server Initialization - start. 2024-08-12T21:56:45.417480Z 0 [System] [MY-013169] [Server] /usr/local/libexec/mysqld (mysqld 8.4.0) initializing of server in progress as process 8836 mysqld: Error on delete of '/var/db/mysql/auto.cnf' (OS errno 2 - No such file or directory) 2024-08-12T21:56:45.511851Z 0 [Warning] [MY-010107] [Server] World-writable config file '/var/db/mysql/auto.cnf' has been removed. [New LWP 100237 of process 8836] 2024-08-12T21:56:45.532560Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started. [New LWP 100238 of process 8836] [LWP 100238 of process 8836 exited] [New LWP 100239 of process 8836] [New LWP 100240 of process 8836] [New LWP 100241 of process 8836] [New LWP 100242 of process 8836] [New LWP 100243 of process 8836] [New LWP 100244 of process 8836] [New LWP 100245 of process 8836] [New LWP 100246 of process 8836] [New LWP 100247 of process 8836] [New LWP 100248 of process 8836] [New LWP 100249 of process 8836] [New LWP 100250 of process 8836] [New LWP 100251 of process 8836] [New LWP 100252 of process 8836] [New LWP 100253 of process 8836] [New LWP 100254 of process 8836] [New LWP 100255 of process 8836] [New LWP 100256 of process 8836] Thread 3 received signal SIGILL, Illegal instruction. Illegal trap. [Switching to LWP 100237 of process 8836] 0x0000000003291848 in hardware::crc32_using_pclmul(unsigned char const*, unsigned long) () (gdb) # Disassembly after the SIGILL gives, in part: (gdb) disas Dump of assembler code for function _ZN8hardware18crc32_using_pclmulEPKhm: ... --Type <RET> for more, q to quit, c to continue without paging-- 0x0000000003291820 <+716>: crc32cx w10, w10, x16 0x0000000003291824 <+720>: ldp x15, x18, [x0, #320] 0x0000000003291828 <+724>: crc32cx w13, w13, x1 0x000000000329182c <+728>: ldp x17, x16, [x0, #432] 0x0000000003291830 <+732>: fmov d0, x10 0x0000000003291834 <+736>: crc32cx w13, w13, x15 0x0000000003291838 <+740>: crc32cx w10, w13, x18 0x000000000329183c <+744>: ldp x13, x15, [x0, #448] 0x0000000003291840 <+748>: crc32cx w14, w14, x17 0x0000000003291844 <+752>: crc32cx w14, w14, x16 => 0x0000000003291848 <+756>: pmull v0.1q, v1.1d, v0.1d 0x000000000329184c <+760>: fmov d1, x10 0x0000000003291850 <+764>: crc32cx w10, w14, x13 0x0000000003291854 <+768>: ldp x13, x14, [x0, #464] 0x0000000003291858 <+772>: crc32cx w10, w10, x15 0x000000000329185c <+776>: pmull v1.1q, v2.1d, v1.1d 0x0000000003291860 <+780>: crc32cx w10, w10, x13 0x0000000003291864 <+784>: ldp x15, x13, [x0, #480] 0x0000000003291868 <+788>: crc32cx w10, w10, x14 0x000000000329186c <+792>: eor v0.16b, v1.16b, v0.16b 0x0000000003291870 <+796>: ldr x14, [x0, #496] 0x0000000003291874 <+800>: add x0, x0, #0x1f8 acheron (https://forums.freebsd.org/members/acheron.15063/) has identified the problem. The return value expression for can_use_poly_mul() should be "return capabilities & HWCAP_PMULL" instead of "return capabilities & HWCAP_CRC32"; This appears to be a simple copying error since the incorrect return expression is the same as the prior function definition, where it is correct. This code is at https://github.com/freebsd/freebsd-ports/blob/2024Q3/databases/mysql84-server/files/patch-storage_innobase_ut_crc32.cc#L20 . Also, the package needs to create /var/db/mysql_tempdir and /var/db/mysql_security with the ownership of "mysql:mysql intead" of "root:wheel". With these changes, mysqld starts and responds to queries. I assume this change will need to be made for all three versions of the server currently offered for FreeBSD 14.1: mysql80-server mysql81-server mysql84-server While I haven't figured out yet why I can't compile them, I do have a system where I can test all three with regard to this one particular bug, if that's helpful. If there's a full regression test suite, I'd certainly be happy to take a look at it. Created attachment 253040 [details]
v0
Does that address the ownership issues for /var/db/mysql_tempdir and /var/db/mysql_security as well? A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=90179a1242bc8aec6bd15491841049ee823ed320 commit 90179a1242bc8aec6bd15491841049ee823ed320 Author: Mikael Urankar <mikael@FreeBSD.org> AuthorDate: 2024-08-23 17:19:58 +0000 Commit: Mikael Urankar <mikael@FreeBSD.org> CommitDate: 2024-08-26 08:13:37 +0000 databases/mysql84-server: Fix SIGILL on aarch64 We want to check HWCAP_PMULL not HWCAP_CRC32. Approved by: portmgr (trivial runtime fix) PR: 280165 databases/mysql84-server/Makefile | 2 +- databases/mysql84-server/files/patch-storage_innobase_ut_crc32.cc | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=5a6ab005518a4351994494df094a0396bc1c0e2e commit 5a6ab005518a4351994494df094a0396bc1c0e2e Author: Mikael Urankar <mikael@FreeBSD.org> AuthorDate: 2024-08-23 17:22:00 +0000 Commit: Mikael Urankar <mikael@FreeBSD.org> CommitDate: 2024-08-26 08:13:37 +0000 databases/mysql81-server: Fix SIGILL on aarch64 We want to check HWCAP_PMULL not HWCAP_CRC32. Approved by: portmgr (trivial runtime fix) PR: 280165 databases/mysql81-server/Makefile | 2 +- databases/mysql81-server/files/patch-storage_innobase_ut_crc32.cc | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=16519bd204a0f2a9c46282591eaf29fd01b87721 commit 16519bd204a0f2a9c46282591eaf29fd01b87721 Author: Mikael Urankar <mikael@FreeBSD.org> AuthorDate: 2024-08-23 17:21:45 +0000 Commit: Mikael Urankar <mikael@FreeBSD.org> CommitDate: 2024-08-26 08:13:37 +0000 databases/mysql80-server: Fix SIGILL on aarch64 We want to check HWCAP_PMULL not HWCAP_CRC32. Approved by: portmgr (trivial runtime fix) PR: 280165 databases/mysql80-server/Makefile | 2 +- databases/mysql80-server/files/patch-storage_innobase_ut_crc32.cc | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) |