Upgraded from 3.8.4 to 3.9.1. using portmaster Built cleanly but starting service crashes and dumps core immediately on startup. % uname -a FreeBSD tuatara 14.1-RELEASE-p2 FreeBSD 14.1-RELEASE-p2 releng/14.1-1eba659e2 GENERIC amd6 nothing logged in rspamd.log. No debugging info found in core file. nothing logged in messages or dmesg. I even tried to start with a fresh /var/db/rspamd dir, same issue. I ended up reverting the VM from snapshot to get back to 3.8.4
I have the same problem after updating from 3.8.4 to 3.9.1 using portupgrade. # uname -a FreeBSD tenchi 13.3-RELEASE-p4 FreeBSD 13.3-RELEASE-p4 releng/13.3-n257438-deb948cd8dc2 TENCHI amd64 Thanks!
I'm sorry but your report contains nothing that could help to understand/fix the issue.
Please follow the instructions listed here: https://rspamd.com/doc/faq.html#how-to-figure-out-why-rspamd-process-crashed
Cant upload the core file, too large to attach. I have it if anyone wants it. below is backtrace from FAQ I did notice in this re-attempt it was updating python 3.9 modules with 3.11 and only python 3.9 is installed. I tried it again but installing python3.11 first, that didnt help. root@tuatara:/ # gdb `which rspamd` -c /rspamd-3.9.1.core GNU gdb (GDB) 14.1 [GDB v14.1 for FreeBSD] Copyright (C) 2023 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-portbld-freebsd14.1". Type "show configuration" for configuration details. For bug reporting instructions, please see: <https://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/local/bin/rspamd... (No debugging symbols found in /usr/local/bin/rspamd) [New LWP 100348] Core was generated by `/usr/local/bin/rspamd -c /usr/local/etc/rspamd/rspamd.conf -u rspamd -g rspamd'. Program terminated with signal SIGSEGV, Segmentation fault. Address not mapped to object. #0 0x0000000821fd766b in ?? () from /usr/local/lib/rspamd/librspamd-server.so (gdb) bt full #0 0x0000000821fd766b in ?? () from /usr/local/lib/rspamd/librspamd-server.so No symbol table info available. #1 0x0000000821fd85e1 in ?? () from /usr/local/lib/rspamd/librspamd-server.so No symbol table info available. #2 0x0000000821fe8c9f in ?? () from /usr/local/lib/rspamd/librspamd-server.so No symbol table info available. #3 0x0000000822002b2f in rspamd_config_parse_ucl () from /usr/local/lib/rspamd/librspamd-server.so No symbol table info available. #4 0x0000000822003689 in rspamd_config_read () from /usr/local/lib/rspamd/librspamd-server.so No symbol table info available. #5 0x0000000000222dfd in ?? () No symbol table info available. #6 0x00000000002222f5 in ?? () No symbol table info available. #7 0x0000000831f55a6a in __libc_start1 (argc=7, argv=0x820ac1090, env=0x820ac10d0, cleanup=<optimized out>, mainX=0x221d40) at /usr/src/lib/libc/csu/libc_start1.c:157 No locals. #8 0x0000000000214840 in ?? () No symbol table info available. (gdb)
Not sure if it's the cause it but rspamd does seem to bundle a lot of libraries that are already in the tree which may cause miscompilations and/or missing platforms specific patches.
(In reply to Daniel Engberg from comment #5) That is what I am thinking. I built a test VM and I slowly added my config to see what is causing the seg fault. So far I cant find it.
this is what I have tried. I have a test box that at least 3.9.1 will not core dump I have re-attempted upgrade making sure the active server: has same revisions of all packages (and has all the packages) on the test server including upgrading python and perl and reinstalling or upgrading modules I have removed all cache and map files and looked for any tmp file on the system related to rspamd I tried disabling BAYES I have commented each part of the config files on the active server, trying to find what in the config is calling to cause the core dump. I also captured a lldb backtrace to see if that adds more info (see below) I configured logging to debug (see below for last part of log file). It looks like it is just crashing after proxies are bound. It is crashing with no mail in the queue. I also turned off postfix and it still crashes at the point. I checked all mail queues and they are empty. Anyone have any idea what is causing the crash or what else i can check? I rolled back the system from snapshot at the end. root@tuatara:/ # lldb `which rspamd` -c ./rspamd-3.9.1.core (lldb) target create "/usr/local/bin/rspamd" --core "./rspamd-3.9.1.core" Core file '/rspamd-3.9.1.core' (x86_64) was loaded. (lldb) bt all * thread #1, name = 'rspamd-3.9.1', stop reason = signal SIGSEGV * frame #0: 0x0000000823ad766b librspamd-server.so`___lldb_unnamed_symbol6062 + 363 frame #1: 0x0000000823ad85e1 librspamd-server.so`___lldb_unnamed_symbol6069 + 161 frame #2: 0x0000000823ae8c9f librspamd-server.so`___lldb_unnamed_symbol6141 + 95 frame #3: 0x0000000823b02b2f librspamd-server.so`rspamd_config_parse_ucl + 287 frame #4: 0x0000000823b03689 librspamd-server.so`rspamd_config_read + 153 frame #5: 0x0000000000222dfd rspamd`___lldb_unnamed_symbol1126 + 93 frame #6: 0x00000000002222f5 rspamd`___lldb_unnamed_symbol1121 + 1461 frame #7: 0x0000000832625a6a libc.so.7`__libc_start1(argc=7, argv=0x0000000820c0ac88, env=0x0000000820c0acc8, cleanup=<unavailable>, mainX=(rspamd`___lldb_unnamed_symbol1121)) at pkg:157:7 frame #8: 0x0000000000214840 rspamd`___lldb_unnamed_symbol1011 + 48 (lldb) 2024-07-26 15:58:44 #94279(controller) <tr95me>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file neural_maybe_lock.lua, sha: 2a40f4606d23e3d080b58a7a86d06dc0f6705056 2024-07-26 15:58:44 #94279(controller) <tr95me>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_cache_check.lua, sha: dc0667e90f70e3df72fc6131e3b332160e82f638 2024-07-26 15:58:44 #94279(controller) <tr95me>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_cache_check.lua, sha: dc0667e90f70e3df72fc6131e3b332160e82f638 2024-07-26 15:58:44 #94279(controller) <tr95me>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_cache_learn.lua, sha: 67e32328bb29b39efed307d5d179a6ccf099da54 2024-07-26 15:58:44 #94279(controller) <tr95me>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_cache_learn.lua, sha: 67e32328bb29b39efed307d5d179a6ccf099da54 2024-07-26 15:58:44 #94279(controller) <tr95me>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_classify.lua, sha: 0075688c9013897c35b1ef045c2b9f55d12d4586 2024-07-26 15:58:44 #94279(controller) <tr95me>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_classify.lua, sha: 0075688c9013897c35b1ef045c2b9f55d12d4586 2024-07-26 15:58:44 #94279(controller) <tr95me>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file neural_maybe_invalidate.lua, sha: dc4556c7a1cd47d361b56026db3e644eea127d94 2024-07-26 15:58:44 #94279(controller) <tr95me>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file neural_maybe_invalidate.lua, sha: dc4556c7a1cd47d361b56026db3e644eea127d94 2024-07-26 15:58:44 #94279(controller) <tr95me>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_stat.lua, sha: 67446449f5ec67766385ef6de3ceaf77137aef46 2024-07-26 15:58:44 #94278(rspamd_proxy) <tr95me>; lua; neural.lua:490: loaded ANN for default:default from rn_default_default_im9g6ykb_1; 351358 bytes compressed; version=1 2024-07-26 15:58:44 #94279(controller) <tr95me>; lua; neural.lua:490: loaded ANN for default:default from rn_default_default_im9g6ykb_1; 351358 bytes compressed; version=1 2024-07-26 15:58:45 #94279(controller) <5t8s4f>; map; http_map_finish: data is not modified for server sa-update.surbl.org, next check at Fri, 26 Jul 2024 21:03:45 GMT (timer based) 2024-07-26 15:58:45 #94279(controller) <9i1dgi>; map; http_map_finish: data is not modified for server www.openphish.com, next check at Fri, 26 Jul 2024 21:03:45 GMT (timer based) 2024-07-26 15:58:54 #94278(rspamd_proxy) <20f6c3>; proxy; proxy_accept_socket: accepted milter connection from 192.168.6.12 port 25200 2024-07-26 15:58:55 #94278(rspamd_proxy) <20f6c3>; milter; rspamd_milter_process_command: got connection from 80.227.147.94:60161 2024-07-26 15:58:55 #94278(rspamd_proxy) <20f6c3>; proxy; proxy_milter_finish_handler: finished milter connection 2024-07-26 15:58:55 #94279(controller) <hz1ks5>; map; http_map_finish: data is not modified for server maps.rspamd.com, next check at Sat, 27 Jul 2024 00:58:12 GMT (http cache based: Sat, 27 Jul 2024 00:58:12 GMT) 2024-07-26 15:58:55 #94278(rspamd_proxy) <87ced9>; proxy; proxy_accept_socket: accepted milter connection from 192.168.6.12 port 63778 2024-07-26 15:58:55 #94277(main) <3e8689>; main; rspamd_term_handler: catch termination signal, waiting for 2 children for 16.00 seconds 2024-07-26 15:58:55 #94278(rspamd_proxy) <3e8689>; main; rspamd_worker_term_handler: terminating after receiving signal Terminated 2024-07-26 15:58:55 #94279(controller) <3e8689>; main; rspamd_worker_term_handler: terminating after receiving signal Terminated 2024-07-26 15:58:55 #94279(controller) rspamd_controller_on_terminate: closing rrd file: /var/db/rspamd/rspamd.rrd 2024-07-26 15:58:55 #94277(main) <3e8689>; main; rspamd_check_termination_clause: controller process 94279 terminated normally 2024-07-26 15:58:55 #94277(main) <3e8689>; main; rspamd_cld_handler: do not respawn process controller after found terminated process with pid 94279 2024-07-26 15:58:55 #94277(main) <3e8689>; main; rspamd_check_termination_clause: rspamd_proxy process 94278 terminated normally 2024-07-26 15:58:55 #94277(main) <3e8689>; main; rspamd_cld_handler: do not respawn process rspamd_proxy after found terminated process with pid 94278 2024-07-26 15:58:56 #94277(main) <3e8689>; main; main: terminating...
sorry, full lldb backtrace root@tuatara:/ # lldb `which rspamd` -c ./rspamd-3.9.1.core (lldb) target create "/usr/local/bin/rspamd" --core "./rspamd-3.9.1.core" Core file '/rspamd-3.9.1.core' (x86_64) was loaded. (lldb) bt all * thread #1, name = 'rspamd-3.9.1', stop reason = signal SIGSEGV * frame #0: 0x0000000823ad766b librspamd-server.so`___lldb_unnamed_symbol6062 + 363 frame #1: 0x0000000823ad85e1 librspamd-server.so`___lldb_unnamed_symbol6069 + 161 frame #2: 0x0000000823ae8c9f librspamd-server.so`___lldb_unnamed_symbol6141 + 95 frame #3: 0x0000000823b02b2f librspamd-server.so`rspamd_config_parse_ucl + 287 frame #4: 0x0000000823b03689 librspamd-server.so`rspamd_config_read + 153 frame #5: 0x0000000000222dfd rspamd`___lldb_unnamed_symbol1126 + 93 frame #6: 0x00000000002222f5 rspamd`___lldb_unnamed_symbol1121 + 1461 frame #7: 0x0000000832625a6a libc.so.7`__libc_start1(argc=7, argv=0x0000000820c0ac88, env=0x0000000820c0acc8, cleanup=<unavailable>, mainX=(rspamd`___lldb_unnamed_symbol1121)) at libc_start1.c:157:7 frame #8: 0x0000000000214840 rspamd`___lldb_unnamed_symbol1011 + 48
Can you please build Rspamd from the ports using `WITH_DEBUG=yes DEBUG=yes STRIP=` arguments to all the `make` commands.
ok, so i made rspamd with WITH_DEBUG=yes DEBUG=yes STRIP= no crash/core dump i decided to try cleaning and making without options above, core dump immediately i make clean then make with those options, no crash I went ahead and reverted to snapshot for now. Below is the tail of the rspamd.log starting and then successfully scanning an email. 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_classify.lua, sha: 0075688c9013897c35b1ef045c2b9f55d12d4586 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_classify.lua, sha: 0075688c9013897c35b1ef045c2b9f55d12d4586 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_learn.lua, sha: 29ea7b39082121d7f0fa26b7562d80e28c8a656b 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_learn.lua, sha: 29ea7b39082121d7f0fa26b7562d80e28c8a656b 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_cache_check.lua, sha: dc0667e90f70e3df72fc6131e3b332160e82f638 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_cache_check.lua, sha: dc0667e90f70e3df72fc6131e3b332160e82f638 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_cache_learn.lua, sha: 1fa86da47c11fe758b3acdedc5d63c9f4a536587 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_cache_learn.lua, sha: 1fa86da47c11fe758b3acdedc5d63c9f4a536587 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_classify.lua, sha: 0075688c9013897c35b1ef045c2b9f55d12d4586 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_classify.lua, sha: 0075688c9013897c35b1ef045c2b9f55d12d4586 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 with id 16, sha: 7821ae9ca73268a540c8a091e880f77421535d39 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 with id 16, sha: 7821ae9ca73268a540c8a091e880f77421535d39 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_stat.lua, sha: 67446449f5ec67766385ef6de3ceaf77137aef46 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file bayes_stat.lua, sha: 67446449f5ec67766385ef6de3ceaf77137aef46 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file ratelimit_check.lua, sha: c0d8df3a08c27e2f6b077fea3ffef7788318d8a0 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file ratelimit_check.lua, sha: c0d8df3a08c27e2f6b077fea3ffef7788318d8a0 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file ratelimit_update.lua, sha: 166518ccdf8d0aecb1bfc8b9334018ac4359e78f 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file ratelimit_update.lua, sha: 166518ccdf8d0aecb1bfc8b9334018ac4359e78f 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file ratelimit_cleanup_pending.lua, sha: cd653186a50918be9389f99095e2e34625004596 2024-07-27 11:23:50 #35899(controller) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file neural_save_unlock.lua, sha: 3f6a1f9fd1ef52a5ac4ddcbf963c6974b4effd21 2024-07-27 11:23:50 #35899(controller) <aen3wo>; lua; lua_redis.lua:1254: uploaded redis script to 192.168.6.12:6379 from file neural_save_unlock.lua, sha: 3f6a1f9fd1ef52a5ac4ddcbf963c6974b4effd21 2024-07-27 11:23:50 #35898(rspamd_proxy) <aen3wo>; lua; neural.lua:490: loaded ANN for default:default from rn_default_default_im9g6ykb_1; 351298 bytes compressed; version=1 2024-07-27 11:23:50 #35899(controller) <aen3wo>; lua; neural.lua:490: loaded ANN for default:default from rn_default_default_im9g6ykb_1; 351298 bytes compressed; version=1 2024-07-27 11:23:50 #35899(controller) <5t8s4f>; map; http_map_finish: data is not modified for server sa-update.surbl.org, next check at Sat, 27 Jul 2024 16:28:50 GMT (timer based) 2024-07-27 11:23:51 #35899(controller) <9i1dgi>; map; http_map_finish: data is not modified for server www.openphish.com, next check at Sat, 27 Jul 2024 16:28:51 GMT (timer based) 2024-07-27 11:25:03 #35898(rspamd_proxy) <804f10>; proxy; proxy_accept_socket: accepted milter connection from 192.168.6.12 port 39967 2024-07-27 11:25:03 #35898(rspamd_proxy) <804f10>; milter; rspamd_milter_process_command: got connection from 192.168.6.11:29982 2024-07-27 11:25:03 #35898(rspamd_proxy) <804f10>; proxy; rspamd_message_parse: loaded message; id: <c9026b9c74f1f93bd896b474ffd9c0db@excelsus.com>; queue-id: <39F2D1692F5>; size: 356; checksum: <14f9c000aff11d614170a64180ee088c> 2024-07-27 11:25:03 #35898(rspamd_proxy) <804f10>; proxy; dkim_symbol_callback: skip DKIM checks for local networks and authorized users 2024-07-27 11:25:03 #35898(rspamd_proxy) <804f10>; lua; spf.lua:189: skip SPF checks for local networks and authorized users 2024-07-27 11:25:03 #35898(rspamd_proxy) <804f10>; lua; dmarc.lua:353: skip DMARC checks as either SPF or DKIM were not checked 2024-07-27 11:25:03 #35898(rspamd_proxy) <804f10>; lua; once_received.lua:102: Skipping once_received for authenticated user or local network 2024-07-27 11:25:03 #35898(rspamd_proxy) <804f10>; proxy; finalize_item: slow asynchronous rule: URIBL_MULTI(502): 404.73 ms; no idle timer is needed 2024-07-27 11:25:03 #35898(rspamd_proxy) <804f10>; proxy; rspamd_stat_check_autolearn: <c9026b9c74f1f93bd896b474ffd9c0db@excelsus.com>: autolearn ham for classifier 'bayes' as message's score is negative: -5.10 2024-07-27 11:25:03 #35898(rspamd_proxy) <804f10>; proxy; rspamd_stat_classifiers_learn: <c9026b9c74f1f93bd896b474ffd9c0db@excelsus.com> contains less tokens than required for bayes classifier: 6 < 11 2024-07-27 11:25:03 #35898(rspamd_proxy) <804f10>; proxy; rspamd_task_process: skip learning: <c9026b9c74f1f93bd896b474ffd9c0db@excelsus.com> contains less tokens than required for bayes classifier: 6 < 11 2024-07-27 11:25:03 #35898(rspamd_proxy) <804f10>; proxy; rspamd_task_write_log: id: <c9026b9c74f1f93bd896b474ffd9c0db@excelsus.com>, qid: <39F2D1692F5>, ip: 192.168.6.11, user: weldon@excelsusphoto.com, from: <weldon@excelsus.com >, (default: F (no action): [-5.10/15.00] [LOCAL_WL_IP(-5.00){192.168.6.11;},MIME_GOOD(-0.10){text/plain;},ARC_NA(0.00){},FROM_EQ_ENVFROM(0.00){},FROM_HAS_DN(0.00){},MID_RHS_MATCH_FROM(0.00){},MIME_TRACE(0.00){0:+;},MISSING_XM_UA( 0.00){},NEURAL_HAM(-0.00){-0.912;},RCPT_COUNT_ONE(0.00){1;},RCVD_COUNT_ZERO(0.00){0;},SINGLE_SHORT_PART(0.00){},TO_DN_ALL(0.00){},TO_MATCH_ENVRCPT_ALL(0.00){}]), len: 356, time: 418.820ms, dns req: 6, digest: <14f9c000aff11d614170 a64180ee088c>, rcpts: <weldon@supersilly.com>, mime_rcpts: <weldon@supersilly.com> 2024-07-27 11:25:03 #35898(rspamd_proxy) <804f10>; proxy; rspamd_protocol_http_reply: regexp statistics: 48 pcre regexps scanned, 1 regexps matched, 180 regexps total, 11 regexps cached, 1.15KiB scanned using pcre, 1.15KiB scanned total 2024-07-27 11:25:03 #35898(rspamd_proxy) <8149b7>; proxy; proxy_milter_finish_handler: finished milter connection
It looks like a heisenbug to me :( Let's try another thing - please try to build the port without `WITH_DEBUG=yes DEBUG=yes` flags but keep `STRIP=` in place. So we will likely have all optimisation flags but will not loose debug symbols at least. I'm not sure it will help but there is some chance that it will.
ah, with just STRIP= on make, it crashed. The backtrace appears to have some more info (lldb) target create "/usr/local/bin/rspamd" --core "./rspamd-3.9.1.core" Core file '/rspamd-3.9.1.core' (x86_64) was loaded. (lldb) bt all * thread #1, name = 'rspamd-3.9.1', stop reason = signal SIGSEGV * frame #0: 0x000000082252466b librspamd-server.so`fmt::v10::appender fmt::v10::detail::copy_str_noinline<char, char const*, fmt::v10::appender>(char const*, char const*, fmt::v10::appender) + 363 frame #1: 0x00000008225255e1 librspamd-server.so`char const* fmt::v10::detail::parse_replacement_field<char, void fmt::v10::detail::vformat_to<char>(fmt::v10::detail::buffer<char>&, fmt::v10::basic_string_view<char>, fmt::v10::detail::vformat_args<char>::type, fmt::v10::detail::locale_ref)::format_handler&>(char const*, char const*, void fmt::v10::detail::vformat_to<char>(fmt::v10::detail::buffer<char>&, fmt::v10::basic_string_view<char>, fmt::v10::detail::vformat_args<char>::type, fmt::v10::detail::locale_ref)::format_handler&) + 161 frame #2: 0x0000000822535c9f librspamd-server.so`void fmt::v10::detail::parse_format_string<false, char, void fmt::v10::detail::vformat_to<char>(fmt::v10::detail::buffer<char>&, fmt::v10::basic_string_view<char>, fmt::v10::detail::vformat_args<char>::type, fmt::v10::detail::locale_ref)::format_handler>(fmt::v10::basic_string_view<char>, void fmt::v10::detail::vformat_to<char>(fmt::v10::detail::buffer<char>&, fmt::v10::basic_string_view<char>, fmt::v10::detail::vformat_args<char>::type, fmt::v10::detail::locale_ref)::format_handler&&) + 95 frame #3: 0x000000082254fb2f librspamd-server.so`rspamd_config_parse_ucl + 287 frame #4: 0x0000000822550689 librspamd-server.so`rspamd_config_read + 153 frame #5: 0x0000000000222dfd rspamd`load_rspamd_config + 93 frame #6: 0x00000000002222f5 rspamd`main + 1461 frame #7: 0x000000083356ca6a libc.so.7`__libc_start1(argc=7, argv=0x00000008206347a0, env=0x00000008206347e0, cleanup=<unavailable>, mainX=(rspamd`main)) at libc_start1.c:157:7 frame #8: 0x0000000000214840 rspamd`_start at crt1_s.S:83 (lldb)
Can you please try this patch: https://github.com/rspamd/rspamd/commit/5a2dedd00eb68a5d4c7acda93fd3cc4f3bb2d358
still crashes, the backtrace almost looks the same root@tuatara:/ # lldb `which rspamd` -c ./rspamd-3.9.1.core (lldb) target create "/usr/local/bin/rspamd" --core "./rspamd-3.9.1.core" Core file '/rspamd-3.9.1.core' (x86_64) was loaded. (lldb) bt all * thread #1, name = 'rspamd-3.9.1', stop reason = signal SIGSEGV * frame #0: 0x0000000822a2e66b librspamd-server.so`fmt::v10::appender fmt::v10::detail::copy_str_noinline<char, char const*, fmt::v10::appender>(char const*, char const*, fmt::v10::appender) + 363 frame #1: 0x0000000822a2f5e1 librspamd-server.so`char const* fmt::v10::detail::parse_replacement_field<char, void fmt::v10::detail::vformat_to<char>(fmt::v10::detail::buffer<char>&, fmt::v10::basic_string_view<char>, fmt::v10::detail::vformat_args<char>::type, fmt::v10::detail::locale_ref)::format_handler&>(char const*, char const*, void fmt::v10::detail::vformat_to<char>(fmt::v10::detail::buffer<char>&, fmt::v10::basic_string_view<char>, fmt::v10::detail::vformat_args<char>::type, fmt::v10::detail::locale_ref)::format_handler&) + 161 frame #2: 0x0000000822a3fc9f librspamd-server.so`void fmt::v10::detail::parse_format_string<false, char, void fmt::v10::detail::vformat_to<char>(fmt::v10::detail::buffer<char>&, fmt::v10::basic_string_view<char>, fmt::v10::detail::vformat_args<char>::type, fmt::v10::detail::locale_ref)::format_handler>(fmt::v10::basic_string_view<char>, void fmt::v10::detail::vformat_to<char>(fmt::v10::detail::buffer<char>&, fmt::v10::basic_string_view<char>, fmt::v10::detail::vformat_args<char>::type, fmt::v10::detail::locale_ref)::format_handler&&) + 95 frame #3: 0x0000000822a59b39 librspamd-server.so`rspamd_config_parse_ucl + 297 frame #4: 0x0000000822a5a6d9 librspamd-server.so`rspamd_config_read + 153 frame #5: 0x0000000000222dfd rspamd`load_rspamd_config + 93 frame #6: 0x00000000002222f5 rspamd`main + 1461 frame #7: 0x0000000834e2ba6a libc.so.7`__libc_start1(argc=7, argv=0x0000000821151ea0, env=0x0000000821151ee0, cleanup=<unavailable>, mainX=(rspamd`main)) at libc_start1.c:157:7 frame #8: 0x0000000000214840 rspamd`_start at crt1_s.S:83
Or we just run it by compiling it with DEBUG options since with that it does not crash?? :-(
Sorry to give bad news. This problem still exists for me with v3.10.2 I did remove cache files as asked for at the end of the build. "find /var/db/rspamd/ -type f -name '*.unser' -delete" still dumps core at start
13.4-RELEASE-p1 FreeBSD 13.4-RELEASE-p1 GENERIC amd64 In Rspamd 3.11.0 after upgrading via portmaster I am seeing # service rspamd start Starting rspamd. 2025-01-26 09:16:14 #73337(main) <5daabb>; main; main: rspamd 3.11.0 is loading configuration, build id: release Segmentation fault (core dumped) /usr/local/etc/rc.d/rspamd: WARNING: failed to start rspamd rspamd.log has the following 2025-01-26 08:35:03 #46568(main) <81690c>; main; rspamd_check_termination_clause: controller process 48066 terminated normally 2025-01-26 08:35:03 #46568(main) <81690c>; main; rspamd_cld_handler: do not respawn process controller after found terminated process with pid 48066 2025-01-26 08:35:19 #46568(main) <81690c>; main; rspamd_worker_wait: terminate worker rspamd_proxy(47393) with SIGKILL 2025-01-26 08:35:19 #46568(main) <81690c>; main; rspamd_check_termination_clause: rspamd_proxy process 47393 terminated abnormally with exit code 0 by signal: Killed but NOT created core file (throttled=no); core file limits: 9223372036854775807 current, 9223372036854775807 max 2025-01-26 08:35:19 #46568(main) <81690c>; main; rspamd_cld_handler: do not respawn process rspamd_proxy after found terminated process with pid 47393 2025-01-26 08:35:19 #46568(main) <81690c>; main; main: terminating...