14.0-RELEASE-p6. x86_64. I updated to GnuTLS 3.8.4 and Samba slowed down to a crawl. The daemon created a strangely named log file (.log) containing the following error, which I'd seen during my previous attempts with GnuTLS 3.8.3. It never happens with GnuTLS 3.7.x and lower. [2024/03/31 20:39:15.341561, 0] ../../lib/util/fault.c:178(smb_panic_log) =============================================================== [2024/03/31 20:39:15.341642, 0] ../../lib/util/fault.c:185(smb_panic_log) INTERNAL ERROR: ../../lib/util/genrand.c:53:generate_random_buffer: GnuTLS could not generate a random buffer: GNUTLS_E_LIB_IN_ERROR_STATE [-402] in rpcd_lsad () () pid 56216 (4.20.0) [2024/03/31 20:39:15.341738, 0] ../../lib/util/fault.c:190(smb_panic_log) If you are running a recent Samba version, and if you think this problem is not yet fixed in the latest versions, please consider reporting this bug, see https://wiki.samba.org/index.php/Bug_Reporting [2024/03/31 20:39:15.341794, 0] ../../lib/util/fault.c:191(smb_panic_log) =============================================================== [2024/03/31 20:39:15.341845, 0] ../../lib/util/fault.c:193(smb_panic_log) PANIC (pid 56216): ../../lib/util/genrand.c:53:generate_random_buffer: GnuTLS could not generate a random buffer: GNUTLS_E_LIB_IN_ERROR_STATE [-402] in 4.20.0 [2024/03/31 20:39:15.347533, 0] ../../lib/util/fault.c:304(log_stack_trace) BACKTRACE: 12 stack frames: #0 0x3c105278ed7 <log_stack_trace+0x37> at /usr/local/lib/samba4/private/libgenrand-private-samba.so #1 0x3c105278fae <smb_panic+0xe> at /usr/local/lib/samba4/private/libgenrand-private-samba.so #2 0x3c105279670 <generate_secret_buffer> at /usr/local/lib/samba4/private/libgenrand-private-samba.so #3 0x3c10527961f <generate_random_buffer+0x2f> at /usr/local/lib/samba4/private/libgenrand-private-samba.so #4 0x3c114b8647d <messaging_dgm_init+0x5ed> at /usr/local/lib/samba4/private/libmessages-dgm-private-samba.so #5 0x3c114b8602d <messaging_dgm_init+0x19d> at /usr/local/lib/samba4/private/libmessages-dgm-private-samba.so #6 0x3c114b88aa0 <messaging_dgm_ref+0xe0> at /usr/local/lib/samba4/private/libmessages-dgm-private-samba.so #7 0x3c0ff2c1460 <messaging_init+0x180> at /usr/local/lib/samba4/libsmbconf.so.0 #8 0x3c0ff2cbb3b <global_messaging_context+0x3b> at /usr/local/lib/samba4/libsmbconf.so.0 #9 0x3c0fae25205 <rpc_worker_main+0x365> at /usr/local/lib/samba4/private/libRPC-WORKER-private-samba.so #10 0x3b8d6505dcc <main+0x2c> at /usr/local/libexec/samba/rpcd_lsad #11 0x3c10cb6d66d <__libc_start1+0x12d> at /lib/libc.so.7 [2024/03/31 20:39:15.348165, 0] ../../source3/lib/dumpcore.c:310(dump_core) unable to change to /var/coredumps/%U refusing to dump core
I have 5 samba server installations - don't see same error. Show more details: samba version, samba build options, parts of the smb4.conf, etc.
This article seems to touch on the subject somewhat, https://www.suse.com/support/kb/doc/?id=000020879 . Can you try adding that to your config and see if it helps? It would also be useful to know what version of Samba you're running as we have several in tree.
Spoolss and all things printing already disabled since time immemorial. Versions? All those in ports plus my own quite successful attempt at 4.20. It doesn't matter which Samba version, but it does matter which GnuTLS version.
GnuTLS 3.8.x is very new in the ports tree, less than 24 h at this point methinks. Did you restart your samba instances after you upgraded the library? I myself played around with 3.8.3 at least a month ago and got the same result.
Created attachment 249637 [details] revert upstream 5ba60f53 Please save this patch as /usr/ports/security/gnutls/files/patch-lib_random.c and rebuild gnutls to see if that solves the problem. This patch is not meant as a solution. It's just to see if https://gitlab.com/gnutls/gnutls/-/commit/5ba60f53a066837a3ba065f015c2dbf9cecf879b is causing it. Also, what is the output of "objdump -p /usr/local/lib/libgnutls.so | grep NEEDED"?
(In reply to Tijl Coosemans from comment #5) The error message no longer appears. # objdump -p /usr/local/lib/libgnutls.so | grep NEEDED NEEDED libidn2.so.0 NEEDED libunistring.so.5 NEEDED libtasn1.so.6 NEEDED libnettle.so.8 NEEDED libhogweed.so.6 NEEDED libgmp.so.10 NEEDED libc.so.7
(In reply to Raivo Hool from comment #6) Oh, with GnuTLS 3.7.10, there is an extra: NEEDED libz.so.6
The error is gone but it is still slow? Can you remove the patch again and try the latest update to the port (version 3.8.4_2)?
(In reply to Tijl Coosemans from comment #8) The error message is gone now even without reverting the commit. KTLS and zlib are apparently picked up. As for the performance, I'll have it tested in wild today or tomorrow as my own one user machine does not see particularly heavy traffic. By the way, I noticed there's a change regarding SRP, and played around with adding the option to enable/disable SRP. It was nice. +SRP_DESC= Secure Remote Password support +SRP_CONFIGURE_ENABLE= srp-authentication
(In reply to Tijl Coosemans from comment #8) Sorry, spoke too soon. The error did not disappear after all. It's nicely all there on another machine with more users to provide the necessary stress.
(In reply to Raivo Hool from comment #10) Also with the upstream commit reverted?
(In reply to Tijl Coosemans from comment #11) With the upstream commit reversed, 3.8.4 works.
Created attachment 249804 [details] samba419 pthreadpool patch Here's a patch for samba419. Please save it as /usr/ports/net/samba419/files/patch-lib_pthreadpool_pthreadpool.c and rebuild samba419. Then rebuild gnutls without the revert patch. This patch should make samba wait for all threads to finish when it stops a pthreadpool, so it's safe for gnutls to free resources used by those threads.
(In reply to Tijl Coosemans from comment #13) Thank you. I did all that, but nothing seems to have changed. Just in case it's of any use, here's the entire error on pastebin: https://pastebin.com/NUjwvDVs. It seems that I had missed the first bit initially.
Worth noting is that reporter is running an out of tree build of Samba (4.20.0)
(In reply to Daniel Engberg from comment #15) It was worth mentioning it and I did, quite early on. I have tried it with all the versions in the tree as well. It makes no difference, that bit of Samba has not been changed and the patch applies cleanly across versions. And they all act up in an identical way.
(In reply to Raivo Hool from comment #14) Can you attach the entire log? The error might be even earlier.
(In reply to Tijl Coosemans from comment #17) I amended the pastebin with the three preceding lines that I had omitted previously. There's nothing before them, and I omitted them because they don't really add anything.
Created attachment 249852 [details] gnutls.keydbg.patch This patch makes gnutls dump some information when pthread_key_create and pthread_key_delete fail. Please remove /usr/ports/security/gnutls/files/patch-lib_random.c and then apply this patch with the following commands: cd /usr/ports git apply /path/to/patch Then rebuild gnutls. The extra information should appear in the samba log.
(In reply to Tijl Coosemans from comment #19) I took my sweet time because I wanted to be very sure, so I turned off ccache and everything and tried various combinations. Went back to Samba 4.19.5 available in ports. Here's the .log with the gnutls.keydbg.patch applied, and pthreadpool patch to Samba *not* applied. https://pastebin.com/uVesLqjn
Created attachment 249909 [details] gnutls pthread patch I believe this patch will fix the problem. You can save it as security/gnutls/files/patch-lib_global.c and delete patch-lib_random.c.
(In reply to Tijl Coosemans from comment #21) Looks good, with 4.19.5, 4.19.6, and 4.20.0 alike.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=9ffc65e659a32a4eb293afd5c4d03be553a61570 commit 9ffc65e659a32a4eb293afd5c4d03be553a61570 Author: Tijl Coosemans <tijl@FreeBSD.org> AuthorDate: 2024-04-12 19:03:35 +0000 Commit: Tijl Coosemans <tijl@FreeBSD.org> CommitDate: 2024-04-12 20:15:31 +0000 security/gnutls: initialise libpthread To ensure thread-safety libgnutls calls libpthread functions but to avoid the overhead for single-threaded programs it does not link with libpthread. It only calls libpthread if the executable or another library links to it. Since 3.8.0 libgnutls calls pthread_key_create from its init function but because it does not link with libpthread libpthread might not have been initialised yet. Patch the libgnutls init function so it initialises libpthread. PR: 278076 security/gnutls/Makefile | 1 + security/gnutls/files/patch-lib_global.c (new) | 43 ++++++++++++++++++++++++++ 2 files changed, 44 insertions(+)
A commit in branch 2024Q2 references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=5842d08c97cdecf1e3f8e2c536adb0aac3191574 commit 5842d08c97cdecf1e3f8e2c536adb0aac3191574 Author: Tijl Coosemans <tijl@FreeBSD.org> AuthorDate: 2024-04-12 19:03:35 +0000 Commit: Tijl Coosemans <tijl@FreeBSD.org> CommitDate: 2024-04-12 20:20:49 +0000 security/gnutls: initialise libpthread To ensure thread-safety libgnutls calls libpthread functions but to avoid the overhead for single-threaded programs it does not link with libpthread. It only calls libpthread if the executable or another library links to it. Since 3.8.0 libgnutls calls pthread_key_create from its init function but because it does not link with libpthread libpthread might not have been initialised yet. Patch the libgnutls init function so it initialises libpthread. PR: 278076 (cherry picked from commit 9ffc65e659a32a4eb293afd5c4d03be553a61570) security/gnutls/Makefile | 1 + security/gnutls/files/patch-lib_global.c (new) | 43 ++++++++++++++++++++++++++ 2 files changed, 44 insertions(+)