During porting of a C++ application to FreeBSD I have discovered problem with using thread_local with the system version of Clang 3.3. As below, it's use produces linking errors to the C++ ABI. troy@freebsd10:~ # cat tl.cpp #include <string> thread_local std::string test; int main() { } troy@freebsd10:~ # clang++ -std=c++11 tl.cpp -o tl /tmp/tl-7sN7Nb.o: In function `__cxx_global_var_init': tl.cpp:(.text+0xbb): undefined reference to `__cxa_thread_atexit' clang++: error: linker command failed with exit code 1 (use -v to see invocation) troy@freebsd10:~ # troy@freebsd10:~ # clang++ -v FreeBSD clang version 3.3 (tags/RELEASE_33/final 183502) 20130610 Target: x86_64-unknown-freebsd10.0 Thread model: posix troy@freebsd10:~ #
Our __cxa_atexit() is in lib/libc/stdlib/atexit.c, but apparently we don't have a corresponding __cxa_thread_atexit() yet. I'm not entirely sure where it should live, though: GNU puts it in libsupc++, so for use that would correspond to libcxxrt...
I did write a __cxa_thread_atexit a while ago, but I haven't yet committed it (I'll try to find it). I didn't commit it because the semantics of _Thread_local are very poorly defined in the presence of shared libraries. I could think of three wrong ways of implementing it (one of which was chosen by libsupc++), but no right way.
(In reply to David Chisnall from comment #2) Any status update on this? I've run into this issue too.
I'm still not completely sure what the correct solution for implementing this is. Clang will always lazily construct thread_local objects on first access (which means that you can't use them to track thread counts and so on, which limits their use somewhat), but the destructors will run when the thread exits. Unfortunately, the destructors might not be signal safe and so there is no *good* way of running them on library unload. A complete solution most likely involves locking the library in memory while it has outstanding thread-local objects.
(In reply to David Chisnall from comment #4) That's what glibc seems to do. When a thread calls dlclose its thread_local variables are destroyed, but if there are other threads with thread_local variables the DSO is kept in memory until those variables are destroyed (when the threads exit or call dlclose). https://sourceware.org/glibc/wiki/Destructor%20support%20for%20thread_local%20variables
I have also encountered this while trying to port something to FreeBSD. Is there no news?
(In reply to David Chisnall from comment #4) > A complete solution most likely involves locking the library in memory while it has outstanding thread-local objects. This sounds like a perfectly decent solution - is it something you are able to implement? This bug is going to bite more frequently as more C++11 is written. :(
Indeed, a fix would be great! Thank you.
Please, be sure to bump OSVERSION, when this is committed -- so that the various work-arounds implemented in the ports in the mean time can be made conditional... Thanks!
This issue is now two years old and I have to work around it more and more often. If a "good solution" is so difficult to make, can we at least have some workaround? Thank you very much!
Kib committed the a fix for this to head recently.
Assign to committer that resolved. @Kib, can you add a reference to the change please, and does this need or has it been MFC'd? Set mfc-stable* flags to + where/when merged, or to - with comment if not required
r303795 | kib | 2016-08-06 06:32:40 -0700 (Sat, 06 Aug 2016) | 37 lines Changed paths: M /head/lib/libc/include/libc_private.h M /head/lib/libc/stdlib/Makefile.inc M /head/lib/libc/stdlib/Symbol.map A /head/lib/libc/stdlib/cxa_thread_atexit.c M /head/lib/libc/stdlib/exit.c M /head/lib/libc/tests/stdlib/Makefile M /head/lib/libc/tests/stdlib/Makefile.depend A /head/lib/libc/tests/stdlib/cxa_thread_atexit_nothr_test.cc A /head/lib/libc/tests/stdlib/cxa_thread_atexit_test.cc M /head/lib/libthr/thread/thr_exit.c Add __cxa_thread_atexit(3) API implementation. This is the backing feature to implement C++11 thread storage duration specified by the thread_local keyword. A destructor for given thread-local object is registered to be executed at the thread termination time using __cxa_thread_atexit(). Libc calls the __cxa_thread_calls_dtors() during exit(3), before finalizers and atexit functions, and libthr calls the function at the thread termination time, after the stack unwinding and thread-specific key destruction. There are several uncertainties in the API which lacks a formal specification. Among them: - is it allowed to register destructors during destructing; we allow, but limiting the nesting level. If too many iterations detected, a diagnostic is issued to stderr and thread forcibly terminates for now. - how to handle destructors which belong to an unloading dso; for now, we ignore destructor calls for such entries, and issue a diagnostic. Linux does prevent dso unload until all threads with destructors from the dso terminated. It is supposed that the diagnostics allow to detect real-world applications relying on the above details and possibly adjust our implementation. Right now the choices were to provide the slim API (but that rarely stands the practice test). Tests are added to check generic functionality and to specify some of the above implementation choices. Submitted by: Mahdi Mokhtari <mokhi64_gmail.com> Reviewed by: theraven Discussed with: dim (detection of -std=c++11 supoort for tests) Sponsored by: The FreeBSD Foundation (my involvement) MFC after: 2 weeks Differential revisions: https://reviews.freebsd.org/D7224, https://reviews.freebsd.org/D7427
Some of the ports could benefit from knowing, the functionality is available. Can the OSVERSION be bumped to reflect it? Thanks!
A commit references this bug: Author: bdrewery Date: Mon Aug 22 15:52:04 UTC 2016 New revision: 304608 URL: https://svnweb.freebsd.org/changeset/base/304608 Log: Bump __FreeBSD_version for C++11 thread_local support in r303795. PR: 192320 Changes: head/sys/sys/param.h
A commit references this bug: Author: bdrewery Date: Mon Aug 22 15:53:33 UTC 2016 New revision: 304609 URL: https://svnweb.freebsd.org/changeset/base/304609 Log: MFC r304608: Bump __FreeBSD_version for C++11 thread_local support in r303795. PR: 192320 Changes: _U stable/11/ stable/11/sys/sys/param.h
A commit references this bug: Author: bdrewery Date: Mon Aug 22 16:04:25 UTC 2016 New revision: 304610 URL: https://svnweb.freebsd.org/changeset/base/304610 Log: MFS r304609: MFC r304608: Bump __FreeBSD_version for C++11 thread_local support in r303795. PR: 192320 Approved by: re (gjb) Changes: _U releng/11.0/ releng/11.0/sys/sys/param.h
A commit references this bug: Author: bdrewery Date: Mon Aug 22 16:35:51 UTC 2016 New revision: 304611 URL: https://svnweb.freebsd.org/changeset/base/304611 Log: MFC r304608: Bump __FreeBSD_version for C++11 thread_local support in r303795. PR: 192320 Changes: _U stable/10/ stable/10/sys/sys/param.h