Bug 277047 - /usr/lib/libcompiler_rt.a(atomic.o) hidden visibility causes non-atomic atomics
Summary: /usr/lib/libcompiler_rt.a(atomic.o) hidden visibility causes non-atomic atomics
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: misc (show other bugs)
Version: Unspecified
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-toolchain (Nobody)
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-02-14 08:21 UTC by Fangrui Song
Modified: 2024-02-15 09:33 UTC (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Fangrui Song 2024-02-14 08:21:13 UTC
lib/libcompiler_rt/ builds llvm-project/compiler-rt including atomics (for arm there is additional sys/arm/arm/stdatomic.c).
Many packages have a dependency on lib/libcompiler_rt. lib/libcompiler_rt is installed to /usr/lib/libcompiler_rt.a. 
The symbols are hidden, which usually lead to benign ODR violations if both an executable and its DT_NEEDED get a copy of libcompiler_rt.a. 

However, atomic.o has a global lock pool, and having different lock pools would lead to non-synchronized atomic operations.

Prepare input files as attached at the end of this post.

% zsh a.sh
/tmp/b-9aaa82.o: reference to __atomic_compare_exchange
/usr/lib/libcompiler_rt.a(atomic.o): definition of __atomic_compare_exchange
/tmp/a-864004.o: reference to __atomic_compare_exchange
/usr/lib/libcompiler_rt.a(atomic.o): definition of __atomic_compare_exchange
9470401
/tmp/b-28139a.o: reference to __atomic_compare_exchange
/usr/lib/libgcc.a(atomic.o): definition of __atomic_compare_exchange
/tmp/a-1dceb9.o: reference to __atomic_compare_exchange
/usr/lib/libgcc.a(atomic.o): definition of __atomic_compare_exchange
9481088

The expected result is 10000000. libgcc.a has the same issue.
In practice it is extremely rare when two pieces of code operating on the same atomic lives in different link units.

---

cat > a.cc <<'eof'
#include <cstdint>
#include <iostream>
#include <thread>
#include <vector>
using namespace std;

#define REP(i, n) for (int i = 0; i < (n); i++)

constexpr int kIterations = 500'000;
__int128_t g;

void loop_b();
void loop_a() {
  REP(i, kIterations) {
    for(;;) {
      auto e = __atomic_load_n(&g, __ATOMIC_RELAXED), desired = e+1;
      if (__atomic_compare_exchange_n(&g, &e, desired, 0, __ATOMIC_RELAXED, __ATOMIC_RELAXED))
        break;
    }
  }
}

int main() {
  vector<thread> ts;
  REP(i, 10) ts.emplace_back(loop_a), ts.emplace_back(loop_b);
  for (auto &t : ts) t.join();
  cout << (int64_t)g << '\n';
}
eof
cat > a.sh <<'eof'
clang++ -O1 -fpic -Wno-atomic-alignment b.cc -shared -lcompiler_rt -o b.so -Wl,-y,__atomic_compare_exchange
clang++ -O1 -Wno-atomic-alignment a.cc ./b.so -lcompiler_rt -pthread -o a -Wl,-y,__atomic_compare_exchange
./a

clang++ -O1 -fpic -Wno-atomic-alignment b.cc -shared -o b.gcc.so -Wl,-y,__atomic_compare_exchange
clang++ -O1 -Wno-atomic-alignment a.cc ./b.gcc.so -pthread -o a.gcc -Wl,-y,__atomic_compare_exchange
./a.gcc
eof
cat > b.cc <<'eof'
#define REP(i, n) for (int i = 0; i < (n); i++)
extern __int128_t g;
constexpr int kIterations = 500'000;
void loop_b() {
  REP(i, kIterations) {
    for(;;) {
      auto e = __atomic_load_n(&g, __ATOMIC_RELAXED);
      auto desired = e + 1;
      if (__atomic_compare_exchange_n(&g, &e, desired, 0, __ATOMIC_RELAXED, __ATOMIC_RELAXED))
        break;
    }
  }
}
eof
Comment 1 Robert Clausecker freebsd_committer freebsd_triage 2024-02-14 12:31:54 UTC
CC kib@
Comment 2 Konstantin Belousov freebsd_committer freebsd_triage 2024-02-15 07:53:20 UTC
There is no atomics ABI.  libcompiler_rt.a (or libgcc.a) provides utilities
private to the specific compiler, and must not leak into the object' ABI.

I do not think that the atomics issue can be solved until all compiler authors
agree on something, and then this agreement is expressed in psABI docs.
Comment 3 David Chisnall freebsd_committer freebsd_triage 2024-02-15 09:33:59 UTC
Both LLVM and GCC generate calls to the same helper functions and expect them to be provided by a platform support library.

I wrote the first version of the code for the LLVM atomics helpers and the expectation was always that system integrators would build it into a libatomics.so, into their C runtime, or their C standard library for dynamic linking, or include it for static linking.

FreeBSD seems build it into compiler-rt.  It was intentionally *not* connected to the compiler-rt build system upstream to avoid this.  It is not intended as part of the compiler-rt static linking interface, it is a file that exists so that system integrators building a toolchain with LLVM components don't have to start from scratch when implementing the atomic helpers.