Programs compiled with: > g++11 -g -fsanitize=address -fno-omit-frame-pointer x.cpp -I /usr/local/include/ -fopenmp fail: > $ ./a.out > ==59390==ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD. 13.1-STABLE
I have reproduced the bug. I will study how to fix it as soon as possible.
The bug is also reproduced with just "g++11 -fsanitize=address x.cpp". I am on 14.0.
I am still investigating but I have not unsterstood the issue yet. I add some people to CC that might be able to help.
I think I have found the issue and the fix. I am going to add the patches for the gcc -devel ports to the bug report and start committing them together with the weekly snapshots. Then I will fix the gcc standard ports too. For now, I have tested the patch for gcc11-devel successfully. I am still testing the other patches. Mind that it is necessary to disable ASLR to run successfully the compiled binaries, which can de done with # sysctl kern.elf64.aslr.pie_enable=1 # sysctl kern.elf64.aslr.enable=1
Created attachment 238286 [details] gcc11-devel - fix sanitizer
Created attachment 238287 [details] gcc12-devel - fix sanitizer
Created attachment 238288 [details] gcc13-devel - fix sanitizer
Please note that the patch for gcc13-devel is slightly different than the ones for gcc11-devel and gcc12-devel. The patch for gcc13-devel is the one that should be sent to upstream, assuming everything is fine. Also please note that the same bug might affect clang in the src repository too: indeed the same file I am patching here is present in contrib/llvm-project/compiler-rt/lib/asan/asan_linux.cpp. dim@, emaste@: I see you have worked on sanitizers in the src repository. Should I submit my patch there too? Or do you think it is better that I submit it directly to LLVM and then FreeBSD will merge it when it imports a new version of LLVM? Do you have any improvement to suggest on the patch itself? By the way, I have tried adding https://github.com/emaste/freebsd/commit/930a7c2ac67e1e8e511aa1d0a31a16c632060ebb into my patch, but it did not work. If I recall correctly, I have got an error 2 from execve or execvpe. I have not insisted testing it because I was unsure that it was really useful for the GCC ports, but I can test it again if you want.
I have tested successfully the patch for gcc12-devel too. However, the patch for gcc13-devel does not seem to be completely successful: the compiled binary starts, but it stops immediately with the following error from ASan. AddressSanitizer:DEADLYSIGNAL ================================================================= ==83034==ERROR: AddressSanitizer: BUS on unknown address (pc 0x0008017ae145 bp 0x00000001445a sp 0x7fffffffafc8 T0) ==83034==The signal is caused by a READ memory access. ==83034==Hint: this fault was caused by a dereference of a high value address (see register values below). Disassemble the provided pc to learn which register was used. #0 0x8017ae145 (/lib/libc.so.7+0x159145) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: BUS (/lib/libc.so.7+0x159145) ==83034==ABORTING I am doing my tests by compiling the following code: #include <iostream> using namespace std; int main() { cout << "Hello world!"; return 0; } I still have to study what this error means, if it is one more bug indipendent from the one we are fixing in this PR, if this is an expected issue in the HEAD of the GCC development or whatelse. If anyone already knows something about it, help is welcome. In the meantime, here is the commit schedule for the other gcc ports: - the commits for gcc11-devel and gcc12-devel will be done within the week together with the updates to the latest snapshots; - the commit for gcc12 will be done after the gcc11-devel and gcc12-devel commits but before the commit of the GCC_DEFAULT update, which should be done soon: indeed the exp run for it (bug #265948) has only one last known issue (bug #266189) for which I have already submitted a patch to the maintainer (which is you Yuri, but as you have probably seen it is only a USE_GCC=11 patch, so you might want to find a better solution); - the commit for gcc11 will be done after the GCC_DEFAULT update. I have built this schedule with the goal to reduce the gcc builds for users that only need GCC_DEFAULT: with this plan, they should only need to build gcc12 once and no gcc11.
> Or do you think it is better that I submit it directly to LLVM and then > FreeBSD will merge it when it imports a new version of LLVM? I think it is best to submit it to LLVM first and iterate on any feedback there first; once it's accepted the same change can be applied to the base system. This way there won't be a conflict during a future update.
> By the way, I have tried adding https://github.com/emaste/freebsd/commit/930a7c2ac67e1e8e511aa1d0a31a16c632060ebb into my patch, but it did not work. If I recall correctly, I have got an error 2 from execve or execvpe. I think this patch is desirable; maybe there is something else going wrong there.
Thanks Ed for you feedback. I will follow your suggestions and investigate more deeply why my first attempt of adding https://github.com/emaste/freebsd/commit/930a7c2ac67e1e8e511aa1d0a31a16c632060ebb failed.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=2b4f6e6f7a0b36f4592dc0058e12f55b668d6be2 commit 2b4f6e6f7a0b36f4592dc0058e12f55b668d6be2 Author: Lorenzo Salvadore <salvadore@FreeBSD.org> AuthorDate: 2022-11-25 22:52:13 +0000 Commit: Lorenzo Salvadore <salvadore@FreeBSD.org> CommitDate: 2022-11-26 15:02:45 +0000 lang/gcc11-devel: Re-execute program with ASLR disabled when necessary Software compiled with -fsanitize=address needs ASLR to be disabled to run successfully. Add patches taken from the commits listed below that improve ASLR detection and re-execute the program with ASLR disabled if necessary. https://cgit.freebsd.org/src/commit/?id=7cafe89f9ce33effe6e471b185339d413da1ca46 https://cgit.freebsd.org/src/commit/?id=930a7c2ac67e1e8e511aa1d0a31a16c632060ebb https://cgit.freebsd.org/src/commit/?id=96fe7c8ab0f65cf829619abd74ae6c126b21e15f PR: 267751 lang/gcc11-devel/Makefile | 2 +- ...er_sanitizer__common_sanitizer__linux.cpp (new) | 61 ++++++++++++++++++++++ ...zer__common_sanitizer__linux__libcdep.cpp (new) | 32 ++++++++++++ 3 files changed, 94 insertions(+), 1 deletion(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=7375f3899ae994888d48e099476d4a6aabf310b2 commit 7375f3899ae994888d48e099476d4a6aabf310b2 Author: Lorenzo Salvadore <salvadore@FreeBSD.org> AuthorDate: 2022-11-23 22:35:46 +0000 Commit: Lorenzo Salvadore <salvadore@FreeBSD.org> CommitDate: 2022-11-26 15:01:55 +0000 lang/gcc11-devel: Fix Address sanitizer Software compiled with -fsanitize=address fails to run with the error message "ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD". This commit fixes the issue by ignoring the [vdso] loaded shared library instead of linux-vdso.so. PR: 267751 Reported by: yuri lang/gcc11-devel/Makefile | 1 + .../patch-libsanitizer_asan_asan__linux.cpp (new) | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+)
(In reply to Ed Maste from comment #11) I have found the issue: I needed to merge two more of your commits. Please see the commit message in comment #13.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=12c4b8c968b78bcfe2532bedf26c5da31d75c60b commit 12c4b8c968b78bcfe2532bedf26c5da31d75c60b Author: Lorenzo Salvadore <salvadore@FreeBSD.org> AuthorDate: 2022-11-26 20:55:26 +0000 Commit: Lorenzo Salvadore <salvadore@FreeBSD.org> CommitDate: 2022-11-27 10:09:11 +0000 lang/gcc12-devel: Re-execute program with ASLR disabled when necessary Software compiled with -fsanitize=address needs ASLR to be disabled to run successfully. Add patches taken from the commits listed below that improve ASLR detection and re-execute the program with ASLR disabled if necessary. https://cgit.freebsd.org/src/commit/?id=7cafe89f9ce33effe6e471b185339d413da1ca46 https://cgit.freebsd.org/src/commit/?id=930a7c2ac67e1e8e511aa1d0a31a16c632060ebb https://cgit.freebsd.org/src/commit/?id=96fe7c8ab0f65cf829619abd74ae6c126b21e15f PR: 267751 lang/gcc12-devel/Makefile | 2 +- ...er_sanitizer__common_sanitizer__linux.cpp (new) | 61 ++++++++++++++++++++++ ...zer__common_sanitizer__linux__libcdep.cpp (new) | 32 ++++++++++++ 3 files changed, 94 insertions(+), 1 deletion(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=b185b74b28f80aa7fbeb4122a326741211af404d commit b185b74b28f80aa7fbeb4122a326741211af404d Author: Lorenzo Salvadore <salvadore@FreeBSD.org> AuthorDate: 2022-11-26 20:51:57 +0000 Commit: Lorenzo Salvadore <salvadore@FreeBSD.org> CommitDate: 2022-11-27 10:09:05 +0000 lang/gcc12-devel: Fix Address sanitizer Software compiled with -fsanitize=address fails to run with the error message "ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD". This commit fixes the issue by ignoring the [vdso] loaded shared library instead of linux-vdso.so. PR: 267751 Reported by: yuri lang/gcc12-devel/Makefile | 1 + .../patch-libsanitizer_asan_asan__linux.cpp (new) | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+)
(In reply to Lorenzo Salvadore from comment #4) > Mind that it is necessary to disable ASLR to run successfully the > compiled binaries, which can de done with > > # sysctl kern.elf64.aslr.pie_enable=1 > # sysctl kern.elf64.aslr.enable=1 I believe this needs to be "=0" in both cases, to *DISABLE* ASLR: # sysctl kern.elf64.aslr.pie_enable=0 # sysctl kern.elf64.aslr.enable=0
Great detective work, by the way, and submitting this to LLVM upstream definitely is the right approach. (Patches for older GCC branches can then be submitted more easily when something is in the GCC 13 branch.)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=d187c559f003650fe5eb4b9bcdc499f623d3b4a3 commit d187c559f003650fe5eb4b9bcdc499f623d3b4a3 Author: Lorenzo Salvadore <salvadore@FreeBSD.org> AuthorDate: 2022-11-26 22:04:19 +0000 Commit: Lorenzo Salvadore <salvadore@FreeBSD.org> CommitDate: 2022-11-30 12:52:57 +0000 lang/gcc12: Re-execute program with ASLR disabled when necessary Software compiled with -fsanitize=address needs ASLR to be disabled to run successfully. Add patches taken from the commits listed below that improve ASLR detection and re-execute the program with ASLR disabled if necessary. https://cgit.freebsd.org/src/commit/?id=7cafe89f9ce33effe6e471b185339d413da1ca46 https://cgit.freebsd.org/src/commit/?id=930a7c2ac67e1e8e511aa1d0a31a16c632060ebb https://cgit.freebsd.org/src/commit/?id=96fe7c8ab0f65cf829619abd74ae6c126b21e15f PR: 267751 lang/gcc12/Makefile | 2 +- ...er_sanitizer__common_sanitizer__linux.cpp (new) | 61 ++++++++++++++++++++++ ...zer__common_sanitizer__linux__libcdep.cpp (new) | 32 ++++++++++++ 3 files changed, 94 insertions(+), 1 deletion(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=27141421f69cb2d2bc6a462d005b251921f2c52b commit 27141421f69cb2d2bc6a462d005b251921f2c52b Author: Lorenzo Salvadore <salvadore@FreeBSD.org> AuthorDate: 2022-11-26 22:02:08 +0000 Commit: Lorenzo Salvadore <salvadore@FreeBSD.org> CommitDate: 2022-11-30 12:52:52 +0000 lang/gcc12: Fix Address sanitizer Software compiled with -fsanitize=address fails to run with the error message "ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD". This commit fixes the issue by ignoring the [vdso] loaded shared library instead of linux-vdso.so. PR: 267751 Reported by: yuri lang/gcc12/Makefile | 2 +- .../patch-libsanitizer_asan_asan__linux.cpp (new) | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-)
(In reply to Gerald Pfeifer from comment #18) Yes Gerald, you are right. I must have copied the commands I used to re-enable ASLR by mistake. Thanks.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=fb398d22e6faf020957b45b43c9f6b9450c04cc9 commit fb398d22e6faf020957b45b43c9f6b9450c04cc9 Author: Lorenzo Salvadore <salvadore@FreeBSD.org> AuthorDate: 2022-11-26 22:12:09 +0000 Commit: Lorenzo Salvadore <salvadore@FreeBSD.org> CommitDate: 2022-12-03 17:24:10 +0000 lang/gcc11: Fix Address sanitizer Software compiled with -fsanitize=address fails to run with the error message "ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD". This commit fixes the issue by ignoring the [vdso] loaded shared library instead of linux-vdso.so. PR: 267751 Reported by: yuri lang/gcc11/Makefile | 2 +- .../patch-libsanitizer_asan_asan__linux.cpp (new) | 18 ++++++++++++++++++ 2 files changed, 19 insertions(+), 1 deletion(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=a051e5a32dd9b36fe2b03fe152b94ee2cdce4926 commit a051e5a32dd9b36fe2b03fe152b94ee2cdce4926 Author: Lorenzo Salvadore <salvadore@FreeBSD.org> AuthorDate: 2022-11-26 22:13:47 +0000 Commit: Lorenzo Salvadore <salvadore@FreeBSD.org> CommitDate: 2022-12-03 17:24:17 +0000 lang/gcc11: Re-execute program with ASLR disabled when necessary Software compiled with -fsanitize=address needs ASLR to be disabled to run successfully. Add patches taken from the commits listed below that improve ASLR detection and re-execute the program with ASLR disabled if necessary. https://cgit.freebsd.org/src/commit/?id=7cafe89f9ce33effe6e471b185339d413da1ca46 https://cgit.freebsd.org/src/commit/?id=930a7c2ac67e1e8e511aa1d0a31a16c632060ebb https://cgit.freebsd.org/src/commit/?id=96fe7c8ab0f65cf829619abd74ae6c126b21e15f PR: 267751 lang/gcc11/Makefile | 2 +- ...er_sanitizer__common_sanitizer__linux.cpp (new) | 61 ++++++++++++++++++++++ ...zer__common_sanitizer__linux__libcdep.cpp (new) | 32 ++++++++++++ 3 files changed, 94 insertions(+), 1 deletion(-)
Created attachment 247061 [details] Possible fix for BUS error on asan init This is a possible fix for the bus error happening on gcc-13 and up when running asan tests. (Or other binaries compiled with -fsanitize=address). I tested it with gmake check-gcc RUNTESTFLAGS=asan.exp on current gcc-13 git branch. (including the patch in FindFirstDSOCallback). There are only the known swapcontext failures. As said, a possible fix. Maybe we have to investigate more in deep. But at least it is a starting point.
(In reply to Andreas Tobler from comment #25) Thanks a lot for this patch. Unfortunately, I am overwhelmed with PRs right now, so it might take some time for me to review and test your patch. But I appreciate it a lot and I hope to merge it before the switch to GCC_DEFAULT=13 so that we do not have a regression on address sanitizer by updating the default GCC version. Thanks.
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=99c1ed39b70e5c82b3d8d0b01eac6e4ab8438693 commit 99c1ed39b70e5c82b3d8d0b01eac6e4ab8438693 Author: Lorenzo Salvadore <salvadore@FreeBSD.org> AuthorDate: 2024-01-12 14:25:09 +0000 Commit: Lorenzo Salvadore <salvadore@FreeBSD.org> CommitDate: 2024-01-13 08:18:27 +0000 lang/gcc14-devel: Fix Address sanitizer Software compiled with -fsanitize=address fails to run with the error message "ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD". This commit fixes the issue by ignoring the [vdso] loaded shared library instead of linux-vdso.so. PR: 267751 Reported by: yuri Co-authored-by: Andreas Tobler <andreast@gcc.gnu.org> lang/gcc14-devel/Makefile | 1 + .../patch-libsanitizer_asan_asan__linux.cpp (new) | 23 ++++++++++++++++++++++ .../patch-libsanitizer_asan_asan__thread.cpp (new) | 13 ++++++++++++ 3 files changed, 37 insertions(+)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=11e18f525a8390fda31c44838bde003e23176b95 commit 11e18f525a8390fda31c44838bde003e23176b95 Author: Lorenzo Salvadore <salvadore@FreeBSD.org> AuthorDate: 2024-01-12 14:34:58 +0000 Commit: Lorenzo Salvadore <salvadore@FreeBSD.org> CommitDate: 2024-01-13 08:18:34 +0000 lang/gcc13: Fix Address sanitizer Software compiled with -fsanitize=address fails to run with the error message "ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD". This commit fixes the issue by ignoring the [vdso] loaded shared library instead of linux-vdso.so. To successfully run the software compiled with -fsanitize=address it is still necessary to disable ASLR. PR: 267751 Reported by: yuri Co-authored-by: Andreas Tobler <andreast@gcc.gnu.org> lang/gcc13/Makefile | 2 +- .../patch-libsanitizer_asan_asan__linux.cpp (new) | 23 ++++++++++++++++++++++ .../patch-libsanitizer_asan_asan__thread.cpp (new) | 13 ++++++++++++ 3 files changed, 37 insertions(+), 1 deletion(-)
A commit in branch main references this bug: URL: https://cgit.FreeBSD.org/ports/commit/?id=9c8633f6b2061d35c78e745fbafb7666357eaa8e commit 9c8633f6b2061d35c78e745fbafb7666357eaa8e Author: Lorenzo Salvadore <salvadore@FreeBSD.org> AuthorDate: 2022-11-23 11:51:44 +0000 Commit: Lorenzo Salvadore <salvadore@FreeBSD.org> CommitDate: 2024-01-13 08:17:23 +0000 lang/gcc13-devel: Fix Address sanitizer Software compiled with -fsanitize=address fails to run with the error message "ASan runtime does not come first in initial library list; you should either link runtime to your application or manually preload it with LD_PRELOAD". This commit fixes the issue by ignoring the [vdso] loaded shared library instead of linux-vdso.so. To successfully run the software compiled with -fsanitize=address it is still necessary to disable ASLR. PR: 267751 Reported by: yuri Co-authored-by: Andreas Tobler <andreast@gcc.gnu.org> lang/gcc13-devel/Makefile | 2 +- .../patch-libsanitizer_asan_asan__linux.cpp (new) | 23 ++++++++++++++++++++++ .../patch-libsanitizer_asan_asan__thread.cpp (new) | 13 ++++++++++++ 3 files changed, 37 insertions(+), 1 deletion(-)
The patch provided by Andreas provided the final element that we were missing for fixing this issue. Now all GCC >= 11 ports have address sanitizer working and we can finally close this bug. Thanks to everyone.
(In reply to Andreas Tobler from comment #25) > This is a possible fix for the bus error happening on gcc-13 and > up when running asan tests. (Or other binaries compiled with > -fsanitize=address). I tested it with > gmake check-gcc RUNTESTFLAGS=asan.exp on current gcc-13 git branch. A belated Thank you! Is this patch upstream as well or are you planning to engage upstream? It would be nice to avoid a FreeBSD Ports-specific patch here...