Created attachment 156437 [details] C source file Compile and run the attached foo.c, then examine the resulting core dump with gdb79 (please see the transcript below). This results in gdb79 dumping core. For comparison purposes, gdb782 has a glitch here (a duplicated backtrace for one thread) but it doesn't crash. [1 ~]$ uname -a FreeBSD xxx.yy 10.1-STABLE FreeBSD 10.1-STABLE #0 r280136: Mon Mar 16 18:59:32 CET 2015 adm@xxx.yy:/usr/obj/usr/src/sys/IZNOGOUD amd64 [2 ~]$ cc -Wall -g -O0 -o foo foo.c -lthr [3 ~]$ ./foo Abort trap (core dumped) [4 ~]$ gdb79 --core=foo.core ./foo GNU gdb (GDB) 7.9 [GDB v7.9 for FreeBSD] Copyright (C) 2015 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-portbld-freebsd10.1". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ./foo...done. [New process 101209] [New process 100339] [New Thread 801406800 (LWP 101209)] [New Thread 801406400 (LWP 100339)] Core was generated by `foo'. Program terminated with signal SIGABRT, Aborted. Invalid selected thread. (gdb) thread apply all bt Thread 4 (Thread 801406400 (LWP 100339)): #0 _nanosleep () at _nanosleep.S:3 #1 0x0000000800829b8c in __thr_nanosleep (time_to_sleep=0x7fffffffe4f8, time_remaining=0x7fffffffe4e8) at /usr/src/lib/libthr/thread/thr_syscalls.c:269 #2 0x0000000800aa2ffb in __sleep (seconds=2147483647) at /usr/src/lib/libc/gen/sleep.c:60 #3 0x0000000000400952 in main () at foo.c:35 Segmentation fault (core dumped) [5 ~]$ /tmp/usr/local/bin/gdb782 --core=foo.core ./foo GNU gdb (GDB) 7.8.2 [GDB v7.8.2 for FreeBSD] Copyright (C) 2014 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-portbld-freebsd10.1". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from ./foo...done. [New process 101209] [New process 100339] [New Thread 801406800 (LWP 101209)] [New Thread 801406400 (LWP 100339)] Core was generated by `foo'. Program terminated with signal SIGABRT, Aborted. #0 thr_kill () at thr_kill.S:3 3 RSYSCALL(thr_kill) (gdb) thread apply all bt Thread 4 (Thread 801406400 (LWP 100339)): #0 _nanosleep () at _nanosleep.S:3 #1 0x0000000800829b8c in __thr_nanosleep (time_to_sleep=0x7fffffffe4f8, time_remaining=0x7fffffffe4e8) at /usr/src/lib/libthr/thread/thr_syscalls.c:269 #2 0x0000000800aa2ffb in __sleep (seconds=2147483647) at /usr/src/lib/libc/gen/sleep.c:60 #3 0x0000000000400952 in main () at foo.c:35 Thread 3 (Thread 801406800 (LWP 101209)): #0 thr_kill () at thr_kill.S:3 #1 0x0000000800b896c6 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:51 #2 0x0000000800b87ea9 in abort () at /usr/src/lib/libc/stdlib/abort.c:65 #3 0x00000000004008f0 in foo_thread (arg=0x0) at foo.c:19 #4 0x0000000800827775 in thread_start (curthread=0x801406800) at /usr/src/lib/libthr/thread/thr_create.c:288 #5 0x0000000000000000 in ?? () Backtrace stopped: Cannot access memory at address 0x7fffdfffe000 Thread 1 (Thread 801406800 (LWP 101209)): #0 thr_kill () at thr_kill.S:3 #1 0x0000000800b896c6 in __raise (s=6) at /usr/src/lib/libc/gen/raise.c:51 #2 0x0000000800b87ea9 in abort () at /usr/src/lib/libc/stdlib/abort.c:65 #3 0x00000000004008f0 in foo_thread (arg=0x0) at foo.c:19 #4 0x0000000800827775 in thread_start (curthread=0x801406800) at /usr/src/lib/libthr/thread/thr_create.c:288 #5 0x0000000000000000 in ?? () Backtrace stopped: Cannot access memory at address 0x7fffdfffe000 (gdb) quit [6 ~]$ gdb --core=foo.core ./foo GNU gdb 6.1.1 [FreeBSD] Copyright 2004 Free Software Foundation, Inc. GDB is free software, covered by the GNU General Public License, and you are welcome to change it and/or distribute copies of it under certain conditions. Type "show copying" to see the conditions. There is absolutely no warranty for GDB. Type "show warranty" for details. This GDB was configured as "amd64-marcel-freebsd"... Core was generated by `foo'. Program terminated with signal 6, Aborted. Reading symbols from /lib/libthr.so.3...Reading symbols from /usr/lib/debug//lib/libthr.so.3.debug...done. done. Loaded symbols for /lib/libthr.so.3 Reading symbols from /lib/libc.so.7...Reading symbols from /usr/lib/debug//lib/libc.so.7.debug...done. done. Loaded symbols for /lib/libc.so.7 Reading symbols from /libexec/ld-elf.so.1...Reading symbols from /usr/lib/debug//libexec/ld-elf.so.1.debug...done. done. Loaded symbols for /libexec/ld-elf.so.1 #0 thr_kill () at thr_kill.S:3 3 RSYSCALL(thr_kill) [New Thread 801406800 (LWP 101209/foo)] [New Thread 801406400 (LWP 100339/foo)] (gdb) thread apply all bt Thread 2 (Thread 801406400 (LWP 100339/foo)): #0 _nanosleep () at _nanosleep.S:3 #1 0x0000000800829b8c in __thr_nanosleep (time_to_sleep=0x7fffffffe4f8, time_remaining=0x7fffffffe4e8) at /usr/src/lib/libthr/thread/thr_syscalls.c:269 #2 0x0000000800aa2ffb in __sleep () at /usr/src/lib/libc/gen/sleep.c:60 #3 0x0000000000400952 in main () at foo.c:35 Current language: auto; currently asm Thread 1 (Thread 801406800 (LWP 101209/foo)): #0 thr_kill () at thr_kill.S:3 #1 0x0000000800b896c6 in __raise (s=<value optimized out>) at /usr/src/lib/libc/gen/raise.c:51 #2 0x0000000800b87ea9 in abort () at /usr/src/lib/libc/stdlib/abort.c:65 #3 0x00000000004008f0 in foo_thread (arg=0x0) at foo.c:19 #4 0x0000000800827775 in thread_start (curthread=0x801406800) at /usr/src/lib/libthr/thread/thr_create.c:288 #5 0x0000000000000000 in ?? () (gdb) [7 ~]$
Created attachment 156468 [details] C source Simpler test case.
I was unable to reproduce on HEAD, will try stable/10 later today. However, in my case I only got "New Thread" lines when parsing the core, I did not get both "New Process" and "New Thread" and I suspect that is related. Did you build gdb79 from ports or install it from a package repository? If the latter, was it from pkg.freebsd.org it from your own build?
Hi, The big suspect here is the thread support and how the thread list is created when the libthr.so is loaded. There was an issue with gdb78, (solved in the revision 365578) and I guess that the fix is not able to handle the load of libthr.so when the process has already more thread (attach or core dump load of multithread programs) In other words, if you run the application inside gdb, libthr.so is processed at the beginning, when only 1 thread is running and everything is fine. gdb79 ./foo tested, it works. If you analyze the core dump or you dynamically attach gdb to a running process, gdb has inconsistent information about threads.
(In reply to John Baldwin from comment #2) I compiled it from ports (using portmaster).
(In reply to luca.pizzamiglio from comment #3) Running foo under gdb79 works for me as well. As the transcript above shows, gdb782 --core=foo.core ./foo does not crash but it seems to produce a duplicate backtrace of one thread, somehow.
Created attachment 156667 [details] clear_thread_list.patch This is the workaround I've been testing. It does fix this, but it's not clear to me what the most correct fix for this is. Other libthread_db targets in gdb for Solaris and Linux just avoid attaching to core dumps entirely. This patch makes the FreeBSD thread target throw away the threads enumerated by the default core dump target and re-enumerate them using the FreeBSD target directly.
Created attachment 157220 [details] get_current_thread before thread enumeration
(In reply to Eric Badger from comment #7) I think Luca's patch can work here, but the setting of ptid happens too late; when enumerating threads, the old thread isn't matched and so a new (duplicate) is created. Attached a possible patch that seems to handle correctly core files and the case of attaching to running processes with multiple threads.
I have given Luca a similar patch (albeit a bit larger) that generally does the same thing to avoid adding a double thread. I think there are still issues with the resume method though that aren't easy to untangle.
Hi to all The patch I gave to Eric is extracted from the John's one. I hopefully thought it could solve the attaching issue. I'm going to integrate the previous submitted patch, that solves this issue, in the next PR I'm preparing.
A commit references this bug: Author: tijl Date: Thu Jun 4 17:08:27 UTC 2015 New revision: 388538 URL: https://svnweb.freebsd.org/changeset/ports/388538 Log: - Update to 7.9.1 - Fix crash on some core files - Fix plist PR: 200001, 200011, 200638 Submitted by: luca.pizzamiglio@gmail.com (maintainer) Changes: head/devel/gdb/Makefile head/devel/gdb/distinfo head/devel/gdb/files/fbsd-threads.c head/devel/gdb/pkg-plist