201626 – dtrace is unable to run on head since ipsec import

Bug 201626 - dtrace is unable to run on head since ipsec import

Summary: dtrace is unable to run on head since ipsec import

Status:	Closed FIXED

Alias:	None

Product:	Base System
Classification:	Unclassified
Component:	kern (show other bugs)
Version:	CURRENT
Hardware:	amd64 Any

Importance:	--- Affects Some People
Assignee:	Mark Johnston

URL:
Keywords:

Depends on:
Blocks:

Reported:	2015-07-16 17:01 UTC by Sean Bruno
Modified:	2017-01-03 17:40 UTC (History)
CC List:	2 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.

Description Sean Bruno freebsd_committer

2015-07-16 17:01:27 UTC

All dtrace scripts seem to error out on head nowadays with GENERIC:

# dtrace -n 'syscall:::entry { @[probefunc] = count(); }'
libdtrace DEBUG: opened provider dtrace
libdtrace DEBUG: opened provider profile
libdtrace DEBUG: opened provider lockstat
libdtrace DEBUG: opened provider fbt
libdtrace DEBUG: opened provider dtmalloc
libdtrace DEBUG: opened 64-bit module kernel () [0]
libdtrace DEBUG: opened 64-bit module hwpmc.ko () [0]
libdtrace DEBUG: opened 64-bit module tmpfs.ko () [0]
libdtrace DEBUG: opened 64-bit module dtraceall.ko () [0]
libdtrace DEBUG: opened 64-bit module opensolaris.ko () [0]
libdtrace DEBUG: opened 64-bit module dtrace.ko () [0]
libdtrace DEBUG: opened 64-bit module dtmalloc.ko () [0]
libdtrace DEBUG: opened 64-bit module dtnfscl.ko () [0]
libdtrace DEBUG: opened 64-bit module fbt.ko () [0]
libdtrace DEBUG: opened 64-bit module fasttrap.ko () [0]
libdtrace DEBUG: opened 64-bit module lockstat.ko () [0]
libdtrace DEBUG: opened 64-bit module sdt.ko () [0]
libdtrace DEBUG: opened 64-bit module systrace.ko () [0]
libdtrace DEBUG: opened 64-bit module systrace_freebsd32.ko () [0]
libdtrace DEBUG: opened 64-bit module profile.ko () [0]
libdtrace DEBUG: created CTF container for C (0x802852580)
libdtrace DEBUG: created CTF container for D (0x802852800)
libdtrace DEBUG: loaded kernel [.SUNW_ctf] (913514 bytes)
libdtrace DEBUG: loaded kernel [.symtab] (1382040 bytes)
libdtrace DEBUG: loaded kernel [.strtab] (1495191 bytes)
libdtrace DEBUG: hashed kernel [.symtab] (57543 symbols)
libdtrace DEBUG: sorted kernel [.symtab] (57490 symbols)
libdtrace DEBUG: loaded CTF container for kernel (0x802852a80)
libdtrace DEBUG: loaded printf conversion %a
libdtrace DEBUG: loaded printf conversion %A
libdtrace DEBUG: loaded printf conversion %c
libdtrace DEBUG: loaded printf conversion %C
libdtrace DEBUG: loaded printf conversion %d
libdtrace DEBUG: loaded printf conversion %e
libdtrace DEBUG: loaded printf conversion %E
libdtrace DEBUG: loaded printf conversion %f
libdtrace DEBUG: loaded printf conversion %g
libdtrace DEBUG: loaded printf conversion %G
libdtrace DEBUG: loaded printf conversion %hd
libdtrace DEBUG: loaded printf conversion %hi
libdtrace DEBUG: loaded printf conversion %ho
libdtrace DEBUG: loaded printf conversion %hu
libdtrace DEBUG: loaded printf conversion %hx
libdtrace DEBUG: loaded printf conversion %hX
libdtrace DEBUG: loaded printf conversion %i
libdtrace DEBUG: loaded printf conversion %I
libdtrace DEBUG: loaded printf conversion %k
libdtrace DEBUG: loaded printf conversion %lc
libdtrace DEBUG: loaded printf conversion %ld
libdtrace DEBUG: loaded printf conversion %li
libdtrace DEBUG: loaded printf conversion %lo
libdtrace DEBUG: loaded printf conversion %lu
libdtrace DEBUG: loaded printf conversion %ls
libdtrace DEBUG: loaded printf conversion %lx
libdtrace DEBUG: loaded printf conversion %lX
libdtrace DEBUG: loaded printf conversion %lld
libdtrace DEBUG: loaded printf conversion %lli
libdtrace DEBUG: loaded printf conversion %llo
libdtrace DEBUG: loaded printf conversion %llu
libdtrace DEBUG: loaded printf conversion %llx
libdtrace DEBUG: loaded printf conversion %llX
libdtrace DEBUG: loaded printf conversion %Le
libdtrace DEBUG: loaded printf conversion %LE
libdtrace DEBUG: loaded printf conversion %Lf
libdtrace DEBUG: loaded printf conversion %Lg
libdtrace DEBUG: loaded printf conversion %LG
libdtrace DEBUG: loaded printf conversion %o
libdtrace DEBUG: loaded printf conversion %p
libdtrace DEBUG: loaded printf conversion %P
libdtrace DEBUG: loaded printf conversion %s
libdtrace DEBUG: loaded printf conversion %S
libdtrace DEBUG: loaded printf conversion %T
libdtrace DEBUG: loaded printf conversion %u
libdtrace DEBUG: loaded printf conversion %wc
libdtrace DEBUG: loaded printf conversion %ws
libdtrace DEBUG: loaded printf conversion %x
libdtrace DEBUG: loaded printf conversion %X
libdtrace DEBUG: loaded printf conversion %Y
libdtrace DEBUG: loaded printf conversion %%
libdtrace DEBUG: populating global idhash from 0x800ac3190
libdtrace DEBUG: loaded hwpmc.ko [.SUNW_ctf] (33044 bytes)
libdtrace DEBUG: loaded hwpmc.ko [.symtab] (9720 bytes)
libdtrace DEBUG: loaded hwpmc.ko [.strtab] (6706 bytes)
libdtrace DEBUG: hashed hwpmc.ko [.symtab] (390 symbols)
libdtrace DEBUG: sorted hwpmc.ko [.symtab] (260 symbols)
libdtrace DEBUG: loaded CTF container for hwpmc.ko (0x802852d00)
libdtrace DEBUG: loaded tmpfs.ko [.SUNW_ctf] (18423 bytes)
libdtrace DEBUG: loaded tmpfs.ko [.symtab] (5736 bytes)
libdtrace DEBUG: loaded tmpfs.ko [.strtab] (3872 bytes)
libdtrace DEBUG: hashed tmpfs.ko [.symtab] (225 symbols)
libdtrace DEBUG: sorted tmpfs.ko [.symtab] (96 symbols)
libdtrace DEBUG: loaded CTF container for tmpfs.ko (0x802852f80)
libdtrace DEBUG: loaded dtraceall.ko [.SUNW_ctf] (1297 bytes)
libdtrace DEBUG: loaded dtraceall.ko [.symtab] (1512 bytes)
libdtrace DEBUG: loaded dtraceall.ko [.strtab] (2050 bytes)
libdtrace DEBUG: hashed dtraceall.ko [.symtab] (52 symbols)
libdtrace DEBUG: sorted dtraceall.ko [.symtab] (41 symbols)
libdtrace DEBUG: loaded CTF container for dtraceall.ko (0x802853200)
libdtrace DEBUG: loaded opensolaris.ko [.SUNW_ctf] (8481 bytes)
libdtrace DEBUG: loaded opensolaris.ko [.symtab] (3000 bytes)
libdtrace DEBUG: loaded opensolaris.ko [.strtab] (2130 bytes)
libdtrace DEBUG: hashed opensolaris.ko [.symtab] (112 symbols)
libdtrace DEBUG: sorted opensolaris.ko [.symtab] (63 symbols)
libdtrace DEBUG: loaded CTF container for opensolaris.ko (0x802853480)
libdtrace DEBUG: loaded dtrace.ko [.SUNW_ctf] (28274 bytes)
libdtrace DEBUG: loaded dtrace.ko [.symtab] (12744 bytes)
libdtrace DEBUG: loaded dtrace.ko [.strtab] (9838 bytes)
libdtrace DEBUG: hashed dtrace.ko [.symtab] (515 symbols)
libdtrace DEBUG: sorted dtrace.ko [.symtab] (384 symbols)
libdtrace DEBUG: loaded CTF container for dtrace.ko (0x802853700)
libdtrace DEBUG: loaded dtmalloc.ko [.SUNW_ctf] (16736 bytes)
libdtrace DEBUG: loaded dtmalloc.ko [.symtab] (1680 bytes)
libdtrace DEBUG: loaded dtmalloc.ko [.strtab] (1435 bytes)
libdtrace DEBUG: hashed dtmalloc.ko [.symtab] (57 symbols)
libdtrace DEBUG: sorted dtmalloc.ko [.symtab] (30 symbols)
libdtrace DEBUG: loaded CTF container for dtmalloc.ko (0x802853c00)
libdtrace DEBUG: loaded dtnfscl.ko [.SUNW_ctf] (16704 bytes)
libdtrace DEBUG: loaded dtnfscl.ko [.symtab] (2184 bytes)
libdtrace DEBUG: loaded dtnfscl.ko [.strtab] (2332 bytes)
libdtrace DEBUG: hashed dtnfscl.ko [.symtab] (78 symbols)
libdtrace DEBUG: sorted dtnfscl.ko [.symtab] (33 symbols)
libdtrace DEBUG: loaded CTF container for dtnfscl.ko (0x802853e80)
libdtrace DEBUG: loaded fbt.ko [.SUNW_ctf] (18767 bytes)
libdtrace DEBUG: loaded fbt.ko [.symtab] (2592 bytes)
libdtrace DEBUG: loaded fbt.ko [.strtab] (1833 bytes)
libdtrace DEBUG: hashed fbt.ko [.symtab] (95 symbols)
libdtrace DEBUG: sorted fbt.ko [.symtab] (48 symbols)
libdtrace DEBUG: loaded CTF container for fbt.ko (0x802854100)
libdtrace DEBUG: loaded fasttrap.ko [.SUNW_ctf] (21427 bytes)
libdtrace DEBUG: loaded fasttrap.ko [.symtab] (5760 bytes)
libdtrace DEBUG: loaded fasttrap.ko [.strtab] (4119 bytes)
libdtrace DEBUG: hashed fasttrap.ko [.symtab] (225 symbols)
libdtrace DEBUG: sorted fasttrap.ko [.symtab] (116 symbols)
libdtrace DEBUG: loaded CTF container for fasttrap.ko (0x802854380)
libdtrace DEBUG: loaded lockstat.ko [.SUNW_ctf] (16613 bytes)
libdtrace DEBUG: loaded lockstat.ko [.symtab] (1584 bytes)
libdtrace DEBUG: loaded lockstat.ko [.strtab] (1380 bytes)
libdtrace DEBUG: hashed lockstat.ko [.symtab] (54 symbols)
libdtrace DEBUG: sorted lockstat.ko [.symtab] (29 symbols)
libdtrace DEBUG: loaded CTF container for lockstat.ko (0x802854880)
libdtrace DEBUG: loaded sdt.ko [.SUNW_ctf] (2981 bytes)
libdtrace DEBUG: loaded sdt.ko [.symtab] (1920 bytes)
libdtrace DEBUG: loaded sdt.ko [.strtab] (1469 bytes)
libdtrace DEBUG: hashed sdt.ko [.symtab] (68 symbols)
libdtrace DEBUG: sorted sdt.ko [.symtab] (30 symbols)
libdtrace DEBUG: loaded CTF container for sdt.ko (0x802854b00)
libdtrace DEBUG: loaded systrace.ko [.SUNW_ctf] (23857 bytes)
libdtrace DEBUG: loaded systrace.ko [.symtab] (1728 bytes)
libdtrace DEBUG: loaded systrace.ko [.strtab] (1429 bytes)
libdtrace DEBUG: hashed systrace.ko [.symtab] (59 symbols)
libdtrace DEBUG: sorted systrace.ko [.symtab] (31 symbols)
libdtrace DEBUG: loaded CTF container for systrace.ko (0x802855280)
libdtrace DEBUG: loaded systrace_freebsd32.ko [.SUNW_ctf] (23600 bytes)
libdtrace DEBUG: loaded systrace_freebsd32.ko [.symtab] (1752 bytes)
libdtrace DEBUG: loaded systrace_freebsd32.ko [.strtab] (1626 bytes)
libdtrace DEBUG: hashed systrace_freebsd32.ko [.symtab] (60 symbols)
libdtrace DEBUG: sorted systrace_freebsd32.ko [.symtab] (31 symbols)
libdtrace DEBUG: loaded CTF container for systrace_freebsd32.ko (0x802855780)
libdtrace DEBUG: loaded profile.ko [.SUNW_ctf] (16786 bytes)
libdtrace DEBUG: loaded profile.ko [.symtab] (2112 bytes)
libdtrace DEBUG: loaded profile.ko [.strtab] (1624 bytes)
libdtrace DEBUG: hashed profile.ko [.symtab] (75 symbols)
libdtrace DEBUG: sorted profile.ko [.symtab] (35 symbols)
libdtrace DEBUG: loaded CTF container for profile.ko (0x802855a00)
libdtrace DEBUG: error parsing library /usr/lib/dtrace/siftr.d: "/usr/lib/dtrace/siftr.d", line 24: program requires module siftr
libdtrace DEBUG: library /usr/lib/dtrace/unistd.d sorted (1/2)
libdtrace DEBUG: library /usr/lib/dtrace/regs_x86.d sorted (3/4)
libdtrace DEBUG: library /usr/lib/dtrace/psinfo.d sorted (5/6)
libdtrace DEBUG: library /usr/lib/dtrace/tcp.d sorted (7/8)
libdtrace DEBUG: library /usr/lib/dtrace/udp.d sorted (9/10)
libdtrace DEBUG: library /usr/lib/dtrace/signal.d sorted (11/12)
libdtrace DEBUG: library /usr/lib/dtrace/errno.d sorted (13/14)
libdtrace DEBUG: library /usr/lib/dtrace/siftr.d sorted (15/16)
libdtrace DEBUG: library /usr/lib/dtrace/io.d sorted (17/18)
libdtrace DEBUG: library /usr/lib/dtrace/ip.d sorted (19/20)
libdtrace DEBUG: typedef pktinfo_t added as id 32798
libdtrace DEBUG: typedef zoneid_t added as id 32799
libdtrace DEBUG: typedef csinfo_t added as id 32801
libdtrace DEBUG: typedef ipinfo_t added as id 32803
libdtrace DEBUG: typedef ifinfo_t added as id 32805
libdtrace DEBUG: typedef ipaddr_t added as id 32806
libdtrace DEBUG: typedef ipha_t added as id 32808
libdtrace DEBUG: typedef ipv4info_t added as id 32811
dtrace: invalid probe specifier syscall:::entry { @[probefunc] = count(); }: "/usr/lib/dtrace/ip.d", line 111: failed to copy typedef in6_addr_t source type: Type information is in parent and unavailable
libdtrace DEBUG: dt_buf_destroy(section headers): size=512 resizes=0
libdtrace DEBUG: dt_buf_destroy(string table): size=512 resizes=0
libdtrace DEBUG: dt_buf_destroy(loadable data): size=512 resizes=0
libdtrace DEBUG: dt_buf_destroy(unloadable data): size=512 resizes=0
libdtrace DEBUG: dt_buf_destroy(probe data): size=512 resizes=0
libdtrace DEBUG: dt_buf_destroy(probe args): size=512 resizes=0
libdtrace DEBUG: dt_buf_destroy(probe offs): size=512 resizes=0
libdtrace DEBUG: dt_buf_destroy(probe is-enabled offs): size=512 resizes=0
libdtrace DEBUG: dt_buf_destroy(probe rels): size=512 resizes=0
libdtrace DEBUG: dt_buf_destroy(xlate members): size=512 resizes=0

Comment 1 Sean Bruno freebsd_committer

2015-07-16 17:01:44 UTC

 # ctfdump -S /boot/kernel/kernel

- CTF Statistics -------------------------------------------------------------

  total number of data objects        = 27090

  total number of functions           = 27781
  total number of function arguments  = 64171
  maximum argument list length        = 19
  average argument list length        = 2.31

  total number of types               = 35195
  total number of integers            = 80
  total number of floats              = 0
  total number of pointers            = 10742
  total number of arrays              = 1491
  total number of func types          = 4137
  total number of structs             = 11327
  total number of unions              = 625
  total number of enums               = 650
  total number of forward tags        = 38
  total number of typedefs            = 5410
  total number of volatile types      = 45
  total number of const types         = 541
  total number of restrict types      = 0
  total number of unknowns (holes)    = 109

  total number of struct members      = 89882
  maximum number of struct members    = 248
  total size of all structs           = 14260270
  maximum size of a struct            = 2968576
  average number of struct members    = 7.94
  average size of a struct            = 1258.96

  total number of union members       = 2096
  maximum number of union members     = 33
  total size of all unions            = 94697
  maximum size of a union             = 14280
  average number of union members     = 3.35
  average size of a union             = 151.52

  total number of enum members        = 6933
  maximum number of enum members      = 1023
  average number of enum members      = 10.67

  total number of unique strings      = 54992
  bytes of string data                = 779435
  maximum string length               = 81
  average string length               = 14.17

Comment 2 Sean Bruno freebsd_committer

2015-07-16 17:03:08 UTC

Removing all sound, wireless and raid controllers lowers the number of symbols for CTF down to an amount that allows dtrace scripts to run again.

Comment 3 Mark Johnston freebsd_committer

2015-07-16 17:05:50 UTC

The problem is that we're running into CTF's typeid limit with amd64 GENERIC now.

Comment 4 Mark Johnston freebsd_committer

2015-07-24 23:08:57 UTC

I'm somewhat confused: when building a GENERIC amd64 kernel, I get substantially fewer types than you:

markj@wkstn-mjohnston: /usr/obj/usr/home/markj/src/freebsd-dev/sys/GENERIC $ ctfdump -S kernel
...
  total number of types               = 30904
...

In particular, this is below the typeid limit, though it is still quite close.

It looks like I really need to reapply r274569. It was reverted in r276848 because it was causing a crash in i386 builds, but I never really got to the bottom of that. However, it fixes a large amount of incorrect duplication in the type graph; a GENERIC kernel build with this change reapplied has substantially fewer types:

markj@wkstn-mjohnston: /usr/obj/usr/home/markj/src/freebsd-dev/sys/GENERIC $ ctfdump -S kernel
...
  total number of types               = 24831
...

I'm working on figuring out the root cause of the crash. Once that's done, I'm going to re-commit r274569, and that will handle the problem for a while without any need to change the CTF format.

Comment 5 Sean Bruno freebsd_committer

2017-01-03 17:40:29 UTC

seems to work fine here on my laptop today.  Thanks!