When attempting to compile the dtrace kernel module for mips64, it fails to build: --- dtrace_asm.o --- cc -isystem /usr/obj/usr/src/tmp/usr/include -L/usr/obj/usr/src/tmp/usr/lib -B/usr/obj/usr/src/tmp/usr/lib --sysroot=/usr/obj/usr/src/tmp -B/usr/obj/usr/src/tmp/usr/bin -O -pipe -DDIS_MEM -DSMP -G0 -EB -mabi=64 -msoft-float -Werror -D_KERNEL -DKLD_MODULE -nostdinc -I/usr/src/sys/cddl/compat/opensolaris -I/usr/src/sys/cddl/dev/dtrace -I/usr/src/sys/cddl/dev/dtrace/mips -I/usr/src/sys/cddl/contrib/opensolaris/uts/common -I/usr/src/sys/cddl/contrib/opensolaris/uts/common/dtrace -I/usr/src/sys/cddl/contrib/opensolaris/common/util -I/usr/src/sys -DHAVE_KERNEL_OPTION_HEADERS -include /usr/obj/usr/src/sys/ERL2/opt_global.h -I. -I/usr/src/sys -fno-common -g -G0 -fno-pic -mno-abicalls -mlong-calls -I/usr/obj/usr/src/sys/ERL2 -MD -MF.depend.dtrace_asm.o -MTdtrace_asm.o -msoft-float -ffreestanding -fwrapv -gdwarf-2 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Winline -Wcast-qual -Wundef -Wno-pointer-sign -fformat-extensions -Wmissing-include-dirs -fdiagnostics-show-option -Wno-unknown-pragmas -Wno-uninitialized -Wno-parentheses -Wno-uninitialized -Wno-cast-qual -Wno-unused -finline-limit=8000 -fms-extensions --param inline-unit-growth=100 --param large-function-growth=1000 -std=iso9899:1999 -include /usr/src/sys/cddl/compat/opensolaris/sys/debug_compat.h -c /usr/src/sys/cddl/dev/dtrace/mips/dtrace_asm.S -o dtrace_asm.o In file included from /usr/src/sys/cddl/dev/dtrace/mips/dtrace_asm.S:40: ./assym.s:61:1: error: "MIPS_KSEG0_START" redefined In file included from /usr/src/sys/cddl/dev/dtrace/mips/dtrace_asm.S:37: ./machine/cpuregs.h:80:1: error: this is the location of the previous definition In file included from /usr/src/sys/cddl/dev/dtrace/mips/dtrace_asm.S:40: ./assym.s:62:1: error: "MIPS_KSEG1_START" redefined In file included from /usr/src/sys/cddl/dev/dtrace/mips/dtrace_asm.S:37: ./machine/cpuregs.h:82:1: error: this is the location of the previous definition In file included from /usr/src/sys/cddl/dev/dtrace/mips/dtrace_asm.S:40: ./assym.s:63:1: error: "MIPS_KSEG2_START" redefined In file included from /usr/src/sys/cddl/dev/dtrace/mips/dtrace_asm.S:37: ./machine/cpuregs.h:88:1: error: this is the location of the previous definition *** [dtrace_asm.o] Error code 1 make[5]: stopped in /usr/src/sys/modules/dtrace/dtrace 1 error
Kurt: What kernconf or what modification did you make to enable the build of dtrace_asm.o ? I just did a mips.mips64 buildworld and an ERL kernel build and I don't have a dtrace.ko module at all.
The kernel configuration file I used is just the ERL file, plus a custom MODULES_OVERRIDE setting. If I remove the "dtrace" from the MODULES_OVERRIDE, I can compile the kernel configuration and the kernel works fine on my ERL. It has the advantage of being significantly smaller than the stock ERL kernel installation. root@erl2-148: diff sys/mips/conf/ERL sys/mips/conf/ERL2 22c22 < ident ERL --- > ident ERL2 27a28,36 > > makeoptions MODULES_OVERRIDE="acl_nfs4 autofs bridgestp cam \ > dtrace dummynet ext2fs fdescfs filemon geom \ > if_bridge if_gif if_gre if_tap if_tun if_vlan \ > ipfw ipsec iscsi iscsi_initiator \ > libalias md msdosfs msdosfs_iconv netgraph \ > nfscl nfscommon nfsd nfslock nfslockd nfssvc \ > nullfs pf pflog pfsync procfs pseudofs \ > smbfs tmpfs ufs unionfs usb zlib"
This seems enough to get dtrace to compile for mips64. Want to see if it actually works? Index: sys/cddl/dev/dtrace/mips/dtrace_asm.S =================================================================== --- sys/cddl/dev/dtrace/mips/dtrace_asm.S (revision 322438) +++ sys/cddl/dev/dtrace/mips/dtrace_asm.S (working copy) @@ -37,7 +37,6 @@ #include <machine/cpuregs.h> #include <machine/regnum.h> -#include "assym.s" .set noreorder # Noreorder is default style! Index: sys/cddl/dev/dtrace/mips/dtrace_subr.c =================================================================== --- sys/cddl/dev/dtrace/mips/dtrace_subr.c (revision 322438) +++ sys/cddl/dev/dtrace/mips/dtrace_subr.c (working copy) @@ -50,6 +50,8 @@ extern dtrace_id_t dtrace_probeid_error; int dtrace_invop(uintptr_t, struct trapframe *, uintptr_t); +void dtrace_invop_init(void); +void dtrace_invop_uninit(void); typedef struct dtrace_invop_hdlr { int (*dtih_func)(uintptr_t, struct trapframe *, uintptr_t);
The following diff is also needed: diff --git a/sys/modules/dtrace/Makefile b/sys/modules/dtrace/Makefile index 8ecaefe21b7..3837978031f 100644 --- a/sys/modules/dtrace/Makefile +++ b/sys/modules/dtrace/Makefile @@ -22,7 +22,7 @@ SUBDIR+= systrace_linux32 .if ${MACHINE_CPUARCH} == "powerpc" SUBDIR+= fbt fasttrap .endif -.if ${MACHINE_CPUARCH} == "amd64" || ${MACHINE_ARCH} == "powerpc64" +.if ${MACHINE_CPUARCH} == "amd64" || ${MACHINE_ARCH} == "powerpc64" || ${MACHINE_ARCH} == "mips64" SUBDIR+= systrace_freebsd32 .endif .if ${MACHINE_CPUARCH} == "aarch64" || \
Here's the ultimate diff between the stock ERL kernel config and the one that will run 'dtrace -l' successfully. Both your patch and my patch are needed as well: root@erl2-154: diff -w ERL ERL2 22c22 < ident ERL --- > ident ERL2 28a29,37 > makeoptions MODULES_OVERRIDE="acl_nfs4 autofs bridgestp cam \ > dtrace dummynet ext2fs fdescfs filemon geom \ > if_bridge if_gif if_gre if_tap if_tun if_vlan \ > ipfw ipsec iscsi iscsi_initiator \ > libalias md msdosfs msdosfs_iconv netgraph \ > nfscl nfscommon nfsd nfslock nfslockd nfssvc \ > nullfs opensolaris pf pflog pfsync procfs pseudofs \ > smbfs tmpfs ufs unionfs usb zlib" > 89,90c98,100 < #options KDTRACE_FRAME # Ensure frames are compiled in < #options KDTRACE_HOOKS # Kernel DTrace hooks --- > options KDTRACE_FRAME # Ensure frames are compiled in > options KDTRACE_HOOKS # Kernel DTrace hooks > options DDB_CTF # Kernel ELF linker loads CTF data 96c106 < #options DDB # Support DDB. --- > options DDB # Support DDB. Note the addition of 'opensolaris' to the modules line, and the inclusion of: KDTRACE_FRAME, KDTRACE_HOOKS, DDB_CTF, and DDB options. I don't know that DDB_CTF is needed, but it seemed to be on in the amd64 config file.
This is good enough to have 'dtrace -l' emit a list of providers in the kernel. None of the scripts in /usr/share/dtrace function, so something is still lacking. root@erl2-158: cd /usr/share/dtrace root@erl2-159: ./blocking dtrace: failed to compile script ./blocking: line 40: macro argument $$1 is not defined root@erl2-160: ./udptrack dtrace: failed to compile script ./udptrack: line 39: failed to resolve translated type for args[4] root@erl2-161: ./tcptrack dtrace: failed to compile script ./tcptrack: line 40: failed to resolve translated type for args[2]
+dtrace warriors. This is now a bit beyond my knowledge.
One thing I just noticed. The number of probe points on the mips64 machine is much, much smaller than on a amd64 machine. root@erl2-152: uname -a FreeBSD erl2.pix.net 12.0-CURRENT FreeBSD 12.0-CURRENT #8 5bec095fcbe(master)-dirty: Fri Aug 18 15:32:05 UTC 2017 root@erl2.pix.net:/usr/obj/usr/src/sys/ERL2 mips root@erl2-153: dtrace -l | wc 3488 16828 265624 root@hydra-129: uname -a FreeBSD hydra.pix.net 11.1-RELEASE-p1 FreeBSD 11.1-RELEASE-p1 #0: Wed Aug 9 11:55:48 UTC 2017 root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC amd64 root@hydra-130: dtrace -l | wc 66421 331245 5040083 I know those are not apples-to-apples comparison, but there's more than an order of magnitude difference here...
Created attachment 185890 [details] output from 'dtrace -l' on mips64 host
Robert: Do you have someone working on MIPS dtrace at the moment?
A commit references this bug: Author: lidl Date: Wed Sep 6 03:19:53 UTC 2017 New revision: 323206 URL: https://svnweb.freebsd.org/changeset/base/323206 Log: Enable dtrace support for mips64 and the ERL kernel config Turn on the required options in the ERL config file, and ensure that the fbt module is listed as a dependency for mips in the modules/dtrace/dtraceall/dtraceall.c file. PR: 220346 Reviewed by: gnn, markj MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D12227 Changes: head/sys/cddl/dev/dtrace/mips/dtrace_asm.S head/sys/cddl/dev/dtrace/mips/dtrace_subr.c head/sys/cddl/dev/fbt/mips/fbt_isa.c head/sys/mips/conf/ERL head/sys/modules/dtrace/Makefile head/sys/modules/dtrace/dtraceall/Makefile head/sys/modules/dtrace/dtraceall/dtraceall.c