There is a new commit on HEAD that broke the loader on powerpc64 on KVM. I tested on both Power8 and Power9. This break the loader once the kernel is loaded: Loading /boot/defaults/loader.conf /boot/kernel/kernel data=0x11ef978+0x4c6e48 syms=[0x8+0x1684e0panic: zfree(0x183c0c0,4224): corrupt memlist2 --> Press a key on the console to reboot <-- Since this crash is in the loader, it break the whole machine, and I am not able to load the old kernel. This is the current kernel I built: /boot ls -la kernel total 84388 drwxr-xr-x 2 root wheel 15872 Jan 19 19:30 . drwxr-xr-x 11 root wheel 1024 Jan 19 19:30 .. -r-xr-xr-x 1 root wheel 7968 Jan 19 19:29 accf_data.ko -r-xr-xr-x 1 root wheel 9208 Jan 19 19:29 accf_dns.ko -r-xr-xr-x 1 root wheel 14032 Jan 19 19:29 accf_http.ko -r-xr-xr-x 1 root wheel 21728 Jan 19 19:29 acl_nfs4.ko -r-xr-xr-x 1 root wheel 13256 Jan 19 19:29 acl_posix1e.ko -r-xr-xr-x 1 root wheel 63448 Jan 19 19:29 agp.ko -r-xr-xr-x 1 root wheel 62304 Jan 19 19:29 aha.ko -r-xr-xr-x 1 root wheel 210952 Jan 19 19:29 ahc.ko -r-xr-xr-x 1 root wheel 29608 Jan 19 19:29 ahc_isa.ko -r-xr-xr-x 1 root wheel 73472 Jan 19 19:29 ahc_pci.ko -r-xr-xr-x 1 root wheel 136064 Jan 19 19:29 ahci.ko -r-xr-xr-x 1 root wheel 304160 Jan 19 19:29 ahd.ko -r-xr-xr-x 1 root wheel 16144 Jan 19 19:29 alias_cuseeme.ko -r-xr-xr-x 1 root wheel 14904 Jan 19 19:29 alias_dummy.ko -r-xr-xr-x 1 root wheel 22696 Jan 19 19:29 alias_ftp.ko -r-xr-xr-x 1 root wheel 19024 Jan 19 19:29 alias_irc.ko -r-xr-xr-x 1 root wheel 19760 Jan 19 19:29 alias_nbt.ko -r-xr-xr-x 1 root wheel 20464 Jan 19 19:29 alias_pptp.ko -r-xr-xr-x 1 root wheel 16440 Jan 19 19:29 alias_skinny.ko -r-xr-xr-x 1 root wheel 22224 Jan 19 19:29 alias_smedia.ko -r-xr-xr-x 1 root wheel 28408 Jan 19 19:29 alpm.ko -r-xr-xr-x 1 root wheel 41616 Jan 19 19:29 alq.ko -r-xr-xr-x 1 root wheel 28304 Jan 19 19:29 amdpm.ko -r-xr-xr-x 1 root wheel 25824 Jan 19 19:29 amdsmb.ko -r-xr-xr-x 1 root wheel 82648 Jan 19 19:29 amr.ko -r-xr-xr-x 1 root wheel 30080 Jan 19 19:29 amr_cam.ko -r-xr-xr-x 1 root wheel 81504 Jan 19 19:29 ata.ko -r-xr-xr-x 1 root wheel 22272 Jan 19 19:29 ataacard.ko -r-xr-xr-x 1 root wheel 28136 Jan 19 19:29 ataacerlabs.ko -r-xr-xr-x 1 root wheel 21000 Jan 19 19:29 ataamd.ko -r-xr-xr-x 1 root wheel 25768 Jan 19 19:29 ataati.ko -r-xr-xr-x 1 root wheel 16464 Jan 19 19:29 atacard.ko -r-xr-xr-x 1 root wheel 15336 Jan 19 19:29 atacenatek.ko -r-xr-xr-x 1 root wheel 18560 Jan 19 19:29 atacypress.ko -r-xr-xr-x 1 root wheel 19104 Jan 19 19:29 atacyrix.ko -r-xr-xr-x 1 root wheel 25360 Jan 19 19:29 atahighpoint.ko -r-xr-xr-x 1 root wheel 45408 Jan 19 19:29 ataintel.ko -r-xr-xr-x 1 root wheel 16080 Jan 19 19:29 ataisa.ko -r-xr-xr-x 1 root wheel 25128 Jan 19 19:29 ataite.ko -r-xr-xr-x 1 root wheel 21808 Jan 19 19:29 atajmicron.ko -r-xr-xr-x 1 root wheel 21136 Jan 19 19:29 atamarvell.ko -r-xr-xr-x 1 root wheel 15560 Jan 19 19:29 atamicron.ko -r-xr-xr-x 1 root wheel 20768 Jan 19 19:29 atanational.ko -r-xr-xr-x 1 root wheel 18336 Jan 19 19:29 atanetcell.ko -r-xr-xr-x 1 root wheel 29704 Jan 19 19:29 atanvidia.ko -r-xr-xr-x 1 root wheel 67272 Jan 19 19:29 atapci.ko -r-xr-xr-x 1 root wheel 55008 Jan 19 19:29 atapromise.ko -r-xr-xr-x 1 root wheel 30376 Jan 19 19:29 ataserverworks.ko -r-xr-xr-x 1 root wheel 30848 Jan 19 19:29 atasiliconimage.ko -r-xr-xr-x 1 root wheel 29832 Jan 19 19:29 atasis.ko -r-xr-xr-x 1 root wheel 35696 Jan 19 19:29 atavia.ko -r-xr-xr-x 1 root wheel 48784 Jan 19 19:29 ath_dfs.ko -r-xr-xr-x 1 root wheel 123056 Jan 19 19:29 ath_hal.ko -r-xr-xr-x 1 root wheel 96792 Jan 19 19:29 ath_hal_ar5210.ko -r-xr-xr-x 1 root wheel 115352 Jan 19 19:29 ath_hal_ar5211.ko -r-xr-xr-x 1 root wheel 246416 Jan 19 19:29 ath_hal_ar5212.ko -r-xr-xr-x 1 root wheel 344640 Jan 19 19:29 ath_hal_ar5416.ko -r-xr-xr-x 1 root wheel 583808 Jan 19 19:29 ath_hal_ar9300.ko -r-xr-xr-x 1 root wheel 363416 Jan 19 19:29 ath_main.ko -r-xr-xr-x 1 root wheel 66224 Jan 19 19:29 ath_rate.ko -r-xr-xr-x 1 root wheel 53776 Jan 19 19:30 atp.ko -r-xr-xr-x 1 root wheel 75880 Jan 19 19:29 autofs.ko -r-xr-xr-x 1 root wheel 69400 Jan 19 19:29 bcma.ko -r-xr-xr-x 1 root wheel 15488 Jan 19 19:29 bcma_bhndb.ko -r-xr-xr-x 1 root wheel 636320 Jan 19 19:29 bhnd.ko -r-xr-xr-x 1 root wheel 40152 Jan 19 19:29 bhnd_pci.ko -r-xr-xr-x 1 root wheel 31512 Jan 19 19:29 bhnd_pci_hostb.ko -r-xr-xr-x 1 root wheel 15264 Jan 19 19:29 bhnd_pcib.ko -r-xr-xr-x 1 root wheel 158984 Jan 19 19:29 bhndb.ko -r-xr-xr-x 1 root wheel 57872 Jan 19 19:29 bhndb_pci.ko -r-xr-xr-x 1 root wheel 57864 Jan 19 19:29 bridgestp.ko -r-xr-xr-x 1 root wheel 22440 Jan 19 19:30 callout_test.ko -r-xr-xr-x 1 root wheel 1123064 Jan 19 19:29 cam.ko -r-xr-xr-x 1 root wheel 44440 Jan 19 19:29 cardbus.ko -r-xr-xr-x 1 root wheel 88696 Jan 19 19:29 carp.ko -r-xr-xr-x 1 root wheel 96944 Jan 19 19:29 cbb.ko -r-xr-xr-x 1 root wheel 36936 Jan 19 19:29 cc_cdg.ko -r-xr-xr-x 1 root wheel 28744 Jan 19 19:29 cc_chd.ko -r-xr-xr-x 1 root wheel 22632 Jan 19 19:29 cc_cubic.ko -r-xr-xr-x 1 root wheel 26192 Jan 19 19:29 cc_dctcp.ko -r-xr-xr-x 1 root wheel 23552 Jan 19 19:29 cc_hd.ko -r-xr-xr-x 1 root wheel 25376 Jan 19 19:29 cc_htcp.ko -r-xr-xr-x 1 root wheel 24952 Jan 19 19:29 cc_vegas.ko -r-xr-xr-x 1 root wheel 63904 Jan 19 19:29 ccr.ko -r-xr-xr-x 1 root wheel 87272 Jan 19 19:29 cd9660.ko -r-xr-xr-x 1 root wheel 8080 Jan 19 19:29 cd9660_iconv.ko -r-xr-xr-x 1 root wheel 45192 Jan 19 19:29 cfi.ko -r-xr-xr-x 1 root wheel 76016 Jan 19 19:29 cfiscsi.ko -r-xr-xr-x 1 root wheel 48560 Jan 19 19:30 cfumass.ko -r-xr-xr-x 1 root wheel 10328 Jan 19 19:29 chacha20.ko -r-xr-xr-x 1 root wheel 95560 Jan 19 19:29 cloudabi.ko -r-xr-xr-x 1 root wheel 24552 Jan 19 19:29 cpufreq.ko -r-xr-xr-x 1 root wheel 310072 Jan 19 19:29 crypto.ko -r-xr-xr-x 1 root wheel 44712 Jan 19 19:29 cryptodev.ko -r-xr-xr-x 1 root wheel 539528 Jan 19 19:29 ctl.ko -r-xr-xr-x 1 root wheel 62080 Jan 19 19:29 cuse.ko -r-xr-xr-x 1 root wheel 55864 Jan 19 19:29 cxgb_t3fw.ko -r-xr-xr-x 1 root wheel 48656 Jan 19 19:29 cyapa.ko -r-xr-xr-x 1 root wheel 32056 Jan 19 19:29 dcons.ko -r-xr-xr-x 1 root wheel 22664 Jan 19 19:29 dcons_crom.ko -r-xr-xr-x 1 root wheel 196920 Jan 19 19:29 drm.ko -r-xr-xr-x 1 root wheel 621832 Jan 19 19:29 drm2.ko -r-xr-xr-x 1 root wheel 20096 Jan 19 19:29 ds1307.ko -r-xr-xr-x 1 root wheel 18960 Jan 19 19:29 ds13rtc.ko -r-xr-xr-x 1 root wheel 24904 Jan 19 19:29 ds3231.ko -r-xr-xr-x 1 root wheel 35992 Jan 19 19:29 dtaudit.ko -r-xr-xr-x 1 root wheel 30432 Jan 19 19:29 dtmalloc.ko -r-xr-xr-x 1 root wheel 42568 Jan 19 19:29 dtnfscl.ko -r-xr-xr-x 1 root wheel 313256 Jan 19 19:29 dtrace.ko -r-xr-xr-x 1 root wheel 20392 Jan 19 19:29 dtrace_test.ko -r-xr-xr-x 1 root wheel 9280 Jan 19 19:29 dtraceall.ko -r-xr-xr-x 1 root wheel 198040 Jan 19 19:29 dummynet.ko -r-xr-xr-x 1 root wheel 108400 Jan 19 19:30 ehci.ko -r-xr-xr-x 1 root wheel 83888 Jan 19 19:29 esp.ko -r-xr-xr-x 1 root wheel 67960 Jan 19 19:29 evdev.ko -r-xr-xr-x 1 root wheel 23656 Jan 19 19:29 exca.ko -r-xr-xr-x 1 root wheel 238336 Jan 19 19:29 ext2fs.ko -r-xr-xr-x 1 root wheel 407608 Jan 19 19:29 fasttrap.ko -r-xr-xr-x 1 root wheel 47632 Jan 19 19:29 fbt.ko -r-xr-xr-x 1 root wheel 83224 Jan 19 19:29 fdc.ko -r-xr-xr-x 1 root wheel 44232 Jan 19 19:29 fdescfs.ko -r-xr-xr-x 1 root wheel 48168 Jan 19 19:29 filemon.ko -r-xr-xr-x 1 root wheel 180832 Jan 19 19:29 firewire.ko -r-xr-xr-x 1 root wheel 31344 Jan 19 19:29 firmware.ko -r-xr-xr-x 1 root wheel 142488 Jan 19 19:29 fuse.ko -r-xr-xr-x 1 root wheel 25864 Jan 19 19:30 g_audio.ko -r-xr-xr-x 1 root wheel 22576 Jan 19 19:30 g_keyboard.ko -r-xr-xr-x 1 root wheel 25720 Jan 19 19:30 g_modem.ko -r-xr-xr-x 1 root wheel 25520 Jan 19 19:30 g_mouse.ko -r-xr-xr-x 1 root wheel 102192 Jan 19 19:29 geom_bde.ko -r-xr-xr-x 1 root wheel 50128 Jan 19 19:29 geom_cache.ko -r-xr-xr-x 1 root wheel 24312 Jan 19 19:29 geom_ccd.ko -r-xr-xr-x 1 root wheel 43912 Jan 19 19:29 geom_concat.ko -r-xr-xr-x 1 root wheel 155448 Jan 19 19:29 geom_eli.ko -r-xr-xr-x 1 root wheel 13488 Jan 19 19:29 geom_flashmap.ko -r-xr-xr-x 1 root wheel 50408 Jan 19 19:29 geom_gate.ko -r-xr-xr-x 1 root wheel 136000 Jan 19 19:29 geom_journal.ko -r-xr-xr-x 1 root wheel 61928 Jan 19 19:29 geom_label.ko -r-xr-xr-x 1 root wheel 47248 Jan 19 19:29 geom_linux_lvm.ko -r-xr-xr-x 1 root wheel 77848 Jan 19 19:29 geom_md.ko -r-xr-xr-x 1 root wheel 170232 Jan 19 19:29 geom_mirror.ko -r-xr-xr-x 1 root wheel 44704 Jan 19 19:29 geom_mountver.ko -r-xr-xr-x 1 root wheel 53520 Jan 19 19:29 geom_multipath.ko -r-xr-xr-x 1 root wheel 33160 Jan 19 19:29 geom_nop.ko -r-xr-xr-x 1 root wheel 27328 Jan 19 19:29 geom_part_apm.ko -r-xr-xr-x 1 root wheel 28792 Jan 19 19:29 geom_part_bsd.ko -r-xr-xr-x 1 root wheel 29328 Jan 19 19:29 geom_part_bsd64.ko -r-xr-xr-x 1 root wheel 26744 Jan 19 19:29 geom_part_ebr.ko -r-xr-xr-x 1 root wheel 48080 Jan 19 19:29 geom_part_gpt.ko -r-xr-xr-x 1 root wheel 42280 Jan 19 19:29 geom_part_ldm.ko -r-xr-xr-x 1 root wheel 26696 Jan 19 19:29 geom_part_mbr.ko -r-xr-xr-x 1 root wheel 26416 Jan 19 19:29 geom_part_vtoc8.ko -r-xr-xr-x 1 root wheel 496528 Jan 19 19:29 geom_raid.ko -r-xr-xr-x 1 root wheel 152304 Jan 19 19:29 geom_raid3.ko -r-xr-xr-x 1 root wheel 61192 Jan 19 19:29 geom_sched.ko -r-xr-xr-x 1 root wheel 39728 Jan 19 19:29 geom_shsec.ko -r-xr-xr-x 1 root wheel 49568 Jan 19 19:29 geom_stripe.ko -r-xr-xr-x 1 root wheel 70856 Jan 19 19:29 geom_uzip.ko -r-xr-xr-x 1 root wheel 215384 Jan 19 19:29 geom_vinum.ko -r-xr-xr-x 1 root wheel 78488 Jan 19 19:29 geom_virstor.ko -r-xr-xr-x 1 root wheel 14456 Jan 19 19:29 geom_zero.ko -r-xr-xr-x 1 root wheel 74016 Jan 19 19:29 gpiobus.ko -r-xr-xr-x 1 root wheel 18352 Jan 19 19:29 gpioiic.ko -r-xr-xr-x 1 root wheel 54888 Jan 19 19:29 gpiokeys.ko -r-xr-xr-x 1 root wheel 14960 Jan 19 19:29 gpioled.ko -r-xr-xr-x 1 root wheel 18960 Jan 19 19:29 gpiopps.ko -r-xr-xr-x 1 root wheel 21344 Jan 19 19:29 gpiospi.ko -r-xr-xr-x 1 root wheel 23624 Jan 19 19:29 gsched_delay.ko -r-xr-xr-x 1 root wheel 32544 Jan 19 19:29 gsched_rr.ko -r-xr-xr-x 1 root wheel 24096 Jan 19 19:29 h_ertt.ko -r-xr-xr-x 1 root wheel 74840 Jan 19 19:29 hifn.ko -r-xr-xr-x 1 root wheel 180832 Jan 19 19:29 hwpmc.ko -r-xr-xr-x 1 root wheel 18312 Jan 19 19:29 icee.ko -r-xr-xr-x 1 root wheel 34504 Jan 19 19:29 ichsmb.ko -r-xr-xr-x 1 root wheel 74392 Jan 19 19:29 if_ae.ko -r-xr-xr-x 1 root wheel 98280 Jan 19 19:29 if_age.ko -r-xr-xr-x 1 root wheel 116392 Jan 19 19:29 if_alc.ko -r-xr-xr-x 1 root wheel 93648 Jan 19 19:29 if_ale.ko -r-xr-xr-x 1 root wheel 118688 Jan 19 19:29 if_an.ko -r-xr-xr-x 1 root wheel 16376 Jan 19 19:29 if_ath.ko -r-xr-xr-x 1 root wheel 57960 Jan 19 19:29 if_ath_pci.ko -r-xr-xr-x 1 root wheel 52048 Jan 19 19:30 if_aue.ko -r-xr-xr-x 1 root wheel 62000 Jan 19 19:30 if_axe.ko -r-xr-xr-x 1 root wheel 53536 Jan 19 19:30 if_axge.ko -r-xr-xr-x 1 root wheel 379568 Jan 19 19:29 if_bce.ko -r-xr-xr-x 1 root wheel 69656 Jan 19 19:29 if_bfe.ko -r-xr-xr-x 1 root wheel 160344 Jan 19 19:29 if_bge.ko -r-xr-xr-x 1 root wheel 55856 Jan 19 19:29 if_bm.ko -r-xr-xr-x 1 root wheel 208168 Jan 19 19:29 if_bnxt.ko -r-xr-xr-x 1 root wheel 98544 Jan 19 19:29 if_bridge.ko -r-xr-xr-x 1 root wheel 229328 Jan 19 19:29 if_bwi.ko -r-xr-xr-x 1 root wheel 564288 Jan 19 19:29 if_bwn.ko -r-xr-xr-x 1 root wheel 22824 Jan 19 19:29 if_bwn_pci.ko -r-xr-xr-x 1 root wheel 83672 Jan 19 19:29 if_cas.ko -r-xr-xr-x 1 root wheel 6224 Jan 19 19:29 if_cc.ko -r-xr-xr-x 1 root wheel 6232 Jan 19 19:29 if_ccv.ko -r-xr-xr-x 1 root wheel 62752 Jan 19 19:30 if_cdce.ko -r-xr-xr-x 1 root wheel 41312 Jan 19 19:30 if_cue.ko -r-xr-xr-x 1 root wheel 365360 Jan 19 19:29 if_cxgb.ko -r-xr-xr-x 1 root wheel 731352 Jan 19 19:29 if_cxgbe.ko -r-xr-xr-x 1 root wheel 65992 Jan 19 19:29 if_cxgbev.ko -r-xr-xr-x 1 root wheel 6232 Jan 19 19:29 if_cxl.ko -r-xr-xr-x 1 root wheel 6256 Jan 19 19:29 if_cxlv.ko -r-xr-xr-x 1 root wheel 114704 Jan 19 19:29 if_dc.ko -r-xr-xr-x 1 root wheel 115768 Jan 19 19:29 if_de.ko -r-xr-xr-x 1 root wheel 24648 Jan 19 19:29 if_disc.ko -r-xr-xr-x 1 root wheel 26840 Jan 19 19:29 if_edsc.ko -r-xr-xr-x 1 root wheel 35432 Jan 19 19:29 if_enc.ko -r-xr-xr-x 1 root wheel 45744 Jan 19 19:29 if_epair.ko -r-xr-xr-x 1 root wheel 59152 Jan 19 19:29 if_ffec.ko -r-xr-xr-x 1 root wheel 38592 Jan 19 19:29 if_fwe.ko -r-xr-xr-x 1 root wheel 59664 Jan 19 19:29 if_fwip.ko -r-xr-xr-x 1 root wheel 104824 Jan 19 19:29 if_fxp.ko -r-xr-xr-x 1 root wheel 78472 Jan 19 19:29 if_gem.ko -r-xr-xr-x 1 root wheel 59312 Jan 19 19:29 if_gif.ko -r-xr-xr-x 1 root wheel 55016 Jan 19 19:29 if_gre.ko -r-xr-xr-x 1 root wheel 64536 Jan 19 19:29 if_hme.ko -r-xr-xr-x 1 root wheel 29272 Jan 19 19:29 if_ic.ko -r-xr-xr-x 1 root wheel 37904 Jan 19 19:30 if_ipheth.ko -r-xr-xr-x 1 root wheel 97512 Jan 19 19:29 if_jme.ko -r-xr-xr-x 1 root wheel 46432 Jan 19 19:30 if_kue.ko -r-xr-xr-x 1 root wheel 115264 Jan 19 19:29 if_lagg.ko -r-xr-xr-x 1 root wheel 67400 Jan 19 19:29 if_le.ko -r-xr-xr-x 1 root wheel 55080 Jan 19 19:29 if_lge.ko -r-xr-xr-x 1 root wheel 119720 Jan 19 19:29 if_lmc.ko -r-xr-xr-x 1 root wheel 110808 Jan 19 19:29 if_malo.ko -r-xr-xr-x 1 root wheel 42456 Jan 19 19:29 if_me.ko -r-xr-xr-x 1 root wheel 52400 Jan 19 19:30 if_mos.ko -r-xr-xr-x 1 root wheel 126160 Jan 19 19:29 if_msk.ko -r-xr-xr-x 1 root wheel 196232 Jan 19 19:29 if_mwl.ko -r-xr-xr-x 1 root wheel 123248 Jan 19 19:29 if_mxge.ko -r-xr-xr-x 1 root wheel 62192 Jan 19 19:29 if_my.ko -r-xr-xr-x 1 root wheel 78376 Jan 19 19:30 if_nge.ko -r-xr-xr-x 1 root wheel 205528 Jan 19 19:30 if_oce.ko -r-xr-xr-x 1 root wheel 106336 Jan 19 19:30 if_otus.ko -r-xr-xr-x 1 root wheel 55280 Jan 19 19:30 if_pcn.ko -r-xr-xr-x 1 root wheel 229904 Jan 19 19:30 if_ral.ko -r-xr-xr-x 1 root wheel 97176 Jan 19 19:30 if_re.ko -r-xr-xr-x 1 root wheel 67352 Jan 19 19:30 if_rl.ko -r-xr-xr-x 1 root wheel 119176 Jan 19 19:30 if_rsu.ko -r-xr-xr-x 1 root wheel 106624 Jan 19 19:30 if_rtwn_pci.ko -r-xr-xr-x 1 root wheel 184264 Jan 19 19:30 if_rtwn_usb.ko -r-xr-xr-x 1 root wheel 54728 Jan 19 19:30 if_rue.ko -r-xr-xr-x 1 root wheel 113712 Jan 19 19:30 if_rum.ko -r-xr-xr-x 1 root wheel 166312 Jan 19 19:30 if_run.ko -r-xr-xr-x 1 root wheel 78592 Jan 19 19:30 if_sf.ko -r-xr-xr-x 1 root wheel 61496 Jan 19 19:30 if_sge.ko -r-xr-xr-x 1 root wheel 70400 Jan 19 19:30 if_sis.ko -r-xr-xr-x 1 root wheel 96400 Jan 19 19:30 if_sk.ko -r-xr-xr-x 1 root wheel 63680 Jan 19 19:30 if_smsc.ko -r-xr-xr-x 1 root wheel 58840 Jan 19 19:30 if_sn.ko -r-xr-xr-x 1 root wheel 74104 Jan 19 19:30 if_ste.ko -r-xr-xr-x 1 root wheel 38088 Jan 19 19:29 if_stf.ko -r-xr-xr-x 1 root wheel 74736 Jan 19 19:30 if_stge.ko -r-xr-xr-x 1 root wheel 55960 Jan 19 19:29 if_tap.ko -r-xr-xr-x 1 root wheel 252072 Jan 19 19:30 if_ti.ko -r-xr-xr-x 1 root wheel 64664 Jan 19 19:30 if_tl.ko -r-xr-xr-x 1 root wheel 56328 Jan 19 19:29 if_tun.ko -r-xr-xr-x 1 root wheel 64832 Jan 19 19:30 if_tx.ko -r-xr-xr-x 1 root wheel 144040 Jan 19 19:30 if_txp.ko -r-xr-xr-x 1 root wheel 91936 Jan 19 19:30 if_uath.ko -r-xr-xr-x 1 root wheel 48176 Jan 19 19:30 if_udav.ko -r-xr-xr-x 1 root wheel 78192 Jan 19 19:30 if_upgt.ko -r-xr-xr-x 1 root wheel 87688 Jan 19 19:30 if_ural.ko -r-xr-xr-x 1 root wheel 57520 Jan 19 19:30 if_ure.ko -r-xr-xr-x 1 root wheel 53408 Jan 19 19:30 if_urndis.ko -r-xr-xr-x 1 root wheel 87632 Jan 19 19:30 if_vge.ko -r-xr-xr-x 1 root wheel 61856 Jan 19 19:29 if_vlan.ko -r-xr-xr-x 1 root wheel 78800 Jan 19 19:30 if_vr.ko -r-xr-xr-x 1 root wheel 70160 Jan 19 19:30 if_vte.ko -r-xr-xr-x 1 root wheel 49376 Jan 19 19:30 if_vx.ko -r-xr-xr-x 1 root wheel 86200 Jan 19 19:29 if_vxlan.ko -r-xr-xr-x 1 root wheel 56736 Jan 19 19:30 if_wb.ko -r-xr-xr-x 1 root wheel 112904 Jan 19 19:30 if_wi.ko -r-xr-xr-x 1 root wheel 93248 Jan 19 19:30 if_xl.ko -r-xr-xr-x 1 root wheel 119088 Jan 19 19:30 if_zyd.ko -r-xr-xr-x 1 root wheel 43392 Jan 19 19:29 ig4.ko -r-xr-xr-x 1 root wheel 23608 Jan 19 19:29 iic.ko -r-xr-xr-x 1 root wheel 26960 Jan 19 19:29 iicbb.ko -r-xr-xr-x 1 root wheel 37096 Jan 19 19:29 iicbus.ko -r-xr-xr-x 1 root wheel 21344 Jan 19 19:29 iicsmb.ko -r-xr-xr-x 1 root wheel 23776 Jan 19 19:29 imgact_binmisc.ko -r-xr-xr-x 1 root wheel 34800 Jan 19 19:29 intpm.ko -r-xr-xr-x 1 root wheel 53152 Jan 19 19:29 ip6_mroute.ko -r-xr-xr-x 1 root wheel 107408 Jan 19 19:29 ip_mroute.ko -r-xr-xr-x 1 root wheel 45672 Jan 19 19:29 ipdivert.ko -r-xr-xr-x 1 root wheel 338208 Jan 19 19:29 ipfw.ko -r-xr-xr-x 1 root wheel 52240 Jan 19 19:29 ipfw_nat.ko -r-xr-xr-x 1 root wheel 125288 Jan 19 19:29 ipfw_nat64.ko -r-xr-xr-x 1 root wheel 43608 Jan 19 19:29 ipfw_nptv6.ko -r-xr-xr-x 1 root wheel 24848 Jan 19 19:29 ipfw_pmod.ko -r-xr-xr-x 1 root wheel 1044856 Jan 19 19:29 ipl.ko -r-xr-xr-x 1 root wheel 192000 Jan 19 19:29 ipsec.ko -r-xr-xr-x 1 root wheel 131376 Jan 19 19:29 iscsi.ko -r-xr-xr-x 1 root wheel 127568 Jan 19 19:29 iscsi_initiator.ko -r-xr-xr-x 1 root wheel 17344 Jan 19 19:29 isl.ko -r-xr-xr-x 1 root wheel 16736 Jan 19 19:29 isl12xx.ko -r-xr-xr-x 1 root wheel 35544 Jan 19 19:29 ismt.ko -r-xr-xr-x 1 root wheel 334176 Jan 19 19:29 isp.ko -r-xr-xr-x 1 root wheel 29576 Jan 19 19:29 isp_1040.ko -r-xr-xr-x 1 root wheel 37936 Jan 19 19:29 isp_1080.ko -r-xr-xr-x 1 root wheel 34664 Jan 19 19:29 isp_12160.ko -r-xr-xr-x 1 root wheel 83376 Jan 19 19:29 isp_2100.ko -r-xr-xr-x 1 root wheel 91160 Jan 19 19:29 isp_2200.ko -r-xr-xr-x 1 root wheel 131984 Jan 19 19:29 isp_2300.ko -r-xr-xr-x 1 root wheel 142648 Jan 19 19:29 isp_2322.ko -r-xr-xr-x 1 root wheel 267224 Jan 19 19:29 isp_2400.ko -r-xr-xr-x 1 root wheel 281728 Jan 19 19:29 isp_2500.ko -r-xr-xr-x 1 root wheel 1050320 Jan 19 19:29 ispfw.ko -r-xr-xr-x 1 root wheel 13864 Jan 19 19:29 jedec_ts.ko -r-xr-xr-x 1 root wheel 17488 Jan 19 19:29 joy.ko -r-xr-xr-x 1 root wheel 48048 Jan 19 19:29 kbdmux.ko -r-xr-xr-x 1 root wheel 17656 Jan 19 19:30 kern_testfrwk.ko -r-xr-xr-x 1 root wheel 22390248 Jan 19 19:28 kernel -r-xr-xr-x 1 root wheel 140528 Jan 19 19:29 kgssapi.ko -r-xr-xr-x 1 root wheel 76208 Jan 19 19:29 kgssapi_krb5.ko -r-xr-xr-x 1 root wheel 228224 Jan 19 19:29 krpc.ko -r-xr-xr-x 1 root wheel 26328 Jan 19 19:29 ksyms.ko -r-xr-xr-x 1 root wheel 124744 Jan 19 19:29 libalias.ko -r-xr-xr-x 1 root wheel 49824 Jan 19 19:29 libiconv.ko -r-xr-xr-x 1 root wheel 21808 Jan 19 19:29 libmchain.ko -rw-r--r-- 1 root wheel 237264 Jan 19 19:30 linker.hints -r-xr-xr-x 1 root wheel 256856 Jan 19 19:29 linuxkpi.ko -r-xr-xr-x 1 root wheel 18440 Jan 19 19:29 lpbb.ko -r-xr-xr-x 1 root wheel 29912 Jan 19 19:29 lpt.ko -r-xr-xr-x 1 root wheel 129712 Jan 19 19:29 mac_biba.ko -r-xr-xr-x 1 root wheel 52176 Jan 19 19:29 mac_bsdextended.ko -r-xr-xr-x 1 root wheel 31920 Jan 19 19:29 mac_ifoff.ko -r-xr-xr-x 1 root wheel 115272 Jan 19 19:29 mac_lomac.ko -r-xr-xr-x 1 root wheel 125744 Jan 19 19:29 mac_mls.ko -r-xr-xr-x 1 root wheel 12312 Jan 19 19:29 mac_none.ko -r-xr-xr-x 1 root wheel 35136 Jan 19 19:29 mac_partition.ko -r-xr-xr-x 1 root wheel 37344 Jan 19 19:29 mac_portacl.ko -r-xr-xr-x 1 root wheel 31376 Jan 19 19:29 mac_seeotheruids.ko -r-xr-xr-x 1 root wheel 86808 Jan 19 19:29 mac_stub.ko -r-xr-xr-x 1 root wheel 324064 Jan 19 19:29 mac_test.ko -r-xr-xr-x 1 root wheel 105616 Jan 19 19:29 mach64.ko -r-xr-xr-x 1 root wheel 11880 Jan 19 19:29 mdio.ko -r-xr-xr-x 1 root wheel 25128 Jan 19 19:29 mem.ko -r-xr-xr-x 1 root wheel 162224 Jan 19 19:29 mfi.ko -r-xr-xr-x 1 root wheel 42776 Jan 19 19:29 mfip.ko -r-xr-xr-x 1 root wheel 111832 Jan 19 19:29 mga.ko -r-xr-xr-x 1 root wheel 289288 Jan 19 19:29 miibus.ko -r-xr-xr-x 1 root wheel 85680 Jan 19 19:29 mlx.ko -r-xr-xr-x 1 root wheel 78160 Jan 19 19:29 mmc.ko -r-xr-xr-x 1 root wheel 49504 Jan 19 19:29 mmcsd.ko -r-xr-xr-x 1 root wheel 368136 Jan 19 19:29 mpr.ko -r-xr-xr-x 1 root wheel 335016 Jan 19 19:29 mps.ko -r-xr-xr-x 1 root wheel 273528 Jan 19 19:29 mpt.ko -r-xr-xr-x 1 root wheel 95832 Jan 19 19:29 mqueuefs.ko -r-xr-xr-x 1 root wheel 181616 Jan 19 19:29 mrsas.ko -r-xr-xr-x 1 root wheel 130456 Jan 19 19:29 msdosfs.ko -r-xr-xr-x 1 root wheel 8104 Jan 19 19:29 msdosfs_iconv.ko -r-xr-xr-x 1 root wheel 93368 Jan 19 19:29 mvs.ko -r-xr-xr-x 1 root wheel 107064 Jan 19 19:29 mw88W8363fw.ko -r-xr-xr-x 1 root wheel 127784 Jan 19 19:29 mxge_eth_z8e.ko -r-xr-xr-x 1 root wheel 130928 Jan 19 19:29 mxge_ethp_z8e.ko -r-xr-xr-x 1 root wheel 159160 Jan 19 19:29 mxge_rss_eth_z8e.ko -r-xr-xr-x 1 root wheel 160080 Jan 19 19:29 mxge_rss_ethp_z8e.ko -r-xr-xr-x 1 root wheel 122560 Jan 19 19:30 netgraph.ko -r-xr-xr-x 1 root wheel 621528 Jan 19 19:30 nfscl.ko -r-xr-xr-x 1 root wheel 218552 Jan 19 19:30 nfscommon.ko -r-xr-xr-x 1 root wheel 435360 Jan 19 19:30 nfsd.ko -r-xr-xr-x 1 root wheel 36640 Jan 19 19:30 nfslock.ko -r-xr-xr-x 1 root wheel 170920 Jan 19 19:30 nfslockd.ko -r-xr-xr-x 1 root wheel 29312 Jan 19 19:29 nfsmb.ko -r-xr-xr-x 1 root wheel 16208 Jan 19 19:30 nfssvc.ko -r-xr-xr-x 1 root wheel 13496 Jan 19 19:30 ng_UI.ko -r-xr-xr-x 1 root wheel 23912 Jan 19 19:29 ng_async.ko -r-xr-xr-x 1 root wheel 14536 Jan 19 19:29 ng_atmllc.ko -r-xr-xr-x 1 root wheel 16168 Jan 19 19:30 ng_bluetooth.ko -r-xr-xr-x 1 root wheel 25600 Jan 19 19:30 ng_bpf.ko -r-xr-xr-x 1 root wheel 40016 Jan 19 19:30 ng_bridge.ko -r-xr-xr-x 1 root wheel 52280 Jan 19 19:30 ng_bt3c.ko -r-xr-xr-x 1 root wheel 250864 Jan 19 19:30 ng_btsocket.ko -r-xr-xr-x 1 root wheel 24192 Jan 19 19:30 ng_car.ko -r-xr-xr-x 1 root wheel 151920 Jan 19 19:29 ng_ccatm.ko -r-xr-xr-x 1 root wheel 28176 Jan 19 19:30 ng_cisco.ko -r-xr-xr-x 1 root wheel 24592 Jan 19 19:30 ng_deflate.ko -r-xr-xr-x 1 root wheel 36136 Jan 19 19:30 ng_device.ko -r-xr-xr-x 1 root wheel 10432 Jan 19 19:30 ng_echo.ko -r-xr-xr-x 1 root wheel 37784 Jan 19 19:30 ng_eiface.ko -r-xr-xr-x 1 root wheel 19096 Jan 19 19:30 ng_etf.ko -r-xr-xr-x 1 root wheel 40520 Jan 19 19:30 ng_ether.ko -r-xr-xr-x 1 root wheel 11192 Jan 19 19:30 ng_ether_echo.ko -r-xr-xr-x 1 root wheel 15080 Jan 19 19:30 ng_frame_relay.ko -r-xr-xr-x 1 root wheel 31896 Jan 19 19:30 ng_gif.ko -r-xr-xr-x 1 root wheel 15464 Jan 19 19:30 ng_gif_demux.ko -r-xr-xr-x 1 root wheel 91744 Jan 19 19:30 ng_hci.ko -r-xr-xr-x 1 root wheel 13344 Jan 19 19:30 ng_hole.ko -r-xr-xr-x 1 root wheel 13272 Jan 19 19:30 ng_hub.ko -r-xr-xr-x 1 root wheel 41120 Jan 19 19:30 ng_iface.ko -r-xr-xr-x 1 root wheel 19096 Jan 19 19:30 ng_ip_input.ko -r-xr-xr-x 1 root wheel 27176 Jan 19 19:30 ng_ipfw.ko -r-xr-xr-x 1 root wheel 43984 Jan 19 19:30 ng_ksocket.ko -r-xr-xr-x 1 root wheel 127440 Jan 19 19:30 ng_l2cap.ko -r-xr-xr-x 1 root wheel 40408 Jan 19 19:30 ng_l2tp.ko -r-xr-xr-x 1 root wheel 27224 Jan 19 19:30 ng_lmi.ko -r-xr-xr-x 1 root wheel 36880 Jan 19 19:30 ng_mppc.ko -r-xr-xr-x 1 root wheel 38520 Jan 19 19:30 ng_nat.ko -r-xr-xr-x 1 root wheel 69488 Jan 19 19:30 ng_netflow.ko -r-xr-xr-x 1 root wheel 21320 Jan 19 19:30 ng_one2many.ko -r-xr-xr-x 1 root wheel 21376 Jan 19 19:30 ng_patch.ko -r-xr-xr-x 1 root wheel 40176 Jan 19 19:30 ng_pipe.ko -r-xr-xr-x 1 root wheel 56056 Jan 19 19:30 ng_ppp.ko -r-xr-xr-x 1 root wheel 44168 Jan 19 19:30 ng_pppoe.ko -r-xr-xr-x 1 root wheel 30256 Jan 19 19:30 ng_pptpgre.ko -r-xr-xr-x 1 root wheel 21576 Jan 19 19:30 ng_pred1.ko -r-xr-xr-x 1 root wheel 24576 Jan 19 19:30 ng_rfc1490.ko -r-xr-xr-x 1 root wheel 51624 Jan 19 19:30 ng_socket.ko -r-xr-xr-x 1 root wheel 38584 Jan 19 19:30 ng_source.ko -r-xr-xr-x 1 root wheel 12680 Jan 19 19:30 ng_split.ko -r-xr-xr-x 1 root wheel 29640 Jan 19 19:30 ng_sppp.ko -r-xr-xr-x 1 root wheel 36120 Jan 19 19:30 ng_sscfu.ko -r-xr-xr-x 1 root wheel 167848 Jan 19 19:30 ng_sscop.ko -r-xr-xr-x 1 root wheel 21240 Jan 19 19:30 ng_tag.ko -r-xr-xr-x 1 root wheel 16192 Jan 19 19:30 ng_tcpmss.ko -r-xr-xr-x 1 root wheel 17048 Jan 19 19:30 ng_tee.ko -r-xr-xr-x 1 root wheel 34160 Jan 19 19:30 ng_tty.ko -r-xr-xr-x 1 root wheel 52280 Jan 19 19:30 ng_ubt.ko -r-xr-xr-x 1 root wheel 235872 Jan 19 19:30 ng_uni.ko -r-xr-xr-x 1 root wheel 29224 Jan 19 19:30 ng_vjc.ko -r-xr-xr-x 1 root wheel 21304 Jan 19 19:30 ng_vlan.ko -r-xr-xr-x 1 root wheel 333912 Jan 19 19:29 ngatmbase.ko -r-xr-xr-x 1 root wheel 25648 Jan 19 19:30 nmdm.ko -r-xr-xr-x 1 root wheel 59768 Jan 19 19:30 nullfs.ko -r-xr-xr-x 1 root wheel 21888 Jan 19 19:29 nxprtc.ko -r-xr-xr-x 1 root wheel 89632 Jan 19 19:30 ohci.ko -r-xr-xr-x 1 root wheel 36760 Jan 19 19:30 opensolaris.ko -r-xr-xr-x 1 root wheel 93624 Jan 19 19:30 otusfw_init.ko -r-xr-xr-x 1 root wheel 13184 Jan 19 19:30 otusfw_main.ko -r-xr-xr-x 1 root wheel 28192 Jan 19 19:30 ow.ko -r-xr-xr-x 1 root wheel 24968 Jan 19 19:30 ow_temp.ko -r-xr-xr-x 1 root wheel 21512 Jan 19 19:30 owc.ko -r-xr-xr-x 1 root wheel 100016 Jan 19 19:30 pccard.ko -r-xr-xr-x 1 root wheel 17864 Jan 19 19:29 pcf.ko -r-xr-xr-x 1 root wheel 396288 Jan 19 19:30 pf.ko -r-xr-xr-x 1 root wheel 30640 Jan 19 19:30 pflog.ko -r-xr-xr-x 1 root wheel 90312 Jan 19 19:30 pfsync.ko -r-xr-xr-x 1 root wheel 50728 Jan 19 19:30 plip.ko -r-xr-xr-x 1 root wheel 19640 Jan 19 19:30 powermac_nvram.ko -r-xr-xr-x 1 root wheel 54024 Jan 19 19:30 ppbus.ko -r-xr-xr-x 1 root wheel 44616 Jan 19 19:30 ppc.ko -r-xr-xr-x 1 root wheel 34144 Jan 19 19:30 ppi.ko -r-xr-xr-x 1 root wheel 24312 Jan 19 19:30 pps.ko -r-xr-xr-x 1 root wheel 60304 Jan 19 19:30 procfs.ko -r-xr-xr-x 1 root wheel 39880 Jan 19 19:29 profile.ko -r-xr-xr-x 1 root wheel 49120 Jan 19 19:30 proto.ko -r-xr-xr-x 1 root wheel 28864 Jan 19 19:29 prototype.ko -r-xr-xr-x 1 root wheel 76248 Jan 19 19:30 pseudofs.ko -r-xr-xr-x 1 root wheel 19272 Jan 19 19:30 pty.ko -r-xr-xr-x 1 root wheel 72840 Jan 19 19:30 puc.ko -r-xr-xr-x 1 root wheel 94896 Jan 19 19:29 r128.ko -r-xr-xr-x 1 root wheel 1784032 Jan 19 19:29 radeonkms.ko -r-xr-xr-x 1 root wheel 16176 Jan 19 19:29 radeonkmsfw_ARUBA_me.ko -r-xr-xr-x 1 root wheel 16192 Jan 19 19:29 radeonkmsfw_ARUBA_pfp.ko -r-xr-xr-x 1 root wheel 15992 Jan 19 19:29 radeonkmsfw_ARUBA_rlc.ko -r-xr-xr-x 1 root wheel 31560 Jan 19 19:29 radeonkmsfw_BARTS_mc.ko -r-xr-xr-x 1 root wheel 12976 Jan 19 19:29 radeonkmsfw_BARTS_me.ko -r-xr-xr-x 1 root wheel 11976 Jan 19 19:29 radeonkmsfw_BARTS_pfp.ko -r-xr-xr-x 1 root wheel 12880 Jan 19 19:29 radeonkmsfw_BTC_rlc.ko -r-xr-xr-x 1 root wheel 31584 Jan 19 19:29 radeonkmsfw_CAICOS_mc.ko -r-xr-xr-x 1 root wheel 13000 Jan 19 19:29 radeonkmsfw_CAICOS_me.ko -r-xr-xr-x 1 root wheel 11984 Jan 19 19:29 radeonkmsfw_CAICOS_pfp.ko -r-xr-xr-x 1 root wheel 31648 Jan 19 19:29 radeonkmsfw_CAYMAN_mc.ko -r-xr-xr-x 1 root wheel 16192 Jan 19 19:29 radeonkmsfw_CAYMAN_me.ko -r-xr-xr-x 1 root wheel 16216 Jan 19 19:29 radeonkmsfw_CAYMAN_pfp.ko -r-xr-xr-x 1 root wheel 11608 Jan 19 19:29 radeonkmsfw_CAYMAN_rlc.ko -r-xr-xr-x 1 root wheel 12976 Jan 19 19:29 radeonkmsfw_CEDAR_me.ko -r-xr-xr-x 1 root wheel 11968 Jan 19 19:29 radeonkmsfw_CEDAR_pfp.ko -r-xr-xr-x 1 root wheel 12920 Jan 19 19:29 radeonkmsfw_CEDAR_rlc.ko -r-xr-xr-x 1 root wheel 13016 Jan 19 19:29 radeonkmsfw_CYPRESS_me.ko -r-xr-xr-x 1 root wheel 12016 Jan 19 19:29 radeonkmsfw_CYPRESS_pfp.ko -r-xr-xr-x 1 root wheel 12968 Jan 19 19:29 radeonkmsfw_CYPRESS_rlc.ko -r-xr-xr-x 1 root wheel 13016 Jan 19 19:29 radeonkmsfw_JUNIPER_me.ko -r-xr-xr-x 1 root wheel 12016 Jan 19 19:29 radeonkmsfw_JUNIPER_pfp.ko -r-xr-xr-x 1 root wheel 12968 Jan 19 19:29 radeonkmsfw_JUNIPER_rlc.ko -r-xr-xr-x 1 root wheel 12952 Jan 19 19:29 radeonkmsfw_PALM_me.ko -r-xr-xr-x 1 root wheel 11952 Jan 19 19:29 radeonkmsfw_PALM_pfp.ko -r-xr-xr-x 1 root wheel 16104 Jan 19 19:29 radeonkmsfw_PITCAIRN_ce.ko -r-xr-xr-x 1 root wheel 40968 Jan 19 19:29 radeonkmsfw_PITCAIRN_mc.ko -r-xr-xr-x 1 root wheel 16112 Jan 19 19:29 radeonkmsfw_PITCAIRN_me.ko -r-xr-xr-x 1 root wheel 16128 Jan 19 19:29 radeonkmsfw_PITCAIRN_pfp.ko -r-xr-xr-x 1 root wheel 15760 Jan 19 19:29 radeonkmsfw_PITCAIRN_rlc.ko -r-xr-xr-x 1 root wheel 11848 Jan 19 19:29 radeonkmsfw_R100_cp.ko -r-xr-xr-x 1 root wheel 11848 Jan 19 19:29 radeonkmsfw_R200_cp.ko -r-xr-xr-x 1 root wheel 11848 Jan 19 19:29 radeonkmsfw_R300_cp.ko -r-xr-xr-x 1 root wheel 11856 Jan 19 19:29 radeonkmsfw_R420_cp.ko -r-xr-xr-x 1 root wheel 11848 Jan 19 19:29 radeonkmsfw_R520_cp.ko -r-xr-xr-x 1 root wheel 28944 Jan 19 19:29 radeonkmsfw_R600_me.ko -r-xr-xr-x 1 root wheel 12128 Jan 19 19:29 radeonkmsfw_R600_pfp.ko -r-xr-xr-x 1 root wheel 12904 Jan 19 19:29 radeonkmsfw_R600_rlc.ko -r-xr-xr-x 1 root wheel 11568 Jan 19 19:29 radeonkmsfw_R700_rlc.ko -r-xr-xr-x 1 root wheel 13016 Jan 19 19:29 radeonkmsfw_REDWOOD_me.ko -r-xr-xr-x 1 root wheel 12024 Jan 19 19:29 radeonkmsfw_REDWOOD_pfp.ko -r-xr-xr-x 1 root wheel 12952 Jan 19 19:29 radeonkmsfw_REDWOOD_rlc.ko -r-xr-xr-x 1 root wheel 11880 Jan 19 19:29 radeonkmsfw_RS600_cp.ko -r-xr-xr-x 1 root wheel 11880 Jan 19 19:29 radeonkmsfw_RS690_cp.ko -r-xr-xr-x 1 root wheel 28976 Jan 19 19:29 radeonkmsfw_RS780_me.ko -r-xr-xr-x 1 root wheel 12152 Jan 19 19:29 radeonkmsfw_RS780_pfp.ko -r-xr-xr-x 1 root wheel 28976 Jan 19 19:29 radeonkmsfw_RV610_me.ko -r-xr-xr-x 1 root wheel 12144 Jan 19 19:29 radeonkmsfw_RV610_pfp.ko -r-xr-xr-x 1 root wheel 28976 Jan 19 19:29 radeonkmsfw_RV620_me.ko -r-xr-xr-x 1 root wheel 12144 Jan 19 19:29 radeonkmsfw_RV620_pfp.ko -r-xr-xr-x 1 root wheel 28976 Jan 19 19:29 radeonkmsfw_RV630_me.ko -r-xr-xr-x 1 root wheel 12152 Jan 19 19:29 radeonkmsfw_RV630_pfp.ko -r-xr-xr-x 1 root wheel 28976 Jan 19 19:29 radeonkmsfw_RV635_me.ko -r-xr-xr-x 1 root wheel 12152 Jan 19 19:29 radeonkmsfw_RV635_pfp.ko -r-xr-xr-x 1 root wheel 28976 Jan 19 19:29 radeonkmsfw_RV670_me.ko -r-xr-xr-x 1 root wheel 12152 Jan 19 19:29 radeonkmsfw_RV670_pfp.ko -r-xr-xr-x 1 root wheel 12912 Jan 19 19:29 radeonkmsfw_RV710_me.ko -r-xr-xr-x 1 root wheel 13232 Jan 19 19:29 radeonkmsfw_RV710_pfp.ko -r-xr-xr-x 1 root wheel 12912 Jan 19 19:29 radeonkmsfw_RV730_me.ko -r-xr-xr-x 1 root wheel 13240 Jan 19 19:29 radeonkmsfw_RV730_pfp.ko -r-xr-xr-x 1 root wheel 12912 Jan 19 19:29 radeonkmsfw_RV770_me.ko -r-xr-xr-x 1 root wheel 13240 Jan 19 19:29 radeonkmsfw_RV770_pfp.ko -r-xr-xr-x 1 root wheel 12976 Jan 19 19:29 radeonkmsfw_SUMO2_me.ko -r-xr-xr-x 1 root wheel 11976 Jan 19 19:29 radeonkmsfw_SUMO2_pfp.ko -r-xr-xr-x 1 root wheel 12952 Jan 19 19:29 radeonkmsfw_SUMO_me.ko -r-xr-xr-x 1 root wheel 11952 Jan 19 19:29 radeonkmsfw_SUMO_pfp.ko -r-xr-xr-x 1 root wheel 12904 Jan 19 19:29 radeonkmsfw_SUMO_rlc.ko -r-xr-xr-x 1 root wheel 16072 Jan 19 19:29 radeonkmsfw_TAHITI_ce.ko -r-xr-xr-x 1 root wheel 40928 Jan 19 19:29 radeonkmsfw_TAHITI_mc.ko -r-xr-xr-x 1 root wheel 16072 Jan 19 19:29 radeonkmsfw_TAHITI_me.ko -r-xr-xr-x 1 root wheel 16088 Jan 19 19:29 radeonkmsfw_TAHITI_pfp.ko -r-xr-xr-x 1 root wheel 15704 Jan 19 19:29 radeonkmsfw_TAHITI_rlc.ko -r-xr-xr-x 1 root wheel 31568 Jan 19 19:29 radeonkmsfw_TURKS_mc.ko -r-xr-xr-x 1 root wheel 12976 Jan 19 19:29 radeonkmsfw_TURKS_me.ko -r-xr-xr-x 1 root wheel 11976 Jan 19 19:29 radeonkmsfw_TURKS_pfp.ko -r-xr-xr-x 1 root wheel 16048 Jan 19 19:29 radeonkmsfw_VERDE_ce.ko -r-xr-xr-x 1 root wheel 40912 Jan 19 19:29 radeonkmsfw_VERDE_mc.ko -r-xr-xr-x 1 root wheel 16048 Jan 19 19:29 radeonkmsfw_VERDE_me.ko -r-xr-xr-x 1 root wheel 16072 Jan 19 19:29 radeonkmsfw_VERDE_pfp.ko -r-xr-xr-x 1 root wheel 15688 Jan 19 19:29 radeonkmsfw_VERDE_rlc.ko -r-xr-xr-x 1 root wheel 34672 Jan 19 19:30 random_OTHER.ko -r-xr-xr-x 1 root wheel 44520 Jan 19 19:30 random_fortuna.ko -r-xr-xr-x 1 root wheel 45552 Jan 19 19:30 random_yarrow.ko -r-xr-xr-x 1 root wheel 6648 Jan 19 19:30 rc4.ko -r-xr-xr-x 1 root wheel 132048 Jan 19 19:30 rsu-rtl8712fw.ko -r-xr-xr-x 1 root wheel 15416 Jan 19 19:30 rt2561fw.ko -r-xr-xr-x 1 root wheel 15448 Jan 19 19:30 rt2561sfw.ko -r-xr-xr-x 1 root wheel 15424 Jan 19 19:30 rt2661fw.ko -r-xr-xr-x 1 root wheel 15424 Jan 19 19:30 rt2860fw.ko -r-xr-xr-x 1 root wheel 21336 Jan 19 19:30 rtwn-rtl8188eufw.ko -r-xr-xr-x 1 root wheel 24608 Jan 19 19:30 rtwn-rtl8192cfwE.ko -r-xr-xr-x 1 root wheel 23808 Jan 19 19:30 rtwn-rtl8192cfwE_B.ko -r-xr-xr-x 1 root wheel 23552 Jan 19 19:30 rtwn-rtl8192cfwT.ko -r-xr-xr-x 1 root wheel 23560 Jan 19 19:30 rtwn-rtl8192cfwU.ko -r-xr-xr-x 1 root wheel 41600 Jan 19 19:30 rtwn-rtl8192eufw.ko -r-xr-xr-x 1 root wheel 41848 Jan 19 19:30 rtwn-rtl8812aufw.ko -r-xr-xr-x 1 root wheel 37584 Jan 19 19:30 rtwn-rtl8821aufw.ko -r-xr-xr-x 1 root wheel 255960 Jan 19 19:30 rtwn.ko -r-xr-xr-x 1 root wheel 15352 Jan 19 19:30 runfw.ko -r-xr-xr-x 1 root wheel 16800 Jan 19 19:29 s35390a.ko -r-xr-xr-x 1 root wheel 74992 Jan 19 19:29 savage.ko -r-xr-xr-x 1 root wheel 87480 Jan 19 19:29 sbp.ko -r-xr-xr-x 1 root wheel 67432 Jan 19 19:29 sbp_targ.ko -r-xr-xr-x 1 root wheel 45656 Jan 19 19:30 scc.ko -r-xr-xr-x 1 root wheel 69504 Jan 19 19:30 sdhci.ko -r-xr-xr-x 1 root wheel 36632 Jan 19 19:30 sdhci_pci.ko -r-xr-xr-x 1 root wheel 35672 Jan 19 19:29 sdt.ko -r-xr-xr-x 1 root wheel 58344 Jan 19 19:30 sem.ko -r-xr-xr-x 1 root wheel 29328 Jan 19 19:30 send.ko -r-xr-xr-x 1 root wheel 74728 Jan 19 19:29 siba.ko -r-xr-xr-x 1 root wheel 20288 Jan 19 19:29 siba_bhndb.ko -r-xr-xr-x 1 root wheel 92064 Jan 19 19:30 siba_bwn.ko -r-xr-xr-x 1 root wheel 49440 Jan 19 19:30 siftr.ko -r-xr-xr-x 1 root wheel 73672 Jan 19 19:30 siis.ko -r-xr-xr-x 1 root wheel 37344 Jan 19 19:29 sis.ko -r-xr-xr-x 1 root wheel 18376 Jan 19 19:29 smb.ko -r-xr-xr-x 1 root wheel 282048 Jan 19 19:30 smbfs.ko -r-xr-xr-x 1 root wheel 18712 Jan 19 19:29 smbus.ko -r-xr-xr-x 1 root wheel 45320 Jan 19 19:30 snd_ad1816.ko -r-xr-xr-x 1 root wheel 72632 Jan 19 19:30 snd_ai2s.ko -r-xr-xr-x 1 root wheel 50152 Jan 19 19:30 snd_als4000.ko -r-xr-xr-x 1 root wheel 55792 Jan 19 19:30 snd_atiixp.ko -r-xr-xr-x 1 root wheel 45816 Jan 19 19:30 snd_cs4281.ko -r-xr-xr-x 1 root wheel 119656 Jan 19 19:30 snd_csa.ko -r-xr-xr-x 1 root wheel 45696 Jan 19 19:30 snd_davbus.ko -r-xr-xr-x 1 root wheel 24200 Jan 19 19:30 snd_driver.ko -r-xr-xr-x 1 root wheel 80488 Jan 19 19:30 snd_ds1.ko -r-xr-xr-x 1 root wheel 70760 Jan 19 19:30 snd_emu10k1.ko -r-xr-xr-x 1 root wheel 153784 Jan 19 19:30 snd_emu10kx.ko -r-xr-xr-x 1 root wheel 75760 Jan 19 19:30 snd_envy24.ko -r-xr-xr-x 1 root wheel 67816 Jan 19 19:30 snd_envy24ht.ko -r-xr-xr-x 1 root wheel 72592 Jan 19 19:30 snd_es137x.ko -r-xr-xr-x 1 root wheel 52040 Jan 19 19:30 snd_ess.ko -r-xr-xr-x 1 root wheel 40328 Jan 19 19:30 snd_fm801.ko -r-xr-xr-x 1 root wheel 310800 Jan 19 19:30 snd_hda.ko -r-xr-xr-x 1 root wheel 52848 Jan 19 19:30 snd_hdspe.ko -r-xr-xr-x 1 root wheel 54312 Jan 19 19:30 snd_ich.ko -r-xr-xr-x 1 root wheel 75288 Jan 19 19:30 snd_maestro.ko -r-xr-xr-x 1 root wheel 74960 Jan 19 19:30 snd_maestro3.ko -r-xr-xr-x 1 root wheel 93392 Jan 19 19:30 snd_neomagic.ko -r-xr-xr-x 1 root wheel 45120 Jan 19 19:30 snd_sb16.ko -r-xr-xr-x 1 root wheel 45024 Jan 19 19:30 snd_sb8.ko -r-xr-xr-x 1 root wheel 39648 Jan 19 19:30 snd_sbc.ko -r-xr-xr-x 1 root wheel 50904 Jan 19 19:30 snd_solo.ko -r-xr-xr-x 1 root wheel 22328 Jan 19 19:30 snd_spicds.ko -r-xr-xr-x 1 root wheel 49648 Jan 19 19:30 snd_t4dwave.ko -r-xr-xr-x 1 root wheel 141792 Jan 19 19:30 snd_uaudio.ko -r-xr-xr-x 1 root wheel 61152 Jan 19 19:30 snd_via8233.ko -r-xr-xr-x 1 root wheel 40576 Jan 19 19:30 snd_via82c686.ko -r-xr-xr-x 1 root wheel 49704 Jan 19 19:30 snd_vibes.ko -r-xr-xr-x 1 root wheel 25080 Jan 19 19:30 snp.ko -r-xr-xr-x 1 root wheel 631632 Jan 19 19:30 sound.ko -r-xr-xr-x 1 root wheel 30536 Jan 19 19:30 spigen.ko -r-xr-xr-x 1 root wheel 134232 Jan 19 19:30 sym.ko -r-xr-xr-x 1 root wheel 90168 Jan 19 19:29 systrace.ko -r-xr-xr-x 1 root wheel 88376 Jan 19 19:29 systrace_freebsd32.ko -r-xr-xr-x 1 root wheel 57912 Jan 19 19:30 sysvmsg.ko -r-xr-xr-x 1 root wheel 62256 Jan 19 19:30 sysvsem.ko -r-xr-xr-x 1 root wheel 59048 Jan 19 19:30 sysvshm.ko -r-xr-xr-x 1 root wheel 577904 Jan 19 19:29 t4fw_cfg.ko -r-xr-xr-x 1 root wheel 696984 Jan 19 19:29 t5fw_cfg.ko -r-xr-xr-x 1 root wheel 740816 Jan 19 19:29 t6fw_cfg.ko -r-xr-xr-x 1 root wheel 27456 Jan 19 19:30 tcpmd5.ko -r-xr-xr-x 1 root wheel 25064 Jan 19 19:29 tdfx.ko -r-xr-xr-x 1 root wheel 130000 Jan 19 19:30 tmpfs.ko -r-xr-xr-x 1 root wheel 38496 Jan 19 19:30 toecore.ko -r-xr-xr-x 1 root wheel 68152 Jan 19 19:30 trm.ko -r-xr-xr-x 1 root wheel 84632 Jan 19 19:30 twe.ko -r-xr-xr-x 1 root wheel 103864 Jan 19 19:30 tws.ko -r-xr-xr-x 1 root wheel 51704 Jan 19 19:30 u3g.ko -r-xr-xr-x 1 root wheel 27040 Jan 19 19:30 uark.ko -r-xr-xr-x 1 root wheel 183056 Jan 19 19:30 uart.ko -r-xr-xr-x 1 root wheel 31992 Jan 19 19:30 ubsa.ko -r-xr-xr-x 1 root wheel 75104 Jan 19 19:30 ubsec.ko -r-xr-xr-x 1 root wheel 28960 Jan 19 19:30 ubser.ko -r-xr-xr-x 1 root wheel 28416 Jan 19 19:30 ubtbcmfw.ko -r-xr-xr-x 1 root wheel 36272 Jan 19 19:30 uchcom.ko -r-xr-xr-x 1 root wheel 51288 Jan 19 19:30 ucom.ko -r-xr-xr-x 1 root wheel 29152 Jan 19 19:30 ucycom.ko -r-xr-xr-x 1 root wheel 34872 Jan 19 19:30 udbp.ko -r-xr-xr-x 1 root wheel 74168 Jan 19 19:30 udf.ko -r-xr-xr-x 1 root wheel 8008 Jan 19 19:30 udf_iconv.ko -r-xr-xr-x 1 root wheel 45664 Jan 19 19:30 udl.ko -r-xr-xr-x 1 root wheel 22936 Jan 19 19:30 uep.ko -r-xr-xr-x 1 root wheel 40096 Jan 19 19:30 uether.ko -r-xr-xr-x 1 root wheel 15480 Jan 19 19:30 ufm.ko -r-xr-xr-x 1 root wheel 45664 Jan 19 19:30 ufoma.ko -r-xr-xr-x 1 root wheel 768952 Jan 19 19:30 ufs.ko -r-xr-xr-x 1 root wheel 62232 Jan 19 19:30 uftdi.ko -r-xr-xr-x 1 root wheel 25928 Jan 19 19:30 ugensa.ko -r-xr-xr-x 1 root wheel 24920 Jan 19 19:30 ugold.ko -r-xr-xr-x 1 root wheel 89880 Jan 19 19:30 uhci.ko -r-xr-xr-x 1 root wheel 32056 Jan 19 19:30 uhid.ko -r-xr-xr-x 1 root wheel 76768 Jan 19 19:30 uhso.ko -r-xr-xr-x 1 root wheel 31856 Jan 19 19:30 uinput.ko -r-xr-xr-x 1 root wheel 41192 Jan 19 19:30 uipaq.ko -r-xr-xr-x 1 root wheel 76952 Jan 19 19:30 ukbd.ko -r-xr-xr-x 1 root wheel 15920 Jan 19 19:30 uled.ko -r-xr-xr-x 1 root wheel 31352 Jan 19 19:30 ulpt.ko -r-xr-xr-x 1 root wheel 73776 Jan 19 19:30 umass.ko -r-xr-xr-x 1 root wheel 46104 Jan 19 19:30 umcs.ko -r-xr-xr-x 1 root wheel 29880 Jan 19 19:30 umct.ko -r-xr-xr-x 1 root wheel 40568 Jan 19 19:30 umodem.ko -r-xr-xr-x 1 root wheel 32360 Jan 19 19:30 umoscom.ko -r-xr-xr-x 1 root wheel 36656 Jan 19 19:30 ums.ko -r-xr-xr-x 1 root wheel 110664 Jan 19 19:30 unionfs.ko -r-xr-xr-x 1 root wheel 39448 Jan 19 19:30 uplcom.ko -r-xr-xr-x 1 root wheel 25784 Jan 19 19:30 urio.ko -r-xr-xr-x 1 root wheel 441120 Jan 19 19:30 usb.ko -r-xr-xr-x 1 root wheel 31128 Jan 19 19:30 usb_quirk.ko -r-xr-xr-x 1 root wheel 59424 Jan 19 19:30 usb_template.ko -r-xr-xr-x 1 root wheel 36344 Jan 19 19:30 usfs.ko -r-xr-xr-x 1 root wheel 71192 Jan 19 19:30 usie.ko -r-xr-xr-x 1 root wheel 40984 Jan 19 19:30 uslcom.ko -r-xr-xr-x 1 root wheel 30272 Jan 19 19:30 uvisor.ko -r-xr-xr-x 1 root wheel 32600 Jan 19 19:30 uvscom.ko -r-xr-xr-x 1 root wheel 93216 Jan 19 19:29 via.ko -r-xr-xr-x 1 root wheel 41184 Jan 19 19:29 viapm.ko -r-xr-xr-x 1 root wheel 35536 Jan 19 19:30 videomode.ko -r-xr-xr-x 1 root wheel 50920 Jan 19 19:30 vkbd.ko -r-xr-xr-x 1 root wheel 719280 Jan 19 19:30 wlan.ko -r-xr-xr-x 1 root wheel 37408 Jan 19 19:30 wlan_acl.ko -r-xr-xr-x 1 root wheel 39184 Jan 19 19:30 wlan_amrr.ko -r-xr-xr-x 1 root wheel 55744 Jan 19 19:30 wlan_ccmp.ko -r-xr-xr-x 1 root wheel 36792 Jan 19 19:30 wlan_rssadapt.ko -r-xr-xr-x 1 root wheel 44016 Jan 19 19:30 wlan_tkip.ko -r-xr-xr-x 1 root wheel 35312 Jan 19 19:30 wlan_wep.ko -r-xr-xr-x 1 root wheel 28464 Jan 19 19:30 wlan_xauth.ko -r-xr-xr-x 1 root wheel 27960 Jan 19 19:30 wmt.ko -r-xr-xr-x 1 root wheel 38008 Jan 19 19:30 wsp.ko -r-xr-xr-x 1 root wheel 111744 Jan 19 19:30 xhci.ko -r-xr-xr-x 1 root wheel 4141720 Jan 19 19:30 zfs.ko -r-xr-xr-x 1 root wheel 64936 Jan 19 19:30 zlib.ko
This error message seems to come from the libsa: This is the code that is detecting this problem: if ((char *)ptr < (char *)mn + mn->mr_Bytes) { panic("zfree(%p,%ju): corrupt memlist2", ptr, (uintmax_t)bytes); }
do you know where the breakage starts?
It showed up in the very first time with commit f375b4dd66bc709573ad4286b3a5ac6c705ba4a8('Implement NUMA support in uma(9) and malloc(9).')
Jeff, can you take a look at this?
This is svn r327900
In order to get more information about the heap, I created a patch that dumps the heap just before the crash. This is the patch: commit e75d21fe2d5ce7408b6280dd796f68580668b1ae (HEAD -> master) Author: Breno Leitao <breno.leitao@gmail.com> Date: Wed Jan 24 13:04:58 2018 +0100 libsa: Dump the heap diff --git a/stand/libsa/zalloc.c b/stand/libsa/zalloc.c index 4d1ec629121..7a5007f4b46 100644 --- a/stand/libsa/zalloc.c +++ b/stand/libsa/zalloc.c @@ -144,6 +144,20 @@ znalloc(MemPool *mp, uintptr_t bytes) * zfree() - free previously allocated memory */ +void dumpHeap(MemPool *mp){ + MemNode **pmn; + MemNode *mn; + + printf("Mempool base %p and end %p\n", mp->mp_Base, mp->mp_End); + printf(" mp_Size %x and mp_Used %x\n", mp->mp_Size, mp->mp_Used); + + for (pmn = &mp->mp_First; (mn = *pmn) != NULL; pmn = &mn->mr_Next) { + printf("MN = %p and Byte is %x\n", mn, mn->mr_Bytes); + } + +} + void zfree(MemPool *mp, void *ptr, uintptr_t bytes) { @@ -220,7 +234,8 @@ zfree(MemPool *mp, void *ptr, uintptr_t bytes) /* NOT REACHED */ } if ((char *)ptr < (char *)mn + mn->mr_Bytes) { - panic("zfree(%p,%ju): corrupt memlist2", ptr, + dumpHeap(mp); + printf("zfree(%p,%ju): corrupt memlist2", ptr, (uintmax_t)bytes); } } Once I boot the broken kernel, this is the info I get: /boot/kernel/kernel data=0x11ef978+0x4c6e48 syms=[0x8+0x1684e0 Mempool base rx1810d40 and end 0x18a7d40 mp_Size 97000 and mp_Used 83e80 MN = 0x182d900 and Byte is 64cab8 zfree(0x183c080,4224): corrupt memlist2 elf64_loadimage: could not read symbols - skipped! (491520 != 1475808) Mempool base rx1810d40 and end 0x18a7d40 mp_Size 97000 and mp_Used 83080 MN = 0x182d900 and Byte is 64cab8 MN = 0x183c080 and Byte is 1080 ---- Trying to understand the problem, there is an allocation of 0x1080 (4224 in decimal) bytes at 0x183c080 but free() and free is complaining that this is not 4224 bytes long. Not sure what might be wrong here.
If I do not load the kernel automatically, but loading it manually in the loader, I see a different face of the issue: OK unload OK load /boot/broken/kernel /boot/broken/kernel data=0x11ef978+0x4c6e48 syms=[0x8+0x1684e0 ( 300 ) Data Storage Exception [ 6865616465720000 ] R0 .. R7 R8 .. R15 R16 .. R23 R24 .. R31 8000000000001000 000000007dc44be0 0000000000000000 000000007dbe0bb0 000000007e45e6f0 000000007e4eb2b0 0000000000000000 0000000000000006 000000007dbffc00 000000007dbe0c24 0000000000000000 000000007dbf6800 000000007dc45000 0000000000000000 000000007dbe0d54 000000007e45f060 0000000000000000 0000000000000018 000000007dbfe070 000000007dbf92b0 6865616465720000 0000000006a6b540 000000000000004b 0000000000000003 000000007e45b010 0000000000000000 0000000001c40707 000000007dbfdff8 6865616465720008 0000000000000000 000000007e566465 000000007e4eb2a0 CR / XER LR / CTR SRR0 / SRR1 DAR / DSISR 8000f424 000000007dbe20f4 000000007dbe131c 6865616465720008 0000000020000000 000000007dbe0c24 8000000000001000 40000000
Any clue to when this regression happened?
( 300 ) Data Storage Exception [ 6865616465720000 ] <-- very suspicious Treating that as ASCII string (looks like it) I see: "header\0\0". Looks like something's trying to access a string as a pointer (use-after-free?)
Good point Justin. The problem seems to happen with different values, but the SRR0 seems to be always the same (000000007dbe131c). So, it seems that there is a crazy pointer being dereferenced. ( 300 ) Data Storage Exception [ 7e45b008 ] R0 .. R7 R8 .. R15 R16 .. R23 R24 .. R31 8000000000001000 000000007dc43f20 0000000000000000 000000007dbe0c74 000000007e45cf50 000000007e4f2b18 0000000000000000 0000000000000006 000000007dbffc00 000000007dbe0bb0 0000000000000000 000000007dbf6800 000000007dc571e8 00000000a47bfd70 000000007dbe0d54 000000007e45d8c0 000000007e4f3a40 000000006f5f7375 000000007dbfe070 000000007dbf92b0 000000007e45b008 0000000006ac0540 000000000000004b 000000000000000b 0000000000000000 0000000000000000 0000000001c40707 000000007e4eb290 5e8987785a729bbe 0000000000000000 000000007e566465 000000007e45b010 CR / XER LR / CTR SRR0 / SRR1 DAR / DSISR 80044422 000000007dbe20f4 000000007dbe131c 5e8987785a729bbe 0000000020000000 000000007dbe0bb0 8000000000001000 40000000
I made some progress on this front. I found that the SRR0 value is not mapped looking at the SLOF mapped area: available 00000000 00004000 00000000 7dbdc000 00000000 7fee0000 00000000 00010000 00000000 7fef3719 00000000 000fc8e7 00000000 7fff0014 00000000 0000ffec Which means that we have address 00004000 until 00004000+7dbdc000 (7DBE0000), so, any address bigger than 7DBE0000 is not mapped, and if you look at SRR it says 000000007dbe131c. That said, it seems that we are using an unmapped address. I also found that the kernel that crashes is ~2x bigger that the original kernel. (I am not sure yet why the kernel size became so big -- was it because f375b4dd66bc709573ad4286b3a5ac6c705ba4a8?) That said, if I increase the HEAP_SIZE, I do not see this problem anymore. With a new HEAP_SIZE, the loader is able to load the kernel properly, but I am not facing this error: panic: acquiring blockable sleep lock with spinlock or critical section held (sleep mutex) vm map (system) @ /root/kernel/freebsd/sys/vm/vm_map.c:4065 cpuid = 48 time = 1 KDB: stack backtrace: 0xc0000000000078b0: at .kdb_backtrace+0x5c 0xc0000000000079e0: at .vpanic+0x1a4 0xc000000000007aa0: at .kassert_panic+0x8c 0xc000000000007b40: at .witness_checkorder+0xf8 0xc000000000007c30: at .__mtx_lock_flags+0xfc 0xc000000000007ce0: at ._vm_map_lock_read+0x34 0xc000000000007d60: at .vm_map_lookup+0x94 0xc000000000007e90: at .vm_fault_hold+0x158 0xc0000000000080a0: at .vm_fault+0x9c 0xc000000000008150: at .trap_pfault+0xd8 0xc0000000000081f0: at .trap+0xec0 0xc0000000000083b0: at .powerpc_interrupt+0x1e0 0xc000000000008450: kernel DSI read trap @ 0xdeadc0dedeadc0e6 by .uma_zalloc_arg+0x35c: srr1=0x8000000000009032 r1=0xc000000000008700 cr=0x22004028 xer=0 ctr=0 r2=0x12e3bd8 sr=0x40000000 0xc000000000008700: at .uma_zalloc_arg+0x260 0xc0000000000087d0: at .malloc+0xe4 0xc000000000008880: at .taskqgroup_adjust+0x9c4 0xc000000000008990: at .taskqgroup_adjust_if_io_tqg+0x28 0xc000000000008a10: at .mi_startup+0x11c 0xc000000000008aa0: at btext+0xb4
I was able to call mallocstats() just before booting the old and the new (HEAD) kernel: Not that on both case, the Weird enough, both of them seems to be the same: Old Kernel: Active Allocations: 872/1663 380928 bytes reserved 244608 bytes allocated 14 fragments (83136 bytes fragmented) HEAD Kernel: Active Allocations: 902/1663 380928 bytes reserved 248704 bytes allocated 14 fragments (83392 bytes fragmented) Anyway, the only way to boot the new kernel is using the following patch: diff --git a/stand/ofw/common/main.c b/stand/ofw/common/main.c index 128542c922f..607455e9515 100644 --- a/stand/ofw/common/main.c +++ b/stand/ofw/common/main.c @@ -42,7 +42,7 @@ u_int32_t acells, scells; static char bootargs[128]; -#define HEAP_SIZE 0x100000 +#define HEAP_SIZE 0x800000 #define OF_puts(fd, text) OF_write(fd, text, strlen(text))
A commit references this bug: Author: imp Date: Fri Feb 2 19:42:02 UTC 2018 New revision: 328806 URL: https://svnweb.freebsd.org/changeset/base/328806 Log: We need more heap space to properly load newer powerpc kernels. PR: 225323 Changes: head/stand/ofw/common/main.c
A commit references this bug: Author: kevans Date: Mon Feb 12 20:51:29 UTC 2018 New revision: 329183 URL: https://svnweb.freebsd.org/changeset/base/329183 Log: MFC Loader Fixes Final: r327612,r327703,r327704,r327878,r327879,r327881, r328007,r328029,r328030,r328031,r328061,r328156,r328169,r328288,r328289, r328290,r328291,r328292,r328411,r328536,r328603,r328614,r328642,r328769, r328779,r328780,r328781,r328782,r328783,r328806,r328808,r328826,r328835, r328911,r328986,r328987,r328990,r328999,r329000,r329019,r329050,r329054, r329060 r327612: Invent new #defines for the biospci_{read,write}_config function r327703: Define __dmadat after #include'ing ufsread.c. r327704: Fix printf missing format variables warnings. r327878: Add GUID for UEFI boot manager variables. r327879: Report the boot order and where we are in that boot order. r327881: Allow this file to be included r328007: Fix booting on some arm64 systems after r327879 by fixing the call to utf8_to_ucs2 r328029: When returning an error and freeing allocated memory from ucs2_to_utf8, NULL the return pointer. r328030: Check the return value from utf8_to_ucs2 instead of whether or not uv is NULL. r328031: Need to free uv after we're done using it. r328061: utf8_to_ucs2() should check for malloc failure r328156: stand: Move sections around to fix stand/ build with ld.lld on armv7 r328169: Remove extra copy of bootinfo.c. It's a bit rotted copy of the one in efi/loader. r328288: Fix some resource leaks. r328289: Don't leak memory when displaying help. r328290: On malloc failure, be sure to close the include file that triggered it. r328291: getenv does not return tainted data in the boot loader. Attempt to clue Coverity into that fact. r328292: There's no tainted data here, tag it as such to avoid false positives. r328411: loader.efi: add missing EFI GUIDs r328536: loader: support for mixed-endianness ELF/loader and POWER8 r328603: Add missing non-POWERPC case to give the scr value something non-zero. r328614: Move libstand.3 to libsa.3. Update libsa.3 to include functions r328642: Break out the interpreters (simple and forth) w/o ifdefs. r328769: Centralize several variables. r328779: Retire pnp.4th and the code needed only for 4th words used here. r328780: These 4th words were an attempt to allow integration into the boot loader scripts. However, that path won't be taken after all it seems. r328781: Remove pcibios forth support. r328782: Now that we no longer conditionally compile some files outside of ficl r328783: Invent new LDR_INTERP for the loader interpreter to use. r328806: We need more heap space to properly load newer powerpc kernels. r328808: Implement strcoll as strcmp. r328826: Make cross-endian loader changes apply only to powerpc r328835: Fix regression introduced in r328806, preventing boot on many platforms. r328911: Ignore relocation tables for non-memory-resident sections. r328986: Fix relative location of USB sources after recent move. r328987: A more definitions to kernel emulation shim in order to build stand/usb. r328990: Move the stand/usb test loader into its own directory. r328999: Fix indentation to FreeBSD standard for interp files r329000: Move simple interpreter 'perform' into interp.c and call it r329019: Move to tabs for indentation and to 8-space notches, per style(9). r329050: Fix build of userboot.so r329054: Set script.lang in the environment to either 'forth' or 'simple' to reflect what scripting language was compiled into the loader. r329060: loader: fix endianness conversion PR: 225323 Changes: _U stable/11/ stable/11/ObsoleteFiles.inc stable/11/stand/arm/uboot/Makefile stable/11/stand/arm/uboot/ldscript.arm stable/11/stand/common/boot.c stable/11/stand/common/bootstrap.h stable/11/stand/common/commands.c stable/11/stand/common/install.c stable/11/stand/common/interp.c stable/11/stand/common/interp_backslash.c stable/11/stand/common/interp_forth.c stable/11/stand/common/interp_parse.c stable/11/stand/common/interp_simple.c stable/11/stand/common/load_elf.c stable/11/stand/common/load_elf_obj.c stable/11/stand/common/misc.c stable/11/stand/common/pnp.c stable/11/stand/defs.mk stable/11/stand/efi/boot1/Makefile stable/11/stand/efi/boot1/boot1.c stable/11/stand/efi/fdt/Makefile stable/11/stand/efi/include/efi.h stable/11/stand/efi/include/efiapi.h stable/11/stand/efi/libefi/Makefile stable/11/stand/efi/libefi/efichar.c stable/11/stand/efi/libefi/env.c stable/11/stand/efi/loader/Makefile stable/11/stand/efi/loader/arch/arm/ldscript.arm stable/11/stand/efi/loader/arch/i386/bootinfo.c stable/11/stand/efi/loader/main.c stable/11/stand/fdt/Makefile stable/11/stand/ficl/Makefile stable/11/stand/ficl.mk stable/11/stand/forth/Makefile stable/11/stand/forth/loader.4th stable/11/stand/forth/pcibios.4th stable/11/stand/forth/pnp.4th stable/11/stand/geli/Makefile stable/11/stand/i386/boot0/Makefile stable/11/stand/i386/btx/btx/Makefile stable/11/stand/i386/btx/btxldr/Makefile stable/11/stand/i386/btx/lib/Makefile stable/11/stand/i386/cdboot/Makefile stable/11/stand/i386/kgzldr/Makefile stable/11/stand/i386/libfirewire/Makefile stable/11/stand/i386/libfirewire/firewire.c stable/11/stand/i386/libi386/Makefile stable/11/stand/i386/libi386/biospci.c stable/11/stand/i386/libi386/comconsole.c stable/11/stand/i386/libi386/libi386.h stable/11/stand/i386/loader/Makefile stable/11/stand/i386/mbr/Makefile stable/11/stand/i386/pmbr/Makefile stable/11/stand/kshim/bsd_kernel.h stable/11/stand/libsa/Makefile stable/11/stand/libsa/environment.c stable/11/stand/libsa/libsa.3 stable/11/stand/libsa/libstand.3 stable/11/stand/libsa/stand.h stable/11/stand/loader.mk stable/11/stand/mips/beri/boot2/Makefile stable/11/stand/mips/beri/boot2/boot2.c stable/11/stand/mips/beri/loader/Makefile stable/11/stand/mips/beri/loader/exec.c stable/11/stand/mips/uboot/Makefile stable/11/stand/ofw/common/main.c stable/11/stand/ofw/libofw/Makefile stable/11/stand/ofw/libofw/elf_freebsd.c stable/11/stand/ofw/libofw/libofw.h stable/11/stand/ofw/libofw/ofw_copy.c stable/11/stand/ofw/libofw/ofw_memory.c stable/11/stand/ofw/libofw/ppc64_elf_freebsd.c stable/11/stand/powerpc/boot1.chrp/Makefile stable/11/stand/powerpc/kboot/Makefile stable/11/stand/powerpc/kboot/conf.c stable/11/stand/powerpc/kboot/host_syscall.S stable/11/stand/powerpc/kboot/host_syscall.h stable/11/stand/powerpc/kboot/hostdisk.c stable/11/stand/powerpc/kboot/kerneltramp.S stable/11/stand/powerpc/kboot/main.c stable/11/stand/powerpc/kboot/metadata.c stable/11/stand/powerpc/kboot/ppc64_elf_freebsd.c stable/11/stand/powerpc/ofw/Makefile stable/11/stand/powerpc/ofw/ldscript.powerpc stable/11/stand/powerpc/uboot/Makefile stable/11/stand/sparc64/boot1/Makefile stable/11/stand/sparc64/loader/Makefile stable/11/stand/uboot/fdt/Makefile stable/11/stand/uboot/lib/Makefile stable/11/stand/usb/Makefile.test stable/11/stand/usb/bsd_usbloader_test.c stable/11/stand/usb/test/ stable/11/stand/usb/usbcore.mk stable/11/stand/userboot/test/Makefile stable/11/stand/userboot/userboot/Makefile stable/11/stand/zfs/Makefile stable/11/sys/kern/link_elf_obj.c
(In reply to commit-hook from comment #13) For the explicit record for head: -r328835 was also required to avoid the heap and kernel from overlapping. The large MFC (stable/11 -r329183 ) includes -r328835 .
(In reply to commit-hook from comment #14) Has enough evidence accumulated for this bugzilla to status to progress past the "New" status?
To Mark M.: is this PR still relevant?
(In reply to Mark Linimon from comment #17) I've not had access to a powerpc64 or powerpc since before my comment #16 and likely will not for weeks or months. [I do occasionally amd64 -> powerpc* cross-build but this report was tied to run-time behavior.] So, while I expect the problem is fixed, I'm not a source of reasonable direct-evidence at this point.
Fixed, comment #13+