Bug 260127

Created attachment 229803 [details]
dmesg kernel panic with PI7C9X2G304SV enabled

Thanks to the recent addition of various NXP/Freescale QorIQ drivers, particularly qoriq_dw_pci, as well as the upcoming DPAA2 driver by dsl@mcusim.org, FreeBSD is very close to being usable on the Traverse Technologies Ten64 board.

Unfortunately there appears to be an issue enumerating the Diodes (Pericom) PI7C9X2G304SV PCIe switch. This is used on this board to create two PCIe slots from one PCIe 2.0 lane (specifically, the MiniPCIe slots, see https://ten64doc.traverse.com.au/hardware/architecture/)

The panic occurs even when there are no downstream devices connected to the PI7C9X2G304SV.

When the PCIe controller that the PI7C9X2G304SV is connected to is disabled in the FDT, FreeBSD is able to boot and use the PCIe controller without issue, including using an NVMe SSD as the boot device.

Here is the PCIe topology from Linux:
# lspci -nn
0000:00:00.0 PCI bridge [0604]: Freescale Semiconductor Inc Device [1957:80c0] (rev 10)
0001:00:00.0 PCI bridge [0604]: Freescale Semiconductor Inc Device [1957:80c0] (rev 10)
0001:01:00.0 PCI bridge [0604]: Pericom Semiconductor Device [12d8:b304] (rev 01)
0001:02:01.0 PCI bridge [0604]: Pericom Semiconductor Device [12d8:b304] (rev 01)
0001:02:02.0 PCI bridge [0604]: Pericom Semiconductor Device [12d8:b304] (rev 01)
0002:00:00.0 PCI bridge [0604]: Freescale Semiconductor Inc Device [1957:80c0] (rev 10)
0002:01:00.0 Non-Volatile memory controller [0108]: Silicon Motion, Inc. SM2263EN/SM2263XT SSD Controller [126f:2263] (rev 03)

# lspci -t
-+-[0002:00]---00.0-[01-ff]----00.0
 +-[0001:00]---00.0-[01-ff]----00.0-[02-04]--+-01.0-[03]--
 |                                           \-02.0-[04]--
 \-[0000:00]---00.0-[01-ff]--

Note: The PCIe controller instance that the switch is connected to is at 0x3500000. The NVMe SSD is connected to pcie@3600000 (bus 0002:00:00.0 in Linux), pcie@3400000 is connected to the M.2 Key B ("cellular" slot) which is empty in this example.

By manipulating the FDT in U-Boot pcie@3500000 can be disabled allowing FreeBSD to boot:

setenv setup_distroboot_efi 'sf read 0x80001000 0x580000 0x40000 && fsl_mc lazyapply dpl 0x80001000; sf read $fdt_addr_r 0x600000 0x40000 && fdt addr $fdt_addr_r && fdt resize 10000 && fdt boardsetup && fdt set "/soc/spi@20c0000" "status" "disabled" && fdt set "/soc/pcie@3500000" "status" "disabled"'
saveenv
reboot