Created attachment 151991 [details] dmesg log I am using FreeBSD on a rack of Sun X4500 to serve up ZFS space. Each has 4 gigabit ethernet ports, but only 2 are working due to a PCI resource allocation failure. This seems similar to another bug (196501) that has been marked as resolved. I've tried 'hint.agp.0.disabled=1' as described in the bug report, but no difference. I also saw a recommendation about hw.pci.realloc_bars=1, but it also made no difference. I am also using several Sun X4600 compute servers. these have a similar complement of em ports and all 4 of them work with the same kernel. The X4500 supports 48 sata/sas disk drives, so it has quite a few sata controllers on the bus. I suspect that they are taking up enough space to make the allocations for the extra em devices fail, but that's just my humble guess. I've attached my dmesg output. thanks for any help or suggestions. ron.
Can you confirm if this is still an issue for you on 10.2-PRERELEASE?
pciconf -lvbc from mailing list. pciconf -lvcb pcib1 at pci0:0:1:0: class="0x060400" card=0x00000000 chip=0x74581022 rev=0x12 hdr=0x01 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X Bridge' class = bridge subclass = PCI-PCI cap 07[60] = PCI-X 64-bit bridge supports 133MHz cap 08[b8] = HT interrupt cap 08[c0] = HT slave cap 08[f4] = HT MSI address window disabled at 0xfee00000 ioapic0 at pci0:0:1:1: class="0x080010" card=0x74591022 chip=0x74591022 rev=0x12 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X IOAPIC' class = base peripheral subclass = interrupt controller bar [10] = type Memory, range 64, base rxfd2ff000, size 4096, enabled pcib2 at pci0:0:2:0: class="0x060400" card=0x00000000 chip=0x74581022 rev=0x12 hdr=0x01 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X Bridge' class = bridge subclass = PCI-PCI cap 07[60] = PCI-X 64-bit bridge supports 133MHz cap 08[b8] = HT interrupt cap 08[c0] = HT revision ID cap 08[f4] = HT MSI address window disabled at 0xfee00000 ioapic1 at pci0:0:2:1: class="0x080010" card=0x74591022 chip=0x74591022 rev=0x12 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X IOAPIC' class = base peripheral subclass = interrupt controller bar [10] = type Memory, range 64, base rxfd2fe000, size 4096, enabled pcib3 at pci0:0:6:0: class="0x060400" card=0x00000000 chip=0x74601022 rev=0x07 hdr=0x01 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8111 PCI' class = bridge subclass = PCI-PCI cap 08[c0] = HT slave cap 08[f0] = HT interrupt isab0 at pci0:0:7:0: class="0x060100" card=0x74681022 chip=0x74681022 rev=0x05 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8111 LPC' class = bridge subclass = PCI-ISA atapci0 at pci0:0:7:1: class="0x01018a" card=0x74691022 chip=0x74691022 rev=0x03 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8111 IDE' class = mass storage subclass = ATA bar [20] = type I/O Port, range 32, base rxffa0, size 16, enabled none0 at pci0:0:7:2: class="0x0c0500" card=0x746a1022 chip=0x746a1022 rev=0x02 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8111 SMBus 2.0' class = serial bus subclass = SMBus bar [10] = type I/O Port, range 32, base rx1900, size 32, enabled none1 at pci0:0:7:3: class="0x068000" card=0x746b1022 chip=0x746b1022 rev=0x05 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8111 ACPI' class = bridge hostb0 at pci0:0:24:0: class="0x060000" card=0x00000000 chip=0x11001022 rev=0x00 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'K8 [Athlon64/Opteron] HyperTransport Technology Configuration' class = bridge subclass = HOST-PCI cap 08[80] = HT host cap 08[a0] = HT host cap 08[c0] = HT host hostb1 at pci0:0:24:1: class="0x060000" card=0x00000000 chip=0x11011022 rev=0x00 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'K8 [Athlon64/Opteron] Address Map' class = bridge subclass = HOST-PCI hostb2 at pci0:0:24:2: class="0x060000" card=0x00000000 chip=0x11021022 rev=0x00 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'K8 [Athlon64/Opteron] DRAM Controller' class = bridge subclass = HOST-PCI hostb3 at pci0:0:24:3: class="0x060000" card=0x00000000 chip=0x11031022 rev=0x00 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'K8 [Athlon64/Opteron] Miscellaneous Control' class = bridge subclass = HOST-PCI hostb4 at pci0:0:25:0: class="0x060000" card=0x00000000 chip=0x11001022 rev=0x00 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'K8 [Athlon64/Opteron] HyperTransport Technology Configuration' class = bridge subclass = HOST-PCI cap 08[80] = HT host cap 08[a0] = HT host cap 08[c0] = HT host hostb5 at pci0:0:25:1: class="0x060000" card=0x00000000 chip=0x11011022 rev=0x00 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'K8 [Athlon64/Opteron] Address Map' class = bridge subclass = HOST-PCI hostb6 at pci0:0:25:2: class="0x060000" card=0x00000000 chip=0x11021022 rev=0x00 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'K8 [Athlon64/Opteron] DRAM Controller' class = bridge subclass = HOST-PCI hostb7 at pci0:0:25:3: class="0x060000" card=0x00000000 chip=0x11031022 rev=0x00 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'K8 [Athlon64/Opteron] Miscellaneous Control' class = bridge subclass = HOST-PCI mvs0 at pci0:1:1:0: class="0x010000" card=0x11ab11ab chip=0x608111ab rev=0x09 hdr=0x00 vendor = 'Marvell Technology Group Ltd.' device = 'MV88SX6081 8-port SATA II PCI-X Controller' class = mass storage subclass = SCSI bar [10] = type Memory, range 64, base rxfae00000, size 1048576, enabled bar [18] = type I/O Port, range 32, base rx7c00, size 256, enabled cap 01[40] = powerspec 2 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit cap 07[60] = PCI-X 64-bit supports 133MHz, 512 burst read, 4 split transactions mvs1 at pci0:2:1:0: class="0x010000" card=0x11ab11ab chip=0x608111ab rev=0x09 hdr=0x00 vendor = 'Marvell Technology Group Ltd.' device = 'MV88SX6081 8-port SATA II PCI-X Controller' class = mass storage subclass = SCSI bar [10] = type Memory, range 64, base rxfb000000, size 1048576, enabled bar [18] = type I/O Port, range 32, base rx8c00, size 256, enabled cap 01[40] = powerspec 2 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit cap 07[60] = PCI-X 64-bit supports 133MHz, 512 burst read, 4 split transactions ohci0 at pci0:3:0:0: class="0x0c0310" card=0x74641022 chip=0x74641022 rev=0x0b hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8111 USB OHCI' class = serial bus subclass = USB bar [10] = type Memory, range 32, base rxfd1fe000, size 4096, enabled ohci1 at pci0:3:0:1: class="0x0c0310" card=0x74641022 chip=0x74641022 rev=0x0b hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8111 USB OHCI' class = serial bus subclass = USB bar [10] = type Memory, range 32, base rxfd1fd000, size 4096, enabled vgapci0 at pci0:3:3:0: class="0x030000" card=0x4531108e chip=0x47521002 rev=0x27 hdr=0x00 vendor = 'Advanced Micro Devices [AMD] nee ATI' device = 'Rage XL' class = display subclass = VGA bar [10] = type Memory, range 32, base rxfc000000, size 16777216, enabled bar [14] = type I/O Port, range 32, base rx9800, size 256, enabled bar [18] = type Memory, range 32, base rxfd1ff000, size 4096, enabled cap 01[5c] = powerspec 2 supports D0 D1 D2 D3 current D0 ohci2 at pci0:3:4:0: class="0x0c0310" card=0x00351033 chip=0x00351033 rev=0x43 hdr=0x00 vendor = 'NEC Corporation' device = 'USB' class = serial bus subclass = USB bar [10] = type Memory, range 32, base rxfd1fc000, size 4096, enabled cap 01[40] = powerspec 2 supports D0 D1 D2 D3 current D0 ohci3 at pci0:3:4:1: class="0x0c0310" card=0x00351033 chip=0x00351033 rev=0x43 hdr=0x00 vendor = 'NEC Corporation' device = 'USB' class = serial bus subclass = USB bar [10] = type Memory, range 32, base rxfd1fb000, size 4096, enabled cap 01[40] = powerspec 2 supports D0 D1 D2 D3 current D0 ehci0 at pci0:3:4:2: class="0x0c0320" card=0x00e01033 chip=0x00e01033 rev=0x04 hdr=0x00 vendor = 'NEC Corporation' device = 'USB 2.0' class = serial bus subclass = USB bar [10] = type Memory, range 32, base rxfd1fac00, size 256, enabled cap 01[40] = powerspec 2 supports D0 D1 D2 D3 current D0 pcib5 at pci0:4:3:0: class="0x060400" card=0x00000000 chip=0x74581022 rev=0x12 hdr=0x01 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X Bridge' class = bridge subclass = PCI-PCI cap 07[60] = PCI-X 64-bit bridge supports 133MHz cap 08[b8] = HT interrupt cap 08[c0] = HT slave cap 08[f4] = HT MSI address window disabled at 0xfee00000 ioapic2 at pci0:4:3:1: class="0x080010" card=0x74591022 chip=0x74591022 rev=0x12 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X IOAPIC' class = base peripheral subclass = interrupt controller bar [10] = type Memory, range 64, base rxfdcff000, size 4096, enabled pcib6 at pci0:4:4:0: class="0x060400" card=0x00000000 chip=0x74581022 rev=0x12 hdr=0x01 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X Bridge' class = bridge subclass = PCI-PCI cap 07[60] = PCI-X 64-bit bridge supports 133MHz cap 08[b8] = HT interrupt cap 08[c0] = HT revision ID cap 08[f4] = HT MSI address window disabled at 0xfee00000 ioapic3 at pci0:4:4:1: class="0x080010" card=0x74591022 chip=0x74591022 rev=0x12 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X IOAPIC' class = base peripheral subclass = interrupt controller bar [10] = type Memory, range 64, base rxfdcfe000, size 4096, enabled pcib7 at pci0:4:5:0: class="0x060400" card=0x00000000 chip=0x74581022 rev=0x12 hdr=0x01 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X Bridge' class = bridge subclass = PCI-PCI cap 07[60] = PCI-X 64-bit bridge supports 133MHz cap 08[b8] = HT interrupt cap 08[c0] = HT slave cap 08[f4] = HT MSI address window disabled at 0xfee00000 ioapic4 at pci0:4:5:1: class="0x080010" card=0x74591022 chip=0x74591022 rev=0x12 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X IOAPIC' class = base peripheral subclass = interrupt controller bar [10] = type Memory, range 64, base rxfdcfd000, size 4096, enabled pcib8 at pci0:4:6:0: class="0x060400" card=0x00000000 chip=0x74581022 rev=0x12 hdr=0x01 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X Bridge' class = bridge subclass = PCI-PCI cap 07[60] = PCI-X 64-bit bridge supports 133MHz cap 08[b8] = HT interrupt cap 08[c0] = HT revision ID cap 08[f4] = HT MSI address window disabled at 0xfee00000 ioapic5 at pci0:4:6:1: class="0x080010" card=0x74591022 chip=0x74591022 rev=0x12 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X IOAPIC' class = base peripheral subclass = interrupt controller bar [10] = type Memory, range 64, base rxfdcfc000, size 4096, enabled mvs2 at pci0:5:1:0: class="0x010000" card=0x11ab11ab chip=0x608111ab rev=0x09 hdr=0x00 vendor = 'Marvell Technology Group Ltd.' device = 'MV88SX6081 8-port SATA II PCI-X Controller' class = mass storage subclass = SCSI bar [10] = type Memory, range 64, base rxfd700000, size 1048576, enabled bar [18] = type I/O Port, range 32, base rxac00, size 256, enabled cap 01[40] = powerspec 2 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit cap 07[60] = PCI-X 64-bit supports 133MHz, 512 burst read, 4 split transactions mvs3 at pci0:6:1:0: class="0x010000" card=0x11ab11ab chip=0x608111ab rev=0x09 hdr=0x00 vendor = 'Marvell Technology Group Ltd.' device = 'MV88SX6081 8-port SATA II PCI-X Controller' class = mass storage subclass = SCSI bar [10] = type Memory, range 64, base rxfd900000, size 1048576, enabled bar [18] = type I/O Port, range 32, base rxbc00, size 256, enabled cap 01[40] = powerspec 2 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit cap 07[60] = PCI-X 64-bit supports 133MHz, 512 burst read, 4 split transactions em0 at pci0:7:1:0: class="0x020000" card=0x10118086 chip=0x10108086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' device = '82546EB Gigabit Ethernet Controller (Copper)' class = network subclass = ethernet bar [10] = type Memory, range 64, base rxfdae0000, size 131072, enabled bar [20] = type I/O Port, range 32, base rxcc00, size 64, enabled cap 01[dc] = powerspec 2 supports D0 D3 current D0 cap 07[e4] = PCI-X 64-bit supports 133MHz, 2048 burst read, 1 split transaction cap 05[f0] = MSI supports 1 message, 64 bit em1 at pci0:7:1:1: class="0x020000" card=0x10118086 chip=0x10108086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' device = '82546EB Gigabit Ethernet Controller (Copper)' class = network subclass = ethernet bar [10] = type Memory, range 64, base rxfdac0000, size 131072, enabled bar [20] = type I/O Port, range 32, base rxc800, size 64, enabled cap 01[dc] = powerspec 2 supports D0 D3 current D0 cap 07[e4] = PCI-X 64-bit supports 133MHz, 2048 burst read, 1 split transaction cap 05[f0] = MSI supports 1 message, 64 bit none2 at pci0:8:1:0: class="0x020000" card=0x10118086 chip=0x10108086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' device = '82546EB Gigabit Ethernet Controller (Copper)' class = network subclass = ethernet bar [10] = type Memory, range 64, base rxfdbe0000, size 131072, enabled bar [20] = type I/O Port, range 32, base rxdc00, size 64, disabled cap 01[dc] = powerspec 2 supports D0 D3 current D0 cap 07[e4] = PCI-X 64-bit supports 133MHz, 2048 burst read, 1 split transaction cap 05[f0] = MSI supports 1 message, 64 bit none3 at pci0:8:1:1: class="0x020000" card=0x10118086 chip=0x10108086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' device = '82546EB Gigabit Ethernet Controller (Copper)' class = network subclass = ethernet bar [10] = type Memory, range 64, base rxfdbc0000, size 131072, enabled bar [20] = type I/O Port, range 32, base rxd800, size 64, disabled cap 01[dc] = powerspec 2 supports D0 D3 current D0 cap 07[e4] = PCI-X 64-bit supports 133MHz, 2048 burst read, 1 split transaction cap 05[f0] = MSI supports 1 message, 64 bit pcib10 at pci0:12:9:0: class="0x060400" card=0x00000000 chip=0x74581022 rev=0x12 hdr=0x01 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X Bridge' class = bridge subclass = PCI-PCI cap 07[60] = PCI-X 64-bit bridge supports 133MHz cap 08[b8] = HT interrupt cap 08[c0] = HT slave cap 08[f4] = HT MSI address window disabled at 0xfee00000 ioapic6 at pci0:12:9:1: class="0x080010" card=0x74591022 chip=0x74591022 rev=0x12 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X IOAPIC' class = base peripheral subclass = interrupt controller bar [10] = type Memory, range 64, base rxfeaff000, size 4096, enabled pcib11 at pci0:12:10:0: class="0x060400" card=0x00000000 chip=0x74581022 rev=0x12 hdr=0x01 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X Bridge' class = bridge subclass = PCI-PCI cap 07[60] = PCI-X 64-bit bridge supports 133MHz cap 08[b8] = HT interrupt cap 08[c0] = HT revision ID cap 08[f4] = HT MSI address window disabled at 0xfee00000 ioapic7 at pci0:12:10:1: class="0x080010" card=0x74591022 chip=0x74591022 rev=0x12 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X IOAPIC' class = base peripheral subclass = interrupt controller bar [10] = type Memory, range 64, base rxfeafe000, size 4096, enabled pcib13 at pci0:9:7:0: class="0x060400" card=0x00000000 chip=0x74581022 rev=0x12 hdr=0x01 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X Bridge' class = bridge subclass = PCI-PCI cap 07[60] = PCI-X 64-bit bridge supports 133MHz cap 08[b8] = HT interrupt cap 08[c0] = HT slave cap 08[f4] = HT MSI address window disabled at 0xfee00000 ioapic8 at pci0:9:7:1: class="0x080010" card=0x74591022 chip=0x74591022 rev=0x12 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X IOAPIC' class = base peripheral subclass = interrupt controller bar [10] = type Memory, range 64, base rxfe6ff000, size 4096, enabled pcib14 at pci0:9:8:0: class="0x060400" card=0x00000000 chip=0x74581022 rev=0x12 hdr=0x01 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X Bridge' class = bridge subclass = PCI-PCI cap 07[60] = PCI-X 64-bit bridge supports 133MHz cap 08[b8] = HT interrupt cap 08[c0] = HT revision ID cap 08[f4] = HT MSI address window disabled at 0xfee00000 ioapic9 at pci0:9:8:1: class="0x080010" card=0x74591022 chip=0x74591022 rev=0x12 hdr=0x00 vendor = 'Advanced Micro Devices [AMD]' device = 'AMD-8132 PCI-X IOAPIC' class = base peripheral subclass = interrupt controller bar [10] = type Memory, range 64, base rxfe6fe000, size 4096, enabled mvs4 at pci0:10:1:0: class="0x010000" card=0x11ab11ab chip=0x608111ab rev=0x09 hdr=0x00 vendor = 'Marvell Technology Group Ltd.' device = 'MV88SX6081 8-port SATA II PCI-X Controller' class = mass storage subclass = SCSI bar [10] = type Memory, range 64, base rxfe300000, size 1048576, enabled bar [18] = type I/O Port, range 32, base rxec00, size 256, enabled cap 01[40] = powerspec 2 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit cap 07[60] = PCI-X 64-bit supports 133MHz, 512 burst read, 4 split transactions mvs5 at pci0:11:1:0: class="0x010000" card=0x11ab11ab chip=0x608111ab rev=0x09 hdr=0x00 vendor = 'Marvell Technology Group Ltd.' device = 'MV88SX6081 8-port SATA II PCI-X Controller' class = mass storage subclass = SCSI bar [10] = type Memory, range 64, base rxfe500000, size 1048576, enabled bar [18] = type I/O Port, range 32, base rxfc00, size 256, disabled cap 01[40] = powerspec 2 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message, 64 bit cap 07[60] = PCI-X 64-bit supports 133MHz, 512 burst read, 4 split transactions
This seems to be a PCI problem from what I am seeing. pcib8: <ACPI PCI-PCI bridge> at device 6.0 on pci4 pcib8: failed to allocate initial I/O port window: 0xd000-0xdfff pci8: <ACPI PCI bus> on pcib8 em2: <Intel(R) PRO/1000 Legacy Network Connection 1.0.6> mem 0xfdbe0000-0xfdbfffff irq 61 at device 1.0 on pci8 em2: 0x40 bytes of rid 0x20 res 4 failed (0, 0xffffffffffffffff). em2: Unable to allocate bus resource: ioport em2: Allocation of PCI resources failed device_attach: em2 attach returned 6 em2: <Intel(R) PRO/1000 Legacy Network Connection 1.0.6> mem 0xfdbc0000-0xfdbdffff irq 62 at device 1.1 on pci8 em2: 0x40 bytes of rid 0x20 res 4 failed (0, 0xffffffffffffffff). em2: Unable to allocate bus resource: ioport em2: Allocation of PCI resources failed device_attach: em2 attach returned 6 none2 at pci0:8:1:0: class="0x020000" card=0x10118086 chip=0x10108086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' device = '82546EB Gigabit Ethernet Controller (Copper)' class = network subclass = ethernet bar [10] = type Memory, range 64, base rxfdbe0000, size 131072, enabled bar [20] = type I/O Port, range 32, base rxdc00, size 64, disabled cap 01[dc] = powerspec 2 supports D0 D3 current D0 cap 07[e4] = PCI-X 64-bit supports 133MHz, 2048 burst read, 1 split transaction cap 05[f0] = MSI supports 1 message, 64 bit none3 at pci0:8:1:1: class="0x020000" card=0x10118086 chip=0x10108086 rev=0x03 hdr=0x00 vendor = 'Intel Corporation' device = '82546EB Gigabit Ethernet Controller (Copper)' class = network subclass = ethernet bar [10] = type Memory, range 64, base rxfdbc0000, size 131072, enabled bar [20] = type I/O Port, range 32, base rxd800, size 64, disabled cap 01[dc] = powerspec 2 supports D0 D3 current D0 cap 07[e4] = PCI-X 64-bit supports 133MHz, 2048 burst read, 1 split transaction cap 05[f0] = MSI supports 1 message, 64 bit
As a shot in the dark, can you add this to your kernel and see if it helps? sys/dev/pci % svn diff Index: pci.c =================================================================== --- pci.c (revision 286233) +++ pci.c (working copy) @@ -247,10 +247,11 @@ { 0x35808086, PCI_QUIRK_DISABLE_MSI, 0, 0 }, /* - * MSI doesn't work with devices behind the AMD 8131 HT-PCIX + * MSI doesn't work with devices behind the AMD 8131/8132 HT-PCIX * bridge. */ { 0x74501022, PCI_QUIRK_DISABLE_MSI, 0, 0 }, + { 0x74521022, PCI_QUIRK_DISABLE_MSI, 0, 0 }, /* * MSI-X allocation doesn't work properly for devices passed through
This is not caused by MSI. The root issue is that the PCI bus only pre-reserves BARs for leaf devices, not the I/O windows of PCI-PCI bridges. The agp0 device probes before pcibX and it allocates a random page of physical address space that overlaps with the PCI-PCI bridge window. That then causes a conflict with reserving the PCI-PCI window. I have another PR I can't find at the moment where I've discussed the possible solutions. The most expedient is probably to have the PCI bus pre-reserve the resources for memory windows (I/O windows are much more complicated, but memory windows would be good enough for all the instances of this bug that I've seen so far). Also, this is not a regression in 10.2. I believe it has been an issue since NEW_PCIB was added, so 10.0 (and in older branches with the option enabled).
Just a quick comment that this indeed seems to be a problem in 10.2-RELEASE also. I tried the MSI-quirks hack described above but it makes no difference. In my case (also using Sun Fire X4500 - Thumper) the following devices seems to be affected: > pcib8: <ACPI PCI-PCI bridge> at device 6.0 on pci4 > pcib8: failed to allocate initial I/O port window: 0xd000-0xdfff > em2: <Intel(R) PRO/1000 Legacy Network Connection 1.0.6> mem 0xfdbe0000-0xfdbfffff irq 61 at device 1.0 on pci8 > em2: 0x40 bytes of rid 0x20 res 4 failed (0, 0xffffffffffffffff). > em2: Unable to allocate bus resource: ioport > em2: Allocation of PCI resources failed > device_attach: em2 attach returned 6 > em2: <Intel(R) PRO/1000 Legacy Network Connection 1.0.6> mem 0xfdbc0000-0xfdbdffff irq 62 at device 1.1 on pci8 > em2: 0x40 bytes of rid 0x20 res 4 failed (0, 0xffffffffffffffff). > em2: Unable to allocate bus resource: ioport > em2: Allocation of PCI resources failed > device_attach: em2 attach returned 6 > pcib14: <ACPI PCI-PCI bridge> at device 8.0 on pci9 > pcib14: failed to allocate initial I/O port window: 0xf000-0xffff > ppc0: cannot reserve I/O port range (the ppc0 is not really a problem though - doubt there even is a parallel printer port :-) - Peter
Yes, the MSI quirk will not help. Disabling agp0 might help as a workaround. I don't currently have a patch to test for my other suggestion of pre-reserving memory windows.
Created attachment 163445 [details] dmesg output from IFM server
Disabling the agp driver in the kernel config and rebuilding a new kernel (and rebooting) doesn't seem to help either (see attached "ifm-dmesg-2015-11-23.log" file). (In reply to John Baldwin from comment #7)
Hi all! Any news about this problem? I have same server X4500 and same problem.
I finally gave up on trying to use FreeBSD on the Sun Fire X4500 servers - due to this bug and the fact that the machine(s) would panic and reboot when put under non-trivial amounts of disk I/O. We switched to OmniOS for the X4500:s which works flawlessly (not surprisingly though since that is based on the OpenSolaris code base).
For bugs matching the following conditions: - Status == In Progress - Assignee == "bugs@FreeBSD.org" - Last Modified Year <= 2017 Do - Set Status to "Open"