Bug 228768
| Summary: | EARLY_AP_STARTUP causes panic on amd machines | ||||||
|---|---|---|---|---|---|---|---|
| Product: | Base System | Reporter: | Roger Hammerstein <cheeky.m> | ||||
| Component: | kern | Assignee: | John Baldwin <jhb> | ||||
| Status: | Closed FIXED | ||||||
| Severity: | Affects Only Me | CC: | jhb, kib, mjg, ota, pablo.ruiz | ||||
| Priority: | --- | Keywords: | patch | ||||
| Version: | CURRENT | Flags: | jhb:
mfc-stable12+
jhb: mfc-stable11+ |
||||
| Hardware: | amd64 | ||||||
| OS: | Any | ||||||
| Attachments: |
|
||||||
|
Description
Roger Hammerstein
2018-06-05 17:33:11 UTC
This issue with SMP and EARLY_AP_STARTUP continues with 12.0-BETA2 r339840.
Here is the latest console output as it double-faults and reboots
How can I help ? What other information could I give?
Table 'FACP' at 0xdfff0290
Table 'APIC' at 0xdfff0390
Table 'SPCR' at 0xdfff0420
Table 'SLIT' at 0xdfff0470
SLIT: Found table at 0xdfff0470
SLIT.Localities: 2
0: 10 10
1: 10 10
PPIM 0: PA=0xa0000, VA=0xffffffff82810000, size=0x10000, mode=0
PPIM 1: PA=0x11ffe7000, VA=0xffffffff82820000, size=0x1000, mode=0x6
pmap: large map 8 PML4 slots (4096 Gb)
VT(vga): resolution 640x480
Preloaded elf kernel "/boot/kernel/kernel" at 0xffffffff82638000.
Preloaded boot_entropy_cache "/boot/entropy" at 0xffffffff826411f8.
Table 'FACP' at 0xdfff0290
FACP: Found table at 0xdfff0290
Calibrating TSC clock ... TSC clock: 2593156801 Hz
CPU: Dual-Core AMD Opteron(tm) Processor 2218 (2593.16-MHz K8-class CPU)
Origin="AuthenticAMD" Id=0x40f12 Family=0xf Model=0x41 Stepping=2
Features=0x178bfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT>
Features2=0x2001<SSE3,CX16>
AMD Features=0xea500800<SYSCALL,NX,MMX+,FFXSR,RDTSCP,LM,3DNow!+,3DNow!>
AMD Features2=0x1f<LAHF,CMP,SVM,ExtAPIC,CR8>
SVM: Features=0x0
Revision=1, ASIDs=64
L1 2MB data TLB: 8 entries, fully associative
L1 2MB instruction TLB: 8 entries, fully associative
L1 4KB data TLB: 32 entries, fully associative
L1 4KB instruction TLB: 32 entries, fully associative
L1 data cache: 64 kbytes, 64 bytes/line, 1 lines/tag, 2-way associative
L1 instruction cache: 64 kbytes, 64 bytes/line, 1 lines/tag, 2-way associative
L2 2MB unified TLB: 0 entries, disabled/not present
L2 4KB data TLB: 512 entries, 4-way associative
L2 4KB instruction TLB: 512 entries, 4-way associative
L2 unified cache: 1024 kbytes, 64 bytes/line, 1 lines/tag, 16-way associative
real memory = 4563402752 (4352 MB)
Physical memory chunk(s):
0x0000000000010000 - 0x000000000009bfff, 573440 bytes (140 pages)
0x0000000000103000 - 0x00000000001fffff, 1036288 bytes (253 pages)
0x0000000002800000 - 0x00000000d8a67fff, 3592847360 bytes (877160 pages)
0x0000000100000000 - 0x000000011ffe5fff, 536764416 bytes (131046 pages)
avail memory = 4112728064 (3922 MB)
MADT: Found CPU APIC ID 0 ACPI ID 1: enabled
SMP: Added CPU 0 (AP)
MADT: Found CPU APIC ID 1 ACPI ID 2: enabled
SMP: Added CPU 1 (AP)
MADT: Found CPU APIC ID 2 ACPI ID 3: enabled
SMP: Added CPU 2 (AP)
MADT: Found CPU APIC ID 3 ACPI ID 4: enabled
SMP: Added CPU 3 (AP)
Event timer "LAPIC" quality 100
LAPIC: ipi_wait() us multiplier 72 (r 3600094 tsc 2593156801)
ACPI APIC Table: <SUN X4200 M2>
Package ID shift: 1
L2 cache ID shift: 0
L1 cache ID shift: 0
Core ID shift: 0
INTR: Adding local APIC 1 as a target
INTR: Adding local APIC 2 as a target
INTR: Adding local APIC 3 as a target
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 2 package(s) x 2 core(s)
Package HW ID = 0
Core HW ID = 0
CPU0 (BSP): APIC ID: 0
Core HW ID = 1
CPU1 (AP): APIC ID: 1
Package HW ID = 1
Core HW ID = 2
CPU2 (AP): APIC ID: 2
Core HW ID = 3
CPU3 (AP): APIC ID: 3
APIC: CPU 0 has ACPI ID 1
APIC: CPU 1 has ACPI ID 2
APIC: CPU 2 has ACPI ID 3
APIC: CPU 3 has ACPI ID 4
Pentium Pro MTRR support enabled
x86bios: IVT 0x000000-0x0004ff at 0xfffff80000000000
x86bios: SSEG 0x098000-0x098fff at 0xfffffe0000614000
x86bios: EBDA 0x09d000-0x09ffff at 0xfffff8000009d000
x86bios: ROM 0x0a0000-0x0fefff at 0xfffff800000a0000
SRAT: CPU 0 has memory domain 0
SRAT: CPU 1 has memory domain 0
SRAT: CPU 2 has memory domain 1
SRAT: CPU 3 has memory domain 1
random: read 3840 bytes from preloaded cache
random: unblocking device.
arc4random: read 32 bytes from preloaded cache
VIMAGE (virtualized network stack) enabled
ULE: setup cpu 0
ULE: setup cpu 1
ULE: setup cpu 2
ULE: setup cpu 3
ACPI: RSDP 0x00000000000F9470 000024 (v02 SUN )
ACPI: XSDT 0x00000000DFFF0100 00006C (v01 SUN X4200 M2 00000039 MSFT 00000097)
ACPI: FACP 0x00000000DFFF0290 0000F4 (v03 SUN X4200 M2 00000039 MSFT 00000097)
Firmware Warning (ACPI): 32/64X length mismatch in FADT/Gpe0Block: 64/32 (20181003/tbfadt-748)
Firmware Warning (ACPI): 32/64X length mismatch in FADT/Gpe1Block: 128/64 (20181003/tbfadt-748)
ACPI: DSDT 0x00000000DFFF04A0 005F64 (v01 SUN X4200 M1 00000039 INTL 20050624)
ACPI: FACS 0x00000000DFFFE000 000040
ACPI: APIC 0x00000000DFFF0390 000086 (v01 SUN X4200 M2 00000039 MSFT 00000097)
ACPI: SPCR 0x00000000DFFF0420 000050 (v01 SUN X4200 M2 00000039 MSFT 00000097)
ACPI: SLIT 0x00000000DFFF0470 000030 (v01 SUN OEMSLIT 00000039 MSFT 00000097)
ACPI: OEMB 0x00000000DFFFE040 000063 (v01 SUN X4200 M2 00000039 MSFT 00000097)
ACPI: HPET 0x00000000DFFF6410 000038 (v01 SUN OEMHPET0 00000039 MSFT 00000097)
ACPI: IPET 0x00000000DFFF6450 000038 (v01 SUN OEMHPET1 00000039 MSFT 00000097)
ACPI: SRAT 0x00000000DFFF6490 000110 (v01 AMD HAMMER 00000001 AMD 00000001)
ACPI: SSDT 0x00000000DFFF65A0 0004F0 (v01 A M I POWERNOW 00000001 AMD 00000001)
MADT: Found IO APIC ID 15, Interrupt 0 at 0xfec00000
ioapic0: ver 0x11 maxredir 0x17
ioapic0: Routing external 8259A's -> intpin 0
MADT: Found IO APIC ID 16, Interrupt 48 at 0xfeafd000
ioapic1: Changing APIC ID to 16
ioapic1: WARNING: intbase 48 != expected base r24
ioapic1: ver 0x11 maxredir 0x06
MADT: Found IO APIC ID 17, Interrupt 56 at 0xfeafc000
ioapic2: Changing APIC ID to 17
ioapic2: WARNING: intbase 56 != expected base r55
ioapic2: ver 0x11 maxredir 0x06
MADT: Found IO APIC ID 14, Interrupt 24 at 0xfeaff000
ioapic3: WARNING: intbase 24 != expected base r63
ioapic3: ver 0x11 maxredir 0x17
MADT: Interrupt override: source 9, irq 9
ioapic0: intpin 9 trigger: level
ioapic3 <Version 1.1> irqs 24-47 on motherboard
ioapic0 <Version 1.1> irqs 0-23 on motherboard
ioapic1 <Version 1.1> irqs 48-54 on motherboard
ioapic2 <Version 1.1> irqs 56-62 on motherboard
lapic: Divisor 2, Frequency 99737098 Hz
cpu0 BSP:
ID: 0x00000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff
lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
AMD ext features: 0x00010003
AMD elvt0: 0x00010000
SMP: AP CPU k#3er nLaelu ntrackph eder1!2 n
t2 s hc pi untr3ap AteP1r:rup
dwi i tsah bin led
l 0t aDni:c:r 0rupx0m30_fault0t 0_h0odlisd:a faul Vtble oRn n:ofdau
e x
8ry,
005Faa0tddalr: 0x 0tra10 fLDffp f1f2fffR81:fc : 0x00p07000a
0ge 0fcpu0au00 DltidF = 2R
fff i mxef w=h ffifl1fe
i nl
K iDBnt: s0:tk ack baerc0kxn0t00rae10cl7 e0:0
0int1dpe:
010pumide: 1 0s=
5x 0A0; utomat4ipicc ir00edo oTtP i=R: n 0011
e 0fas0ecu0ol0ndst v -i000 r prtSVeRsus: a al k 0eyx ao00n0dd0 1frthfeess co= nsol0
to x2 abtimort4
er
e : f0ax000100euf thelrm: t0x000 1000c0 err: 0xo00d0000f0 pmc: 0x000104Re0
b oot iAngM.D .=ex.
[reboots]
--------
a second boot's panic, showing more interleaved-like text
cpu0 BSP:
ID: 0x00000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff
lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff
timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400
AMD ext features: 0x00010003
AMD elvt0: 0x00010000
SMP: AP CPU
erkkkernkkkkkkk
neFla ttarla p 12 dwitho uibntlerreupt sf daiusablledt
r
i
Fpata l 0txrafp f12:f pfagfe ffauflt fw8h0ib17ee in1 keernel8 mod e
rcspuid p= 1 ; 0axpfififc
efradulri t0vi xfffvisrffotura r8e0al080d in05 4as0d00d
rrt8 resuct0sxi5o r =9 0xn0f,x 4ff0p
ata, potfufe8l1tc fcc7478toiodne vr=1 0siolup ervis0xao2r t0iread od
a1 nr
c 1ige 0 rno1t 2prenss e0txrntuf
instrfutcfitonfi foen0po 0p0io0i60n0nterter 080= = 0r013xx 22000x:f0f:xfff0fxffff8f0ff0f8ff0f0ff8514fff008c00b77e6
4r178bc
4
s s0x1ta ckr tapcko1inte 5rp 0x ff ffo f800 0i31d en58=te0r 0rxf2la8gs: 0 x01x f0 ff f0f4e6 00=0
cs 002xa028xc9:c02
f s=m0e ffpfofiesn0s t00x028 2dse ra 07 0x2e8 e s 000x2 8
f 0 xr2a80 gsx 2m80:0x2x8
feffff posbiase fe0n g0sba00ste2 0ea0xfc9cr 0
cfode f ff s fefg mef n8t2 = 0=b0ase f20 0x00 0k,g lsxib2am8s:i0te x 0ffx0fxfffffffffff, ffftf8y1ffpe ffc8020000xf2190b
cod
in e= DcPpLuis 0e,dgm preens= t 13, =l;o ngabp ica1s ,ie d0x de0f,3=2 0 l 0,i g3mra
t1 p0xa
pfnffriffco: dcoeubsles foa rul etfyplate
=3ug id0 =sx1b
r e
LI 0,O tisu meme= = DP, 1
b KPDB: L spres= 1,t aclkong 0
f32acckt uraced:
e U 0p,ti gmrer:an 1s
Aut om1
Lpratto piroccc erebesoostso i nr 15 sef=ela congsd1 s= 1- press (iare dlek:s eumcpeyu ,o InO )P
ons=he ap0c n
courlmree ntbot praeborrotc
es -s-> =P res1s2 a = 11
p ke( onan tihec: pageifacuonsdoll
ec:to rp ebicopot,u 21)tim
=
-tr
y: KDap noum sbwesttcrck b o f=c ktt a1e2e
stem
nnic:ime: 1s
agReb ooting.f..
Hmm, I thought had sent a mail about this earlier, but I guess not. Are you able to put DDB in your kernel config and get a db> prompt at all? If so, a stack trace would be really helpful. If not, one thing to try first that might help is to get down from 4 CPUs to 2 CPUs. You can set these hints at the loader prompt (or in loader.conf) to disable CPUs 2 and 3: hint.lapic.2.disabled=1 hint.lapic.3.disabled=1 (In reply to John Baldwin from comment #2) With DDB and WITNESS, MADT: Interrupt override: source 9, irq 9 ioapic0: intpin 9 trigger: level ioapic3 <Version 1.1> irqs 24-47 on motherboard ioapic0 <Version 1.1> irqs 0-23 on motherboard ioapic1 <Version 1.1> irqs 48-54 on motherboard ioapic2 <Version 1.1> irqs 56-62 on motherboard lapic: Divisor 2, Frequency 99737090 Hz cpu0 BSP: ID: 0x00000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 AMD ext features: 0x00010003 AMD elvt0: 0x00010000 SMP: AP CPU FtKernel page fault with the following non-sleepable locks held: exclusive spin mutex clk (clk) r = 0 (0xffffffff81e70d78) locked @ /usr/src/sys/x86/isa/clock.c:209 exclusive spin mutex ap boot (ap boot) r = 0 (0xffffffff81fc4788) locked @ /usr/src/sys/x86/x86/mp_x86.c:1013 stack backtrace: Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 01 fault virtual address = 0xfffff80080054000 fault code = supervisor read instruction, protection violation instruction pointer = 0x20:0xfffff80080054000 stack pointer = 0x28:0xfffffe0002a8f8b0 frame pointer = 0x28:0x0 code segment = base rx0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 11 (idle: cpu1) [ thread pid 11 tid 100004 ] Stopped at 0xfffff80080054000 db> trace Tracing pid 11 tid 100004 td 0xfffff80080054580 ??() at 0xfffff80080054000 db> Any other commands to run? db> show locks exclusive spin mutex clk (clk) r = 0 (0xffffffff81e70d78) locked @ /usr/src/sys/x86/isa/clock.c:209 exclusive spin mutex ap boot (ap boot) r = 0 (0xffffffff81fc4788) locked @ /usr/src/sys/x86/x86/mp_x86.c:1013 db> db> show alllocks Process 0 (kernel) thread 0xffffffff81e8a3e0 (100000) exclusive sleep mutex Giant (Giant) r = 0 (0xffffffff818020c0) locked @ /usr/src/sys/kern/kern_module.c:116 db> db> bt Tracing pid 11 tid 100004 td 0xfffff80080054580 ??() at 0xfffff80080054000 db> db> ps pid ppid pgrp uid state wmesg wchan cmd 12 0 0 0 WL (threaded) [intr] 100008 I [swi6: task queue] 100009 I [swi6: Giant taskq] 100011 I [swi5: fast taskq] 11 0 0 0 RL (threaded) [idle] 100003 CanRun [idle: cpu0] 100004 CanRun [idle: cpu1] 100005 CanRun [idle: cpu2] 100006 CanRun [idle: cpu3] 1 0 0 0 ?L [kernel] 10 0 0 0 RL [audit] 0 0 0 0 RLs (threaded) [kernel] 100000 Run CPU 0 [swapper] 100007 RunQ [aiod_kick taskq] 100010 RunQ [thread taskq] 100012 RunQ [config_0] 100013 RunQ [kqueue_ctx taskq] db> db> show pcpu cpuid = 1 dynamic pcpu = 0xfffffe0081100ec0 curthread = 0xfffff80080054580: pid 11 tid 100004 "idle: cpu1" curpcb = 0 fpcurthread = none idlethread = 0xfffff80080054580: tid 100004 "idle: cpu1" curpmap = 0xffffffff81e8aaa8 tssp = 0xffffffff81fb5e88 commontssp = 0xffffffff81fb5e88 rsp0 = 0x0 gs32p = 0xffffffff81fbcac0 ldt = 0xffffffff81fbcb00 tss = 0xffffffff81fbcaf0 curvnet = 0 spin locks held: exclusive spin mutex clk (clk) r = 0 (0xffffffff81e70d78) locked @ /usr/src/sys/x86/isa/clock.c:209 exclusive spin mutex ap boot (ap boot) r = 0 (0xffffffff81fc4788) locked @ /usr/src/sys/x86/x86/mp_x86.c:1013 db> db> show reg cs 0x20 ds 0x28 ll+0x7 es 0x28 ll+0x7 fs 0x28 ll+0x7 gs 0x28 ll+0x7 ss 0x28 ll+0x7 rax 0xfffff80080054580 rcx 0x5a8 ll+0x587 rdx 0xffffffff811b4939 do_execve.fexecv_proc_title+0x98a5 rbx 0 rsp 0xfffffe0002a8f8b0 rbp 0 rsi 0x8 rdi 0xffffffff81d93998 w_mtx r8 0xffffffff811bd5a6 r9 0x6c ll+0x4b r10 0xfffff800dfd79600 r11 0 r12 0xffffffff81dd7718 r13 0 r14 0xffffffff8125cb1b r15 0xfffff800dfd78b80 rip 0xfffff80080054000 rflags 0x10286 0xfffff80080054000 db> db> alltrace Tracing command intr pid 12 tid 100008 td 0xfffff80080053580 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100009 td 0xfffff80080053000 fork_trampoline() at fork_trampoline Tracing command intr pid 12 tid 100011 td 0xfffff800031f1000 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100003 td 0xfffff800031df000 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100004 td 0xfffff80080054580 ??() at 18446735279764357120 Tracing command idle pid 11 tid 100005 td 0xfffff80080054000 fork_trampoline() at fork_trampoline Tracing command idle pid 11 tid 100006 td 0xfffff800031de580 fork_trampoline() at fork_trampoline Tracing command kernel pid 1 tid 100002 td 0xfffff800031df580 fork_trampoline() at fork_trampoline Tracing command audit pid 10 tid 100001 td 0xfffff80080055000 fork_trampoline() at fork_trampoline Tracing command kernel pid 0 tid 100000 td 0xffffffff81e8a3e0 (CPU 0) ??() at 0 KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+43/frame 0xfffffe0002a8f040 kdb_reenter() at kdb_reenter+47/frame 0xfffffe0002a8f050 trap() at trap+92/frame 0xfffffe0002a8f160 calltrap() at calltrap+8/frame 0xfffffe0002a8f160 --- trap 12, rip = 18446744071578560736, rsp = 18446741874730922544, rbp = 18446741874730922688 --- db_read_bytes() at db_read_bytes+128/frame 0xfffffe0002a8f2c0 db_get_value() at db_get_value+51/frame 0xfffffe0002a8f300 db_backtrace() at db_backtrace+567/frame 0xfffffe0002a8f390 db_trace_thread() at db_trace_thread+61/frame 0xfffffe0002a8f3b0 _db_stack_trace_all() at _db_stack_trace_all+244/frame 0xfffffe0002a8f450 db_command() at db_command+633/frame 0xfffffe0002a8f520 db_command_loop() at db_command_loop+100/frame 0xfffffe0002a8f530 db_trap() at db_trap+239/frame 0xfffffe0002a8f5c0 kdb_trap() at kdb_trap+451/frame 0xfffffe0002a8f630 trap_fatal() at trap_fatal+689/frame 0xfffffe0002a8f680 trap_pfault() at trap_pfault+98/frame 0xfffffe0002a8f6d0 trap() at trap+670/frame 0xfffffe0002a8f7e0 calltrap() at calltrap+8/frame 0xfffffe0002a8f7e0 --- trap 12, rip = 18446735279764357120, rsp = 18446741874730924208, rbp = 0 --- ??() at 18446735279764357120 *** error reading from address 8 *** KDB: reentering KDB: stack backtrace: db_trace_self_wrapper() at db_trace_self_wrapper+43/frame 0xfffffe0002a8f2b0 kdb_reenter() at kdb_reenter+47/frame 0xfffffe0002a8f2c0 db_get_value() at db_get_value+77/frame 0xfffffe0002a8f300 db_backtrace() at db_backtrace+567/frame 0xfffffe0002a8f390 db_trace_thread() at db_trace_thread+61/frame 0xfffffe0002a8f3b0 _db_stack_trace_all() at _db_stack_trace_all+244/frame 0xfffffe0002a8f450 db_command() at db_command+633/frame 0xfffffe0002a8f520 db_command_loop() at db_command_loop+100/frame 0xfffffe0002a8f530 db_trap() at db_trap+239/frame 0xfffffe0002a8f5c0 kdb_trap() at kdb_trap+451/frame 0xfffffe0002a8f630 trap_fatal() at trap_fatal+689/frame 0xfffffe0002a8f680 trap_pfault() at trap_pfault+98/frame 0xfffffe0002a8f6d0 trap() at trap+670/frame 0xfffffe0002a8f7e0 calltrap() at calltrap+8/frame 0xfffffe0002a8f7e0 --- trap 12, rip = 18446735279764357120, rsp = 18446741874730924208, rbp = 0 --- ??() at 18446735279764357120 db> db> show lapic lapic ID = 1 version = 1.0 max LVT = 5 SVR = ff (enabled) TPR = 00 In-service Interrupts: TMR Interrupts: IRR Interrupts: db> with the cpus 2 and 3 disabled: ---------------------------------------------------- ioapic3 <Version 1.1> irqs 24-47 on motherboard ioapic0 <Version 1.1> irqs 0-23 on motherboard ioapic1 <Version 1.1> irqs 48-54 on motherboard ioapic2 <Version 1.1> irqs 56-62 on motherboard lapic: Divisor 2, Frequency 99736985 Hz cpu0 BSP: ID: 0x00000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 AMD ext features: 0x00010003 AMD elvt0: 0x00010000 SMP: AP CPU #1 Launched! cpu1 AP: ID: 0x01000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 AMD ext features: 0x00010003 AMD elvt0: 0x00010000 [hangs here] ---------------------------------- with just DDB and w/o WITNESS: ---------------------------------- defini8254_delay() at) a i8254_delay+0x1e/f_lock_indefinite_check() at _mtx_lock_indefinite_check+0x21/frame 0xfffffe0 002986680 _mtx_lock_spin_cookie() at0_mtx_lock_spin_cookie+0xd4/frame 0xfffffe00029866f0 at getit() atkg_spin_cookie+0xd4/frame 0xfffffe0000606190 getit() atfgfe0002986710 rami8254_delay() at 0xfffffe00006061b0 /frame 0xfffffe0002986730 at _mi8254_delay+0x1e/frame 0xfffffe00006061d0 at_mtx_lock_indefinite_check() atk_mtx_lock_indefinite_check+0x21/fram 0x_fffffe0002986 740 in_cookie()ck_s_mtx_lock_spin_cookie+0xd4/frame 0xfffffe0000606250 getit() atmgetit+0x91/frame 0xfffffe0000606270 0xfffffe0054_delay() at i8ge254_delay+0x1e/frame 0xfffffe0000606290 _mtx_lgeock_indefinite_check() at _mtx_lock_indefinite_ch eck+0x21/frame 0xfffffe00006062a0 0xfffffe00029867d0 pin_cookie() i8254_delay+0x1e/frame 0xfffffe00029867f0 at _mtx_lock_spin_cookie+0xd4/frame 0x_mtx_lock_indefinite_check+ 21/frgetit() at 0xfffffe0002986800 0xfffffe0000606330 _spi8254_delay() at i8254_delay+0x1e/ atf_mtx_lock_spin_cookie+0xd4/frame 0xfffffde0002986870 eckg() ati_mtx_lock_indefinite_check+0x21/frame at geti_mtx_lock_spin_cookie() at _mtx_lock_s pin_cookie+0xd4/frame i0xfffffe00006063d0 at getit() ateglay+it+0x91/frame 0xfffffe00006063f0 i8254_delay() atfi8254_delay+0x1e/_mtx_lock_indefinite_check() at 0xfffffe0000606410 fra_mme 0xfffffe0002efinite_check() at_mtx_lock_spin_cookie() atck+0x2_lock_spin_cookie+0xd4/frame 0xfffffe000 2986930 0xfffffe0() at6g420 it+0x91/frame 0xfffffe0002986950 ie() at4_delay() atmie 0xfffffe0000606490 e/fgetit() at 0xfffffe0002986970 0x_mtx_lock_indefinite_checki8254_delay() at i8254_delay+0x1e/frame 0xfffffe00006064d0 _mtx_mtx_lock_indefinite_check+0x21/frame 0xfff at002986980 _in_mdefinite_check+0x21/frame 0xfffffe0000 at _mtx_lock_spin_cookie+0xd4/frame 0xfffffe000298 at _mtx_l ock_spin_cookie+0xd4/frame 0xfffffe0000606550 getit(getit+0x91/frame 0xfffffe0002986a10 ati8254_dt+0x91/frame 0xfffffe0000606570 i8254_delai8254_delay+0x1e/frame 0xfffffe0002986a30 a_mitx_lock_indefinite_check() afra_mtx_lock_indefinite_check+0x21/frame 0xfffffe0002986a40 0xfffffe0000606590 spi_n_cookie() ate_mtx_lock_spin_cookie+0xd4/frame 0x at8_mtx_lock_indefinite_c atame 0xfffffe00006065a0 0xf_mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd4/frame 0xfffffe0000606610 atgietit() atlgay+0x1e/frame 0x fffi8ffe0002986af0 at _mtx_lock_indefinite_check() atx_ffffe0000606650 defin_mtx_lock_indefinite_check() at 0xfffffe0002986b00 fin_mtx_lock_spin_cookie() atf_mtx_lock_sp0606660 kie_mtx_lock_spin_cookie() at _mtx_lock_spin_cookie+0xd4/frame 0xfffffe00006066d0 0xfffffe0002986b70 getitge() atfgrame 0xfffffe00006066f0 0xfi8254_delay() at i8254_delay+0x1e/frame 0xfffffe0000606710 at _mtx_lockdelay+0x1e/frame 0xfffffe0002986bb0 at__mtx_lock_indefinite_check+0x21/frame 0xfffffe0000606720 _atx_mtx_lock_indefinite_check+0x21/frame 0xfffffe0002986bc0 a_m_mtx_lock_sn_cookie() atm_mtx_lock_spin_cookie+0xd4/frame 0xffff getit() at0g000606790 and it eventually reboots itselfs without going to a db> prompt ------------------------------------------------------------ Oh, I think I have seen this once before. The issue is that the mtx_lock_spin() code tries to use DELAY(1) and DELAY() is trying to use the "clock_lock" in getit(). In your case, 'tsc_is_invariant' isn't true so DELAY() in sys/x86/x86/delay.c isn't using the TSC. I think the error is probably that we shouldn't be using DELAY(1) in our mutex code as it is too high level and/or we should special case DELAY(1) (vs DELAY(n)). Older versions of FreeBSD would map DELAY(1) to just 'inb(0x84)' without talking to the 8254 at all. It looks like i8254_delay() already does that when inside of KDB. Also, checking for tsc_is_invariant in DELAY is probably overly-correct. We aren't going to change P-states while we are spinning since we are actively spinning on the CPU. The worst that might happen is that if the CPU were running at a lower clock, we might wait too long. Given how DELAY is used, that's probably better than falling back to a slow timecounter with locks, etc. Created attachment 198774 [details]
lock_delay.patch
FYI, I'd also probably go for changing delay_tc()'s special case of the very-common TSC case to just use the simpler loop without worrying about overflow, etc. That said, I think the idea of having cpu_lock_delay() is probably worth fixing even if other changes to DELAY() would effectively fix this anyway. (In reply to John Baldwin from comment #5) this patch worked, but i needed two include statements. patch < ./lock.patch Hmm... Looks like a unified diff to me... The text leading up to this was: -------------------------- |--- sys/kern/kern_mutex.c (revision 339923) |+++ sys/kern/kern_mutex.c (working copy) -------------------------- Patching file sys/kern/kern_mutex.c using Plan A... Hunk #1 succeeded at 1206. Hmm... The next patch looks like a unified diff to me... The text leading up to this was: -------------------------- |--- sys/x86/include/x86_var.h (revision 339923) |+++ sys/x86/include/x86_var.h (working copy) -------------------------- Patching file sys/x86/include/x86_var.h using Plan A... Hunk #1 succeeded at 119. Hmm... The next patch looks like a unified diff to me... The text leading up to this was: -------------------------- |--- sys/x86/x86/delay.c (revision 339923) |+++ sys/x86/x86/delay.c (working copy) -------------------------- Patching file sys/x86/x86/delay.c using Plan A... Hunk #1 succeeded at 110. done --- kern_mutex.o --- /usr/src/sys/kern/kern_mutex.c:1209:3: error: implicit declaration of function 'cpu_lock_delay' is invalid in C99 [-Werror, -Wimplicit-function-declaration] cpu_lock_delay(); ^ /usr/src/sys/kern/kern_mutex.c:1209:3: note: did you mean 'lock_delay'? /usr/src/sys/sys/lock.h:252:6: note: 'lock_delay' declared here void lock_delay(struct lock_delay_arg *); ^ /usr/src/sys/kern/kern_mutex.c:1209:3: error: this function declaration is not a prototype [-Werror,-Wstrict-prototypes] cpu_lock_delay(); ^ --- kern_physio.o --- added #include <x86/include/x86_var.h> to sys/kern/kern_mutex.c and then --- delay.o --- cc -target x86_64-unknown-freebsd12.0 --sysroot=/usr/obj/usr/src/amd64.amd64/tmp -B/usr/obj/usr/src/amd64.amd64/tmp/usr/bin -c -O -pipe -g -nostdinc -I. -I/usr/src/sys -I/usr/src/sys/contrib/ck/include -I/usr/src/sys/contrib/libfdt -D_KERNEL -DHAVE_KERNEL_OPTION_HEADERS -include opt_global.h -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -MD -MF.depend.delay.o -MTdelay.o -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -msoft-float -fno-asynchronous-unwind-tables -ffreestanding -fwrapv -fstack-protector -gdwarf-2 -Wall -Wredundant-decls -Wnested-externs -Wstrict-prototypes -Wmissing-prototypes -Wpointer-arith -Wcast-qual -Wundef -Wno-pointer-sign -D__printf__=__freebsd_kprintf__ -Wmissing-include-dirs -fdiagnostics-show-option -Wno-unknown-pragmas -Wno-error-tautological-compare -Wno-error-empty-body -Wno-error-parentheses-equality -Wno-error-unused-function -Wno-error-pointer-sign -Wno-error-shift-negative-value -Wno-address-of-packed-member -mno-aes -mno-avx -std=iso9899:1999 -Werror /usr/src/sys/x86/x86/delay.c /usr/src/sys/x86/x86/delay.c:115:1: error: no previous prototype for function 'cpu_lock_delay' [-Werror,-Wmissing-prototypes] cpu_lock_delay(void) ^ 1 error generated. *** [delay.o] Error code 1 added #include <x86/include/x86_var.h> to /usr/src/sys/x86/x86/delay.c newest boot: INTR: Adding local APIC 1 as a target INTR: Adding local APIC 2 as a target INTR: Adding local APIC 3 as a target FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 2 package(s) x 2 core(s) Package HW ID = 0 Core HW ID = 0 CPU0 (BSP): APIC ID: 0 Core HW ID = 1 CPU1 (AP): APIC ID: 1 Package HW ID = 1 Core HW ID = 2 CPU2 (AP): APIC ID: 2 Core HW ID = 3 CPU3 (AP): APIC ID: 3 APIC: CPU 0 has ACPI ID 1 APIC: CPU 1 has ACPI ID 2 APIC: CPU 2 has ACPI ID 3 APIC: CPU 3 has ACPI ID 4 x86bios: IVT 0x000000-0x0004ff at 0xfffff80000000000 x86bios: SSEG 0x098000-0x098fff at 0xfffffe0000736000 x86bios: EBDA 0x09d000-0x09ffff at 0xfffff8000009d000 x86bios: ROM 0x0a0000-0x0fefff at 0xfffff800000a0000 SRAT: CPU 0 has memory domain 0 SRAT: CPU 1 has memory domain 0 SRAT: CPU 2 has memory domain 1 SRAT: CPU 3 has memory domain 1 Pentium Pro MTRR support enabled random: read 3840 bytes from preloaded cache random: unblocking device. arc4random: read 32 bytes from preloaded cache VIMAGE (virtualized network stack) enabled ULE: setup cpu 0 ULE: setup cpu 1 ULE: setup cpu 2 ULE: setup cpu 3 ACPI: RSDP 0x00000000000F9470 000024 (v02 SUN ) ACPI: XSDT 0x00000000DFFF0100 00006C (v01 SUN X4200 M2 00000039 MSFT 00000097) ACPI: FACP 0x00000000DFFF0290 0000F4 (v03 SUN X4200 M2 00000039 MSFT 00000097) Firmware Warning (ACPI): 32/64X length mismatch in FADT/Gpe0Block: 64/32 (20181003/tbfadt-748) Firmware Warning (ACPI): 32/64X length mismatch in FADT/Gpe1Block: 128/64 (20181003/tbfadt-748) ACPI: DSDT 0x00000000DFFF04A0 005F64 (v01 SUN X4200 M1 00000039 INTL 20050624) ACPI: FACS 0x00000000DFFFE000 000040 ACPI: APIC 0x00000000DFFF0390 000086 (v01 SUN X4200 M2 00000039 MSFT 00000097) ACPI: SPCR 0x00000000DFFF0420 000050 (v01 SUN X4200 M2 00000039 MSFT 00000097) ACPI: SLIT 0x00000000DFFF0470 000030 (v01 SUN OEMSLIT 00000039 MSFT 00000097) ACPI: OEMB 0x00000000DFFFE040 000063 (v01 SUN X4200 M2 00000039 MSFT 00000097) ACPI: HPET 0x00000000DFFF6410 000038 (v01 SUN OEMHPET0 00000039 MSFT 00000097) ACPI: IPET 0x00000000DFFF6450 000038 (v01 SUN OEMHPET1 00000039 MSFT 00000097) ACPI: SRAT 0x00000000DFFF6490 000110 (v01 AMD HAMMER 00000001 AMD 00000001) ACPI: SSDT 0x00000000DFFF65A0 0004F0 (v01 A M I POWERNOW 00000001 AMD 00000001) MADT: Found IO APIC ID 15, Interrupt 0 at 0xfec00000 ioapic0: ver 0x11 maxredir 0x17 ioapic0: Routing external 8259A's -> intpin 0 MADT: Found IO APIC ID 16, Interrupt 48 at 0xfeafd000 ioapic1: Changing APIC ID to 16 ioapic1: WARNING: intbase 48 != expected base r24 ioapic1: ver 0x11 maxredir 0x06 MADT: Found IO APIC ID 17, Interrupt 56 at 0xfeafc000 ioapic2: Changing APIC ID to 17 ioapic2: WARNING: intbase 56 != expected base r55 ioapic2: ver 0x11 maxredir 0x06 MADT: Found IO APIC ID 14, Interrupt 24 at 0xfeaff000 ioapic3: WARNING: intbase 24 != expected base r63 ioapic3: ver 0x11 maxredir 0x17 MADT: Interrupt override: source 9, irq 9 ioapic0: intpin 9 trigger: level ioapic3 <Version 1.1> irqs 24-47 on motherboard ioapic0 <Version 1.1> irqs 0-23 on motherboard ioapic1 <Version 1.1> irqs 48-54 on motherboard ioapic2 <Version 1.1> irqs 56-62 on motherboard lapic: Divisor 2, Frequency 99736935 Hz cpu0 BSP: ID: 0x00000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 AMD ext features: 0x00010003 AMD elvt0: 0x00010000 SMP: AP CPU #1 Launched! cpu1 AP: ID: 0x01000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 AMD ext features: 0x00010003 AMD elvt0: 0x00010000 SMP: AP CPU #2 Launched! cpu2 AP: ID: 0x02000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 AMD ext features: 0x00010003 AMD elvt0: 0x00010000 SMP: AP CPU #3 Launched! cpu3 AP: ID: 0x03000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 AMD ext features: 0x00010003 AMD elvt0: 0x00010000 TSC timecounter discards lower 1 bit(s) Timecounter "TSC-low" frequency 1296582722 Hz quality -100 random: entropy device external interface wlan: <802.11 Link Layer> snd_unit_init() u=0x00ff8000 [512] d=0x00007c00 [32] c=0x000003ff [1024] feeder_register: snd_unit=-1 snd_maxautovchans=16 latency=5 feeder_rate_min=1 feeder_rate_max=2016000 feeder_rate_round=25 EFI systbl not available [ath_hal] loaded nfslock: pseudo-device crypto: <crypto core> tcp_log: tcp_log device module_register_init: MOD_LOAD (vesa, 0xffffffff81066bb0, 0) error 19 io: <I/O> kbd: new array size 4 kbd1 at kbdmux0 mem: <memory> netmap: loaded module null: <full device, null device, zero device> hpt27xx: RocketRAID 27xx controller driver v1.2.8 hptnr: R750/DC7280 controller driver v1.1.5 hptrr: RocketRAID 17xx/2xxx SATA controller driver v1.2 nexus0 vtvga0: <VT VGA driver> on motherboard cryptosoft0: <software crypto> on motherboard crypto: assign cryptosoft0 driver id 0, flags 0x6000000 crypto: cryptosoft0 registers alg 1 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 2 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 3 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 4 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 5 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 16 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 6 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 7 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 32 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 18 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 19 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 20 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 8 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 15 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 9 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 10 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 13 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 14 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 34 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 35 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 36 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 37 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 11 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 22 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 23 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 25 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 24 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 26 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 27 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 28 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 21 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 17 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 29 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 30 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 31 flags 0 maxoplen 0 crypto: cryptosoft0 registers alg 38 flags 0 maxoplen 0 acpi0: <SUN X4200 M2> on motherboard ACPI: 2 ACPI AML tables successfully acquired and loaded ioapic0: routing intpin 9 (ISA IRQ 9) to lapic 0 vector 48 acpi0: Power Button (fixed) acpi0: wakeup code va 0xfffffe00007f5000 pa 0x99000 Table 'FACP' at 0xdfff0290 FACP: Found table at 0xdfff0290 Table 'FACP' at 0xdfff0290 FACP: Found table at 0xdfff0290 Table 'FACP' at 0xdfff0290 FACP: Found table at 0xdfff0290 Table 'FACP' at 0xdfff0290 FACP: Found table at 0xdfff0290 Table 'FACP' at 0xdfff0290 FACP: Found table at 0xdfff0290 acpi0: reservation of 0, a0000 (3) failed acpi0: reservation of 100000, dff00000 (3) failed cpu0: Processor \134_PR_.P001 (ACPI ID 1) -> APIC ID 0 cpu0: <ACPI CPU> on acpi0 cpu0: switching to generic Cx mode cpu1: Processor \134_PR_.P002 (ACPI ID 2) -> APIC ID 1 cpu1: <ACPI CPU> on acpi0 cpu2: Processor \134_PR_.P003 (ACPI ID 3) -> APIC ID 2 cpu2: <ACPI CPU> on acpi0 cpu3: Processor \134_PR_.P004 (ACPI ID 4) -> APIC ID 3 cpu3: <ACPI CPU> on acpi0 Table 'FACP' at 0xdfff0290 FACP: Found table at 0xdfff0290 attimer0: <AT timer> port 0x40-0x43 irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 attimer0: Can't map interrupt. Table 'FACP' at 0xdfff0290 FACP: Found table at 0xdfff0290 atrtc0: <AT realtime clock> port 0x70-0x71 irq 8 on acpi0 atrtc0: registered as a time-of-day clock, resolution 1.000000s ioapic0: routing intpin 8 (ISA IRQ 8) to lapic 1 vector 48 ioapic0: routing intpin 8 (ISA IRQ 8) to lapic 0 vector 49 12.0-BETA2 FreeBSD 12.0-BETA2 #3 r339840M Ok, I will work on getting a cleaned up version of this ready and merged for 11 and 12. First I will add cpu_lock_delay() and then I will follow up with a second change to add a real version of it for x86. I still need to do some more testing, but you can grab an updated patch from https://github.com/freebsd/freebsd/compare/master...bsdjhb:cpu_lock_delay.diff This is what I'm planning to commit as a series of 3 separate patches. (In reply to John Baldwin from comment #9) yes, that patch also works. I updated to a slightly newer version and applied that patch: 12.0-BETA3 FreeBSD 12.0-BETA3 #4 r340076M MADT: Interrupt override: source 9, irq 9 ioapic0: intpin 9 trigger: level ioapic3 <Version 1.1> irqs 24-47 on motherboard ioapic0 <Version 1.1> irqs 0-23 on motherboard ioapic1 <Version 1.1> irqs 48-54 on motherboard ioapic2 <Version 1.1> irqs 56-62 on motherboard lapic: Divisor 2, Frequency 99737060 Hz cpu0 BSP: ID: 0x00000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 AMD ext features: 0x00010003 AMD elvt0: 0x00010000 SMP: AP CPU #1 Launched! cpu1 AP: ID: 0x01000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 AMD ext features: 0x00010003 AMD elvt0: 0x00010000 SMP: AP CPU #3 Launched! cpu3 AP: ID: 0x03000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 AMD ext features: 0x00010003 AMD elvt0: 0x00010000 SMP: AP CPU #2 Launched! cpu2 AP: ID: 0x02000000 VER: 0x80050010 LDR: 0x00000000 DFR: 0xffffffff lint0: 0x00010700 lint1: 0x00000400 TPR: 0x00000000 SVR: 0x000001ff timer: 0x000100ef therm: 0x00010000 err: 0x000000f0 pmc: 0x00010400 AMD ext features: 0x00010003 AMD elvt0: 0x00010000 TSC timecounter discards lower 1 bit(s) Timecounter "TSC-low" frequency 1296579466 Hz quality -100 A commit references this bug: Author: jhb Date: Mon Nov 5 22:54:04 UTC 2018 New revision: 340170 URL: https://svnweb.freebsd.org/changeset/base/340170 Log: Add a custom implementation of cpu_lock_delay() for x86. Avoid using DELAY() since it can try to use spin locks on CPUs without a P-state invariant TSC. For cpu_lock_delay(), always use the TSC if it exists (even if it is not P-state invariant) to delay for a microsecond. If the TSC does not exist, read from I/O port 0x84 to delay instead. PR: 228768 Reported by: Roger Hammerstein <cheeky.m@live.com> Reviewed by: kib MFC after: 3 days Differential Revision: https://reviews.freebsd.org/D17851 Changes: head/sys/amd64/include/cpu.h head/sys/i386/include/cpu.h head/sys/x86/x86/delay.c A commit references this bug: Author: jhb Date: Thu Nov 8 22:39:40 UTC 2018 New revision: 340269 URL: https://svnweb.freebsd.org/changeset/base/340269 Log: MFC 340164,340168,340170: Add custom cpu_lock_delay() for x86. 340164: Add a KPI for the delay while spinning on a spin lock. Replace a call to DELAY(1) with a new cpu_lock_delay() KPI. Currently cpu_lock_delay() is defined to DELAY(1) on all platforms. However, platforms with a DELAY() implementation that uses spin locks should implement a custom cpu_lock_delay() doesn't use locks. 340168: Add a delay_tsc() static function for when DELAY() uses the TSC. This uses slightly simpler logic than the existing code by using the full 64-bit counter and thus not having to worry about counter overflow. 340170: Add a custom implementation of cpu_lock_delay() for x86. Avoid using DELAY() since it can try to use spin locks on CPUs without a P-state invariant TSC. For cpu_lock_delay(), always use the TSC if it exists (even if it is not P-state invariant) to delay for a microsecond. If the TSC does not exist, read from I/O port 0x84 to delay instead. PR: 228768 Approved by: re (gjb) Changes: _U stable/12/ stable/12/sys/amd64/include/cpu.h stable/12/sys/arm/include/cpu.h stable/12/sys/arm64/include/cpu.h stable/12/sys/i386/include/cpu.h stable/12/sys/kern/kern_mutex.c stable/12/sys/mips/include/cpu.h stable/12/sys/powerpc/include/cpu.h stable/12/sys/riscv/include/cpu.h stable/12/sys/sparc64/include/cpu.h stable/12/sys/x86/x86/delay.c A commit references this bug: Author: jhb Date: Thu Nov 8 22:42:58 UTC 2018 New revision: 340270 URL: https://svnweb.freebsd.org/changeset/base/340270 Log: MFC 340164,340168,340170: Add custom cpu_lock_delay() for x86. 340164: Add a KPI for the delay while spinning on a spin lock. Replace a call to DELAY(1) with a new cpu_lock_delay() KPI. Currently cpu_lock_delay() is defined to DELAY(1) on all platforms. However, platforms with a DELAY() implementation that uses spin locks should implement a custom cpu_lock_delay() doesn't use locks. 340168: Add a delay_tsc() static function for when DELAY() uses the TSC. This uses slightly simpler logic than the existing code by using the full 64-bit counter and thus not having to worry about counter overflow. 340170: Add a custom implementation of cpu_lock_delay() for x86. Avoid using DELAY() since it can try to use spin locks on CPUs without a P-state invariant TSC. For cpu_lock_delay(), always use the TSC if it exists (even if it is not P-state invariant) to delay for a microsecond. If the TSC does not exist, read from I/O port 0x84 to delay instead. PR: 228768 Changes: _U stable/11/ stable/11/sys/amd64/include/cpu.h stable/11/sys/arm/include/cpu.h stable/11/sys/arm64/include/cpu.h stable/11/sys/i386/include/cpu.h stable/11/sys/kern/kern_mutex.c stable/11/sys/mips/include/cpu.h stable/11/sys/powerpc/include/cpu.h stable/11/sys/riscv/include/cpu.h stable/11/sys/sparc64/include/cpu.h stable/11/sys/x86/x86/delay.c *** Bug 225450 has been marked as a duplicate of this bug. *** |
