Was seeing load average of around 5, even with next to nothing running and the system was nearly completely idle. Keyboard was also slow to react and shutting down paused for longer than 60 seconds for bufdaemon. Touching the keyboard would make it continue. Setting kern.eventtimer.timer=HPET seemed to resolve the issue. See https://lists.freebsd.org/pipermail/freebsd-current/2014-July/051498.html for others who have experienced the same thing. Opening bug per Adrian's request.
Line "Event timer "LAPIC" quality 400" in verbose dmesg tells that LAPIC timer in your CPU (used by default) is not compatible with CPU C-states (deep idle states). Can you make sure that C-states are not used on your system by checking 'sysctl dev.cpu | grep cx_usage' and 'sysctl -a | grep cx_lowest'? Timer code was not changing for quite some time already, while C-states were enabled by default less time ago, IIRC.
Theoretically C-states should be blocked automatically in such situation, but who knows... Also check whether your BIOS support C1E state, and if it does, try to disable it.
(In reply to Alexander Motin from comment #1) > Line "Event timer "LAPIC" quality 400" in verbose dmesg tells that LAPIC > timer in your CPU (used by default) is not compatible with CPU C-states > (deep idle states). Can you make sure that C-states are not used on your > system by checking 'sysctl dev.cpu | grep cx_usage' and 'sysctl -a | grep > cx_lowest'? > > Timer code was not changing for quite some time already, while C-states were > enabled by default less time ago, IIRC. meatwad% sysctl dev.cpu | grep cx_usage dev.cpu.0.cx_usage: 15.53% 82.84% 1.61% last 149us dev.cpu.0.cx_usage_counters: 1509220 8046504 156453 dev.cpu.1.cx_usage: 15.21% 77.78% 6.99% last 1803us dev.cpu.1.cx_usage_counters: 462080 2361986 212335 dev.cpu.2.cx_usage: 15.54% 78.02% 6.43% last 1601us dev.cpu.2.cx_usage_counters: 542112 2721324 224459 dev.cpu.3.cx_usage: 14.02% 79.92% 6.05% last 1014us dev.cpu.3.cx_usage_counters: 496716 2831459 214487 dev.cpu.4.cx_usage: 16.10% 77.77% 6.11% last 5971us dev.cpu.4.cx_usage_counters: 467800 2258637 177471 dev.cpu.5.cx_usage: 15.24% 77.76% 6.99% last 612us dev.cpu.5.cx_usage_counters: 462824 2361157 212369 dev.cpu.6.cx_usage: 19.44% 74.33% 6.21% last 872us dev.cpu.6.cx_usage_counters: 640034 2446184 204690 dev.cpu.7.cx_usage: 15.40% 79.04% 5.55% last 51us dev.cpu.7.cx_usage_counters: 464464 2382488 167297 meatwad% sysctl -a | grep cx_lowest hw.acpi.cpu.cx_lowest: C8 dev.cpu.0.cx_lowest: C8 dev.cpu.1.cx_lowest: C8 dev.cpu.2.cx_lowest: C8 dev.cpu.3.cx_lowest: C8 dev.cpu.4.cx_lowest: C8 dev.cpu.5.cx_lowest: C8 dev.cpu.6.cx_lowest: C8 dev.cpu.7.cx_lowest: C8 meatwad% Note this is after setting kern.eventtimer.timer=HPET as mentioned previously. If it matters, I can undo that and run these again.
(In reply to Alexander Motin from comment #2) > Theoretically C-states should be blocked automatically in such situation, > but who knows... Also check whether your BIOS support C1E state, and if it > does, try to disable it. How would I check that? Just look for something related to C1E in BIOS configuration?
(In reply to Steve Wills from comment #3) > (In reply to Alexander Motin from comment #1) > > Line "Event timer "LAPIC" quality 400" in verbose dmesg tells that LAPIC > > timer in your CPU (used by default) is not compatible with CPU C-states > > (deep idle states). Can you make sure that C-states are not used on your > > system by checking 'sysctl dev.cpu | grep cx_usage' and 'sysctl -a | grep > > cx_lowest'? > > > > Timer code was not changing for quite some time already, while C-states were > > enabled by default less time ago, IIRC. > > meatwad% sysctl dev.cpu | grep cx_usage > dev.cpu.0.cx_usage: 15.53% 82.84% 1.61% last 149us > dev.cpu.0.cx_usage_counters: 1509220 8046504 156453 > dev.cpu.1.cx_usage: 15.21% 77.78% 6.99% last 1803us > dev.cpu.1.cx_usage_counters: 462080 2361986 212335 > dev.cpu.2.cx_usage: 15.54% 78.02% 6.43% last 1601us > dev.cpu.2.cx_usage_counters: 542112 2721324 224459 > dev.cpu.3.cx_usage: 14.02% 79.92% 6.05% last 1014us > dev.cpu.3.cx_usage_counters: 496716 2831459 214487 > dev.cpu.4.cx_usage: 16.10% 77.77% 6.11% last 5971us > dev.cpu.4.cx_usage_counters: 467800 2258637 177471 > dev.cpu.5.cx_usage: 15.24% 77.76% 6.99% last 612us > dev.cpu.5.cx_usage_counters: 462824 2361157 212369 > dev.cpu.6.cx_usage: 19.44% 74.33% 6.21% last 872us > dev.cpu.6.cx_usage_counters: 640034 2446184 204690 > dev.cpu.7.cx_usage: 15.40% 79.04% 5.55% last 51us > dev.cpu.7.cx_usage_counters: 464464 2382488 167297 > meatwad% sysctl -a | grep cx_lowest > hw.acpi.cpu.cx_lowest: C8 > dev.cpu.0.cx_lowest: C8 > dev.cpu.1.cx_lowest: C8 > dev.cpu.2.cx_lowest: C8 > dev.cpu.3.cx_lowest: C8 > dev.cpu.4.cx_lowest: C8 > dev.cpu.5.cx_lowest: C8 > dev.cpu.6.cx_lowest: C8 > dev.cpu.7.cx_lowest: C8 > meatwad% > > Note this is after setting kern.eventtimer.timer=HPET as mentioned > previously. If it matters, I can undo that and run these again. That does not tell much after you enabled HPET, except that your system does have C-states. Reboot without switching to HPET and check again. If C-states block working, you should not see states deeper then C1 used even if enabled.
(In reply to Steve Wills from comment #4) > (In reply to Alexander Motin from comment #2) > > Theoretically C-states should be blocked automatically in such situation, > > but who knows... Also check whether your BIOS support C1E state, and if it > > does, try to disable it. > > How would I check that? Just look for something related to C1E in BIOS > configuration? Usually there is some option mentioning C1E state somewhere under the CPU options.
(In reply to Alexander Motin from comment #6) > (In reply to Steve Wills from comment #4) > > (In reply to Alexander Motin from comment #2) > > > Theoretically C-states should be blocked automatically in such situation, > > > but who knows... Also check whether your BIOS support C1E state, and if it > > > does, try to disable it. > > > > How would I check that? Just look for something related to C1E in BIOS > > configuration? > > Usually there is some option mentioning C1E state somewhere under the CPU > options. But that is after you make sure that normal C-states are not used by FreeBSD itself.
(In reply to Alexander Motin from comment #5) > That does not tell much after you enabled HPET, except that your system does > have C-states. Reboot without switching to HPET and check again. If C-states > block working, you should not see states deeper then C1 used even if enabled. Ok, rebooted without setting HPET: meatwad% sysctl kern.timecounter.hardware kern.eventtimer.timer kern.timecounter.hardware: TSC-low kern.eventtimer.timer: LAPIC meatwad% sysctl dev.cpu | grep cx_usage dev.cpu.0.cx_usage: 22.28% 77.71% 0.00% last 1359us dev.cpu.0.cx_usage_counters: 17048 59466 0 dev.cpu.1.cx_usage: 21.54% 78.45% 0.00% last 939us dev.cpu.1.cx_usage_counters: 17294 62983 0 dev.cpu.2.cx_usage: 31.10% 68.89% 0.00% last 1146us dev.cpu.2.cx_usage_counters: 33021 73155 0 dev.cpu.3.cx_usage: 21.15% 78.84% 0.00% last 738us dev.cpu.3.cx_usage_counters: 18109 67474 0 dev.cpu.4.cx_usage: 21.94% 78.05% 0.00% last 447us dev.cpu.4.cx_usage_counters: 18022 64084 0 dev.cpu.5.cx_usage: 22.70% 77.29% 0.00% last 682us dev.cpu.5.cx_usage_counters: 18103 61637 0 dev.cpu.6.cx_usage: 26.53% 73.46% 0.00% last 430us dev.cpu.6.cx_usage_counters: 33056 91541 0 dev.cpu.7.cx_usage: 22.60% 77.39% 0.00% last 6us dev.cpu.7.cx_usage_counters: 16724 57250 0 meatwad% sysctl -a | grep cx_lowest hw.acpi.cpu.cx_lowest: C8 dev.cpu.0.cx_lowest: C8 dev.cpu.1.cx_lowest: C8 dev.cpu.2.cx_lowest: C8 dev.cpu.3.cx_lowest: C8 dev.cpu.4.cx_lowest: C8 dev.cpu.5.cx_lowest: C8 dev.cpu.6.cx_lowest: C8 dev.cpu.7.cx_lowest: C8 meatwad%
(In reply to Steve Wills from comment #8) > (In reply to Alexander Motin from comment #5) > > That does not tell much after you enabled HPET, except that your system does > > have C-states. Reboot without switching to HPET and check again. If C-states > > block working, you should not see states deeper then C1 used even if enabled. > > Ok, rebooted without setting HPET: > > meatwad% sysctl kern.timecounter.hardware kern.eventtimer.timer > kern.timecounter.hardware: TSC-low > kern.eventtimer.timer: LAPIC > meatwad% sysctl dev.cpu | grep cx_usage > dev.cpu.0.cx_usage: 22.28% 77.71% 0.00% last 1359us > dev.cpu.0.cx_usage_counters: 17048 59466 0 > dev.cpu.1.cx_usage: 21.54% 78.45% 0.00% last 939us > dev.cpu.1.cx_usage_counters: 17294 62983 0 > dev.cpu.2.cx_usage: 31.10% 68.89% 0.00% last 1146us > dev.cpu.2.cx_usage_counters: 33021 73155 0 > dev.cpu.3.cx_usage: 21.15% 78.84% 0.00% last 738us > dev.cpu.3.cx_usage_counters: 18109 67474 0 > dev.cpu.4.cx_usage: 21.94% 78.05% 0.00% last 447us > dev.cpu.4.cx_usage_counters: 18022 64084 0 > dev.cpu.5.cx_usage: 22.70% 77.29% 0.00% last 682us > dev.cpu.5.cx_usage_counters: 18103 61637 0 > dev.cpu.6.cx_usage: 26.53% 73.46% 0.00% last 430us > dev.cpu.6.cx_usage_counters: 33056 91541 0 > dev.cpu.7.cx_usage: 22.60% 77.39% 0.00% last 6us > dev.cpu.7.cx_usage_counters: 16724 57250 0 > meatwad% sysctl -a | grep cx_lowest > hw.acpi.cpu.cx_lowest: C8 > dev.cpu.0.cx_lowest: C8 > dev.cpu.1.cx_lowest: C8 > dev.cpu.2.cx_lowest: C8 > dev.cpu.3.cx_lowest: C8 > dev.cpu.4.cx_lowest: C8 > dev.cpu.5.cx_lowest: C8 > dev.cpu.6.cx_lowest: C8 > dev.cpu.7.cx_lowest: C8 > meatwad% Hmm. It looks like the deepest C-state was actually disabled automatically. Then it is surprising to have any problem with LAPIC. But, just to be sure that BIOS does not fool us, could you disable all C-states completely with something like this in rc.conf: performance_cx_lowest="C1" economy_cx_lowest="C1" , and if it won't help, also disable C1E in BIOS.
(In reply to Alexander Motin from comment #9) > > Hmm. It looks like the deepest C-state was actually disabled automatically. > Then it is surprising to have any problem with LAPIC. But, just to be sure > that BIOS does not fool us, could you disable all C-states completely with > something like this in rc.conf: > performance_cx_lowest="C1" > economy_cx_lowest="C1" > , and if it won't help, also disable C1E in BIOS. Setting performance_cx_lowest="C1" and economy_cx_lowest="C1" in rc.conf did seem to help.
(In reply to Steve Wills from comment #10) > Setting performance_cx_lowest="C1" and economy_cx_lowest="C1" in rc.conf did > seem to help. Then I guess your ACPI BIOS lies to OS about C2 state semantics -- it should not affect LAPIC timer, but seems somehow does.
Can you attach a verbose dmesg please? Can you also try the patch attached to bug 192316 and see if that makes any difference?