Bug 218262

Summary: [hwpstate] does not set overclocked frequency on AMD Ryzen CPU
Product: Base System Reporter: Nils Beyer <nbe>
Component: kernAssignee: freebsd-bugs (Nobody) <bugs>
Status: Closed FIXED    
Severity: Affects Only Me CC: avg, cem, seanpr, val
Priority: ---    
Version: CURRENT   
Hardware: amd64   
OS: Any   
Attachments:
Description Flags
zenstate.c none

Description Nils Beyer 2017-03-31 16:00:21 UTC 
System: FreeBSD 12.0-CURRENT #0 334829e6c(drm-next)-dirty
CPU: AMD Ryzen 1700 OCed to 3400MHz

Direct after booting the frequency is 2700MHz. Trying to set it to the over-
clocked frequency 3400MHz results in:
---------------------------------------------------------------------------------
hwpstate0: setting P0-state on cpu0
hwpstate0: setting P0-state on cpu1
hwpstate0: setting P0-state on cpu2
hwpstate0: setting P0-state on cpu3
hwpstate0: setting P0-state on cpu4
hwpstate0: setting P0-state on cpu5
hwpstate0: setting P0-state on cpu6
hwpstate0: setting P0-state on cpu7
hwpstate0: setting P0-state on cpu8
hwpstate0: setting P0-state on cpu9
hwpstate0: setting P0-state on cpu10
hwpstate0: setting P0-state on cpu11
hwpstate0: setting P0-state on cpu12
hwpstate0: setting P0-state on cpu13
hwpstate0: setting P0-state on cpu14
hwpstate0: setting P0-state on cpu15
hwpstate0: result: P1-state on cpu0
hwpstate0: error: loop is not enough.
hwpstate0: result: P1-state on cpu1
hwpstate0: error: loop is not enough.
hwpstate0: result: P1-state on cpu2
hwpstate0: error: loop is not enough.
hwpstate0: result: P1-state on cpu3
hwpstate0: error: loop is not enough.
hwpstate0: result: P1-state on cpu4
hwpstate0: error: loop is not enough.
hwpstate0: result: P1-state on cpu5
hwpstate0: error: loop is not enough.
hwpstate0: result: P1-state on cpu6
hwpstate0: error: loop is not enough.
hwpstate0: result: P1-state on cpu7
hwpstate0: error: loop is not enough.
hwpstate0: result: P1-state on cpu8
hwpstate0: error: loop is not enough.
hwpstate0: result: P1-state on cpu9
hwpstate0: error: loop is not enough.
hwpstate0: result: P1-state on cpu10
hwpstate0: error: loop is not enough.
hwpstate0: result: P1-state on cpu11
hwpstate0: error: loop is not enough.
hwpstate0: result: P1-state on cpu12
hwpstate0: error: loop is not enough.
hwpstate0: result: P1-state on cpu13
hwpstate0: error: loop is not enough.
hwpstate0: result: P1-state on cpu14
hwpstate0: error: loop is not enough.
hwpstate0: result: P1-state on cpu15
hwpstate0: error: loop is not enough.
hwpstate0: set freq failed, err 6
---------------------------------------------------------------------------------

sysctl dev.cpu.0
---------------------------------------------------------------------------------
dev.cpu.0.cx_method: C1/hlt
dev.cpu.0.cx_usage_counters: 24787146
dev.cpu.0.cx_usage: 100.00% last 3273us
dev.cpu.0.cx_lowest: C1
dev.cpu.0.cx_supported: C1/1/0
dev.cpu.0.freq_levels: 3400/3562 2700/2835 1550/1312
dev.cpu.0.freq: 2700
dev.cpu.0.%parent: acpi0
dev.cpu.0.%pnpinfo: _HID=none _UID=0
dev.cpu.0.%location: handle=\_PR_.P000
dev.cpu.0.%driver: cpu
dev.cpu.0.%desc: ACPI CPU
---------------------------------------------------------------------------------

That CPU runs at 3400MHz - both in BIOS and Ubuntu. Any ideas besides turning
off overclocking?



TIA and regards,
Nils
Comment 1 Val Packett 2017-09-29 20:06:44 UTC 
Just encountered this problem. (Also seems related: bug 221621)

Looks like Linux is writing the MSR in the loop
https://github.com/torvalds/linux/blob/35dbba31be52c59a89bd96d49f170999d57c9234/drivers/cpufreq/powernow-k8.c#L169-L175 but we write it once and then only check in the loop.

This shouldn't fix anything but I'll try…
Comment 2 Conrad Meyer freebsd_committer freebsd_triage 2017-09-29 20:08:41 UTC 
Yeah, I see the same thing setting P1 on Threadripper (bug 221621 as Greg notes).
Comment 3 Conrad Meyer freebsd_committer freebsd_triage 2017-09-29 20:12:03 UTC 
(In reply to Greg V from comment #1)
Note that Linux MSR_FIDVID_CTL is 0xc0010041 while we're writing MSR_AMD_10H_11H_CONTROL (0xc0010062).  Don't know if that makes any difference.
Comment 4 Val Packett 2017-09-29 20:15:28 UTC 
(In reply to Conrad Meyer from comment #3)
Seems like we're writing the correct MSR, as it matches AMD docs https://support.amd.com/TechDocs/54945_PPR_Family_17h_Models_00h-0Fh.pdf

Also I just built&installed the kernel in 2:54 minutes, before the overclock it was 3:20 or so. I'm pretty sure my CPU is actually overclocked.
Comment 5 Val Packett 2017-09-29 20:41:55 UTC 
(In reply to Conrad Meyer from comment #3)
Oh. FIDVID is for K7 processors, Family 17h is a newer generation, K8. That's why AMD doesn't mention FIDVID in the docs. Anyway… 

This interesting Linux patch https://lwn.net/Articles/443375/ unified AMD and Intel p-state handling, they basically just select between functions that write to different MSRs. And they do write to 0xc0010062.

---

I tried making the loop longer (1000 iterations) and writing in the loop, still it doesn't read the right value :(
Comment 6 Val Packett 2017-09-29 21:38:59 UTC 
Created attachment 186809 [details]
zenstate.c

Okay, it's definitely overclocked, just ran Cinebench R15 under Wine, got 1755 points. (same 3.9GHz clock in Windows 10 gives me ~1715, 3.0GHz is significantly less).

Here's the interesting bit from the AMD docs though: "This field provides the 
frequency component of the current non-boosted P-state of the core".

I guess "non-boosted" means "not ever returning P0"?!

How is it providing "the frequency component" if it returns the number of the P-state?! This documentation makes ZERO sense.

Also, here's a tiny program that sets P-state 0 from userspace via cpuctl, it's easier to experiment with this than recompiling the kernel and stuff.
Comment 7 Conrad Meyer freebsd_committer freebsd_triage 2017-11-30 20:33:16 UTC 
Resolved by r326383 + r326407.  Now we simply do not verify if the value set reads back from MSR.