I've been having a difficult time using the new CPU percentage limits in FreeBSD 10. It seems that if you specify a limit beyond 110% (say, 500%), the process (or jail, etc.) that you are trying to control becomes unthrottled, and can use the CPU until all logical cores are busy. Fix: My workaround is to remove these lines in kern_racct.c from function racct_alloc_resource(): if ((resource == RACCT_PCTCPU) && (racct->r_resources[RACCT_PCTCPU] > 100 * 1000000)) racct->r_resources[RACCT_PCTCPU] = 100 * 1000000; I'm not sure if there needs to really be any cap on the reported %CPU from the kernel. If so, it should be at least the number of logical cores available * 100 * 1000000. How-To-Repeat: Run a process that uses 20 cores at 100% utilization (2000%) in total. Attempt to limit that process to only 1000% using rctl. rctl -a 'process:PID:pcpu:deny=1000/process' Note that the process is not throttled.
Created attachment 151701 [details] Patch for kern_racct.c that permits CPU limiting beyond 100% This is the patch I've been using on 10.1-STABLE #11 r277139M. It uses a more reasonable CPU limit which is 100% * MAXCPU. It would be nice to get this rolled into the distribution; I can't imagine multi-core CPU limits have worked for anyone in 10/10.1. For anyone who wants to try this, apply as follows before rebuilding the kernel: patch /usr/src/sys/kern/kern_racct.c < kern_racct.c.patch
Edward, Would there be a better alternative to using MAXCPU here? Any objections to me committing this patch?
How about using mp_ncpus?
While mp_ncpus might be a more correct limit, just dropping it in place would incur one or two additional int64 multiplications every time the limit was checked, which could be frequent. I can avoid that by creating a state variable in one of the racct structures to compare against, but that may be more complicated than is needed for this fix (even then it might not be a perfect accounting of CPU resources, such as on architectures that can disable cores to save power or hyperthreaded cores that aren't always available). I'm not convinced that the cap is even necessary, but since it seems the original author only wanted to avoid outrageous pctcpu values, I think limiting to a multiple of MAXCPU would still accomplish that.
Yeah, I agree with dustinwenz@. Now, have anyone actually tested it in production? Any unintended side effects?
I tested it for individual processes originally, and it seemed to work as intended. Part of the reason this came up just now is that we may have a need to use this in production. I've just deployed cpu limiting on several systems with a few dozen jails each, limiting pcpu to 300% for specific users. I'm going to let it burn for a few days and see if anything explodes. The risk for this is probably low. The feature is broken enough that I hardly expect many people to be using it (please correct me if I'm wrong on that assumption).
I agree. Go for it; if something breaks we can always just back it out or try fix in a better way.
A commit references this bug: Author: jpaetzel Date: Tue Nov 10 14:14:33 UTC 2015 New revision: 290662 URL: https://svnweb.freebsd.org/changeset/base/290662 Log: Fix a bug in the CPU % limiting code If you attempt to set a pcpu limit that is higher than 110% using rctl (for instance, you want a jail to be able to use 2 cores on your system so you set pcpu to 200%) the thing you are trying to limit becomes unthrottled. PR: 189870 Submitted by: dustinwenz@ebureau.com Reviewed by: trasz MFC after: 1 week Changes: head/sys/kern/kern_racct.c
A commit references this bug: Author: jpaetzel Date: Fri Nov 20 15:13:50 UTC 2015 New revision: 291100 URL: https://svnweb.freebsd.org/changeset/base/291100 Log: MFC 290662 Fix a bug in the CPU % limiting code If you attempt to set a pcpu limit that is higher than 110% using rctl (for instance, you want a jail to be able to use 2 cores on your system so you set pcpu to 200%) the thing you are trying to limit becomes unthrottled. PR: 189870 Submitted by: dustinwenz@ebureau.com Reviewed by: trasz Changes: _U stable/10/ stable/10/sys/kern/kern_racct.c
Committed back in 2015.