Bug 241927 - kernel crash in kern_cpuset_setdomain(CPU_LEVEL_CPUSET, CPU_WHICH_PID, -1, ...)
Summary: kernel crash in kern_cpuset_setdomain(CPU_LEVEL_CPUSET, CPU_WHICH_PID, -1, ...)
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 12.0-RELEASE
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-bugs (Nobody)
URL:
Keywords: crash
Depends on:
Blocks:
 
Reported: 2019-11-12 20:12 UTC by Brice Goglin
Modified: 2019-11-13 09:58 UTC (History)
1 user (show)

See Also:


Attachments
screenshot of the backtrace of the crash (25.58 KB, image/png)
2019-11-12 20:12 UTC, Brice Goglin
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Brice Goglin 2019-11-12 20:12:11 UTC
Created attachment 209111 [details]
screenshot of the backtrace of the crash

Hello

While adding FreeBSD NUMA support in hwloc, we found a reproducible way to crash the kernel. Just call cpuset_setdomain(CPU_LEVEL_CPUSET, CPU_WHICH_PID, -1, ...).

Here's the minimalistic userland program to reproduce:

#include <sys/types.h>
#include <sys/sysctl.h>
#include <sys/param.h>
#include <sys/cpuset.h>
#include <sys/domainset.h>

int main()
{
  domainset_t mask;
  DOMAINSET_ZERO(&mask);
  DOMAINSET_SET(0, &mask);
  cpuset_setdomain(CPU_LEVEL_CPUSET, CPU_WHICH_PID, -1, sizeof(mask),
&mask, DOMAINSET_POLICY_PREFER);
}

I don't understand what this call is supposed to do, we actually wanted to use CPU_LEVEL_WHICH instead of CPU_LEVEL_CPUSET :)

We were running this inside a KVM VM with 4 NUMA domains but the crash also occurs without multiple NUMA domains.

I am attaching a screenshot of the backtrace after the crash. I don't know how to debug more.

Brice
Comment 1 Mark Johnston freebsd_committer freebsd_triage 2019-11-12 20:32:34 UTC
Are you by any chance able to test on 12.1?  We had fixed a bug in cpuset_setdomain() which is likely the one you ran into.  If not, can you please provide a standalone reproducible?  I'm not able to crash -current with

#include <sys/param.h>                                                                                                                                        
#include <sys/cpuset.h>                                                                                                                                       
#include <sys/domainset.h>                                                                                                                                    
                                                                                                                                                              
#include <string.h>                                                                                                                                           
                                                                                                                                                              
int                                                                                                                                                           
main(int argc, char **argv)                                                                                                                                   
{                                                                                                                                                             
        domainset_t ds;                                                                                                                                       
                                                                                                                                                              
        memset(&ds, 0, sizeof(ds));                                                                                                                           
        cpuset_setdomain(CPU_LEVEL_CPUSET, CPU_WHICH_PID,                                                                                                     
            -1, sizeof(ds), &ds, DOMAINSET_POLICY_ROUNDROBIN);                                                                                                
        return (0);                                                                                                                                           
}
Comment 2 Brice Goglin 2019-11-12 20:35:58 UTC
Yes we're planning to install a 12.1 VM tomorrow, I'll let you know.
Comment 3 Brice Goglin 2019-11-13 09:58:53 UTC
12.1 works fine, thanks !