Bug 200802

Summary: MAXCPU bump to 256 breaks ABI for cpuset_getaffinity/setaffinity
Product: Base System Reporter: rr
Component: kernAssignee: freebsd-bugs (Nobody) <bugs>
Status: Closed FIXED    
Severity: Affects Some People CC: adrian, ae, dchagin, emaste, jhb
Priority: ---    
Version: 10.1-STABLE   
Hardware: amd64   
OS: Any   
See Also: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=269572

Description rr 2015-06-12 01:31:17 UTC
The bump to MAXCPU 256 makes cpuset_t larger, so minimum setsize checks in cpuset_* fail for binaries compiled against earlier versions (with MAXCPU 64).
Comment 1 Andrey V. Elsukov freebsd_committer freebsd_triage 2015-06-19 21:58:12 UTC
I seen one report that 10.1-STABLE instantly reboots after updating to r283303. Could this be the cause of that?
Comment 2 John Baldwin freebsd_committer freebsd_triage 2015-06-20 11:53:28 UTC
Hmm, the simplest change is to revert MAXCPU for now.  Longer term it might be nice to rework the cpusetsize changes so that they only fail if there are CPU IDs that don't fit in the requested size.

I think doing that would involve:

1) Replacing 'sizeof(cpuset_t)' with something like '(mp_maxid + NBBY - 1) / NBBY'
2) When working with a cpuset internally rounding up the size to 'sizeof(cpuset_t)' to pass around internally and then truncating to the requested size when doing a copyout.
Comment 3 commit-hook freebsd_committer freebsd_triage 2015-06-23 06:30:53 UTC
A commit references this bug:

Author: kib
Date: Tue Jun 23 06:30:37 UTC 2015
New revision: 284720
URL: https://svnweb.freebsd.org/changeset/base/284720

Log:
  Revert part of the r283303 (by jhb):

  Revert MFC of r270223, which bumped MAXCPU on amd64 from 64 to 256.
  The cpuset_getaffinity(2) and cpuset_setaffinity(2) check minimum set
  size, which now fails for binaries compiled on 10.0 with MAXCPU == 64.

  Submitted by:	jhb
  PR:	  200802

Changes:
_U  stable/10/
  stable/10/sys/amd64/include/param.h
  stable/10/sys/sys/param.h
Comment 4 Adrian Chadd freebsd_committer freebsd_triage 2015-06-23 13:59:58 UTC
Shouldn't cpuset users be querying the cpuset size from sysctl and using that as their allocation base?
Comment 5 Glen Barber freebsd_committer freebsd_triage 2015-07-07 17:09:44 UTC
Offending commit has been reverted.
Comment 6 Dmitry Chagin freebsd_committer freebsd_triage 2023-02-15 21:28:35 UTC
well, seem I fixed this, like jhb@ suggested above.
All commits merged to stable/13 including pthread affinity methods, except the compat shim which is allows to run a newer world on old kernel. I plan to MFC it after 3 days
Comment 7 Ed Maste freebsd_committer freebsd_triage 2023-02-15 21:40:37 UTC
(In reply to Dmitry Chagin from comment #6)

I guess the main change related to this is:

commit f35093f8d6d8155ab2e56c11ee03d474688b16a2
Author: Dmitry Chagin <dchagin@FreeBSD.org>
Date:   Wed May 11 10:36:01 2022 +0300

    Use Linux semantics for the thread affinity syscalls.
    
    Linux has more tolerant checks of the user supplied cpuset_t's.
    
    Minimum cpuset_t size that the Linux kernel permits in case of
    getaffinity() is the maximum CPU id, present in the system / NBBY,
    the maximum size is not limited.
    For setaffinity(), Linux does not limit the size of the user-provided
    cpuset_t, internally using only the meaningful part of the set, where
    the upper bound is the maximum CPU id, present in the system, no larger
    than the size of the kernel cpuset_t.
    Unlike FreeBSD, Linux ignores high bits if set in the setaffinity(),
    so clear it in the sched_setaffinity() and Linuxulator itself.
    
    Reviewed by:            Pau Amma (man pages)
    In collaboration with:  jhb
    Differential revision:  https://reviews.freebsd.org/D34849
    MFC after:              2 weeks
Comment 8 Dmitry Chagin freebsd_committer freebsd_triage 2023-02-15 21:48:12 UTC
(In reply to Ed Maste from comment #7)
yeah, and this:

commit 01f74ccd5a0d1a444703e931339709c7de5296b5
Author:     Dmitry Chagin <dchagin@FreeBSD.org>
AuthorDate: 2023-01-29 12:35:18 +0000
Commit:     Dmitry Chagin <dchagin@FreeBSD.org>
CommitDate: 2023-01-29 12:35:18 +0000

    libthr: Fix pthread_attr_[g|s]etaffinity_np to match it's manual and the kernel.

    Since f35093f8 semantics of a thread affinity functions is changed to be a
    compatible with Linux:

    In case of getaffinity(), the minimum cpuset_t size that the kernel permits is
    the maximum CPU id, present in the system, / NBBY bytes, the maximum size is not
    limited.
    In case of setaffinity(), the kernel does not limit the size of the user-provided
    cpuset_t, internally using only the meaningful part of the set, where the upper
    bound is the maximum CPU id, present in the system, no larger than the size of
    the kernel cpuset_t.

    To match pthread_attr_[g|s]etaffinity_np checks of the user-provided cpusets to
    the kernel behavior export the minimum cpuset_t size allowed by running kernel
    via new sysctl kern.sched.cpusetsizemin and use it in checks.

    Reviewed by:
    Differential Revision:  https://reviews.freebsd.org/D38112
    MFC after:              1 week