Bug 223732 - mmap(2) causes unkillable denial of service with specific flags
Summary: mmap(2) causes unkillable denial of service with specific flags
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 11.1-RELEASE
Hardware: Any Any
: --- Affects Many People
Assignee: Konstantin Belousov
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-11-18 14:09 UTC by Arto Pekkanen
Modified: 2018-05-28 19:35 UTC (History)
7 users (show)

See Also:


Attachments
mmap(2) local denial of service with MAP_STACK|MAP_ALIGNED(N) flags (570 bytes, text/plain)
2017-11-18 14:09 UTC, Arto Pekkanen
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Arto Pekkanen 2017-11-18 14:09:02 UTC
Created attachment 188093 [details]
mmap(2) local denial of service with MAP_STACK|MAP_ALIGNED(N) flags

I have found a set of flags that will cause a process to saturate an entire execution unit at 100% median while the process becomes unkillable even with kill -9. Sometimes the process might go under 100% CPU usage, but it gets back too 100% after a while. The process state stays "running". This is effectively a perfect local denial of service, which cannot be undone by administrators or counter-intrusion software.

The only way to get out of this DoS is to reboot the entire system.

Furthermore, if you the process a number of times greater than the number of physical CPU cores available to the system (at least on amd64), the system becomes totally unresponsive, and then only a forced power down from a power button is effective.

To reproduce this denial of service, compile the following source code with clang and execute it:

#include <sys/mman.h>

int main(int argc, char** argv) {
    void *stack = mmap(0, 1<<12, PROT_READ|PROT_WRITE, MAP_STACK|MAP_ALIGNED(12), -1, 0);
    /* Never reached.
     * Process will saturate one entire execution unit!
     * ALSO THIS PROCESS CANNOT BE KILLED! 
     * 
     * DoS manifests when the value of 2nd argument (len) 
     * is less than or equal to 1<<N, where N is the argument
     * N to MAP_ALIGNED(N) macro 
     * 
     * MAP_STACK|MAP_ALIGNED(N) is the combination of flags
     * that seem to cause this DoS */
    return 0;
}

Like the above source code comments, the DoS manifests when flags MAP_STACK|MAP_ALIGNED(N) are used together, and the value of 2nd argument (len) is less than or equal to 1<<N where N is the argument N to MAP_ALIGNED(N) flag macro.

I have not tested if this affects jails, but I assume it does.

I marked this bug to affect all hardware, because I think this should be tested on all possible hardware, yet I have only amd64 boxes.

Source code to reproduce this bug also attached as a file for convenience.

Output of uname -a:
FreeBSD xxx.xxx 11.1-RELEASE-p4 FreeBSD 11.1-RELEASE-p4 #0: Tue Nov 14 06:12:40 UTC 2017     root@amd64-builder.daemonology.net:/usr/obj/usr/src/sys/GENERIC  amd64
Comment 1 Arto Pekkanen 2017-11-18 15:32:54 UTC
(In reply to Arto Pekkanen from comment #0)

Also an important note (which might not be obvious) is that the user need not be a super user. An access to a normal user account is enough to hose the entire system by forking the above piece of code in an infinite loop.
Comment 2 op 2017-11-18 21:56:10 UTC
op@opn op# procstat -kk 83681
  PID    TID COMM                TDNAME              KSTACK                       
83681 100141 dos                 -                   vm_map_findspace+0x5f vm_map_find+0x162 vm_map_find_min+0x4b vm_mmap_object+0x34b kern_mmap+0x57d sys_mmap+0x2a amd64_syscall+0x749 Xfast_syscall+0xfb 
op@opn op# procstat -kk 83681
  PID    TID COMM                TDNAME              KSTACK                       
83681 100141 dos                 -                   vm_map_find+0x162 vm_map_find_min+0x4b vm_mmap_object+0x34b kern_mmap+0x57d sys_mmap+0x2a amd64_syscall+0x749 Xfast_syscall+0xfb
Comment 3 Remko Lodder freebsd_committer freebsd_triage 2017-11-19 10:40:33 UTC
Dear Kib,

Can you please have a look at this and see how we can fix this.
While causing an outage and such, this is probably not a security issue on itself. There are multiple ways to "turn off" a local system by forkbombing the system etc, which does not classify as a SA. If you think that this should be different, please email us with that and we'll toss it in the group for discussion :-)

Thanks,
Remko
Comment 4 Konstantin Belousov freebsd_committer freebsd_triage 2017-11-19 12:20:38 UTC
I put the supposed fix into https://reviews.freebsd.org/D13155.

The issue is a local DoS, we do not issue SAs for such kind of bugs.
Comment 5 Arto Pekkanen 2017-11-19 16:49:27 UTC
(In reply to Remko Lodder from comment #3)

Although I have to say that for me this was the very first local DoS unpreventable by policies or background watchdog processes! I have to proudly announce that other than this small bug here, FreeBSD has been extremely resilient against my accidental attempts at local DoSsing while developing software :)
Comment 6 commit-hook freebsd_committer freebsd_triage 2017-11-22 16:45:41 UTC
A commit references this bug:

Author: kib
Date: Wed Nov 22 16:45:28 UTC 2017
New revision: 326098
URL: https://svnweb.freebsd.org/changeset/base/326098

Log:
  Return different error code for the guard page layout violation.

  On KERN_NO_SPACE error, as it is returned now, vm_map_find() continues
  the loop searching for the suitable range for the requested mapping
  with specific alignment.  Since the vm_map_findspace() succesfully
  finds the same place, the loop never ends.

  The errors returned from vm_map_stack() completely repeat the behavior
  of vm_map_insert() now, as suggested by Alan.

  Reported by:	Arto Pekkanen <aksyom@gmail.com>
  PR:	223732
  Reviewed by:	alc, markj
  Discussed with:	jhb
  Sponsored by:	The FreeBSD Foundation
  MFC after:	3 days
  Differential revision:	https://reviews.freebsd.org/D13186

Changes:
  head/sys/vm/vm_map.c
Comment 7 commit-hook freebsd_committer freebsd_triage 2017-11-23 12:55:17 UTC
A commit references this bug:

Author: pho
Date: Thu Nov 23 12:54:18 UTC 2017
New revision: 326126
URL: https://svnweb.freebsd.org/changeset/base/326126

Log:
  Add a regression test.

  PR:		223732
  Sponsored by:	Dell EMC Isilon

Changes:
  user/pho/stress2/misc/mmap32.sh
Comment 8 commit-hook freebsd_committer freebsd_triage 2017-11-25 14:48:24 UTC
A commit references this bug:

Author: kib
Date: Sat Nov 25 14:47:25 UTC 2017
New revision: 326188
URL: https://svnweb.freebsd.org/changeset/base/326188

Log:
  MFC r326098:
  Return different error code for the guard page layout violation.

  PR:	223732

Changes:
_U  stable/11/
  stable/11/sys/vm/vm_map.c
Comment 9 commit-hook freebsd_committer freebsd_triage 2017-11-25 14:52:30 UTC
A commit references this bug:

Author: kib
Date: Sat Nov 25 14:51:40 UTC 2017
New revision: 326189
URL: https://svnweb.freebsd.org/changeset/base/326189

Log:
  MFC r326098:
  Return different error code for the guard page layout violation.

  PR:	223732

Changes:
_U  stable/10/
  stable/10/sys/vm/vm_map.c