Bug 18209

Summary: rlimits are never checked in exec() if executable format is ELF
Product: Base System Reporter: jason <jason>
Component: kernAssignee: Brian Feldman <green>
Status: Closed FIXED    
Severity: Affects Only Me    
Priority: Normal    
Version: 4.0-RELEASE   
Hardware: Any   
OS: Any   
Attachments:
Description Flags
file.diff none

Description jason 2000-04-25 18:20:01 UTC
when handling exec*() system calls, FreeBSD checks if the new executable image
memory requirements fit into active RLIMIT_* bounds. if they do not fit,
exec*() fails returning ENOMEM. The problem is, this check is never done for
ELF binaries, thus, ELF-format program can grab all available memory, no matter
how low *rlimits are.

How-To-Repeat: 
/*
 *  This program demonstrates a leak in freebsd exec(), which [erroneously]
 *  allows ELF program to get as much memory as it wishes, ignoring *rlimit
 *  settings, while [correctly] denying any non-ELF program to exceed *rlimit
 *  settings on exec().
 *  How to use:
 *        gcc -elf -o m_elf m.c        # compat 3 probably required for fbsd4
 *        gcc -aout -o m_aout m.c
 *        /usr/bin/limits -d 1 ./m_aout
 *        /usr/bin/limits -d 1 ./m_elf
 *  you'll see the difference.
 */

#include <stdio.h>
#include <unistd.h>
#include <string.h>

char            a[ 10000000 ] ;

int             main( int ac, char **av ) {
        memset( a, 1, sizeof(a) ) ;
        puts( "sleeping" ) ;
        sleep( 100 ) ;
        return 0 ;
}
Comment 1 vova 2000-05-15 15:05:02 UTC
More detailed fix for ELF exec() and fix for mmap() limits problem there:
http://www.redline.ru/~jason/bsd.html

--
TSB Russian Express, Moscow
Vladimir B. Grebenschikov, vova@express.ru
Comment 2 Kris Kennaway freebsd_committer freebsd_triage 2000-09-22 09:33:15 UTC
Responsible Changed
From-To: freebsd-bugs->green

I hereby volunteer green to look at this problem
Comment 3 Brian Feldman freebsd_committer freebsd_triage 2001-02-19 21:42:15 UTC
State Changed
From-To: open->feedback

This needs modification to actually count all the loadable segments 
and total them, THEN check against the size.  That's about it, though; 
otherwise, this seems almost right so if that's fixed, it should be 
good.
Comment 4 uitm 2002-05-17 19:11:02 UTC
A collection of these patches updated for RELENG_4 with respect to
now definable variables like MAXDSIZ could be found here:
http://www.blackflag.ru/patches/


-- 
Andrey Alekseyev. Zenon N.S.P.
Comment 5 John-Mark Gurney freebsd_committer freebsd_triage 2003-06-21 04:16:11 UTC
State Changed
From-To: feedback->closed

This functionality has already been added.  If you want the other 
elf loader improvements committed, update the patches, and open a new 
PR. 

Thanks.