Bug 31310

On Tue, Oct 16, 2001 at 04:16:51PM +0800, justin@skysoft.com.tw wrote:
> 
> >Description:
> KVA_PAGES in kernel config is set to 512, and I had 'make world'
> to update /usr/lib.
> 
> When I run any program that use pthread (linked with libc_r), 
> it prints:
> 
>  
> Fatal error 'Cannot allocate red zone for initial thread' at line ? in file
> /usr/src/lib/libc_r/uthread/uthread_init.c (errno = ?)
> 
> Abort trap (core dumped)
>  
> 
> I didn't have this problem on 4.4-RC0 with ldscript.i386 and pmap.h 
> patched as the following to enlarge kernel address space to 2G.

I'm not sure I understand what you're doing, so let me try to summarize my
understanding before guessing what the problem is.  It sounds like you are
changing the amount of address space reserved for the kernel, which affects
where the userland stack is mapped.  Look in
src/lib/libc_r/uthread/uthread_init.c for the mmap() call that creates the
red zone:

		/*
		 * Create a red zone below the main stack.  All other stacks are
		 * constrained to a maximum size by the paramters passed to
		 * mmap(), but this stack is only limited by resource limits, so
		 * this stack needs an explicitly mapped red zone to protect the
		 * thread stack that is just beyond.
		 */
		if (mmap((void *) USRSTACK - PTHREAD_STACK_INITIAL -
		    PTHREAD_STACK_GUARD, PTHREAD_STACK_GUARD, 0, MAP_ANON,
		    -1, 0) == MAP_FAILED)
			PANIC("Cannot allocate red zone for initial thread");

The code depends on the USRSTACK macro when determining where to create the
red zone, and I suspect that its value is somewhere in kernel memory in
your case.

From src/sys/i386/include/vmparam.h:

#define	VM_MAXUSER_ADDRESS	VADDR(UMAXPTDI, UMAXPTEOFF)

#define	USRSTACK		VM_MAXUSER_ADDRESS

Please let me know if after further diagnostic effort you discover this to
be an actual problem in libc_r.

Jason

Thanks to Jason.

I solved this problem by adding -DKVA_PAGES=512 in CFLAGS at etc/make.conf

Libc_r include /usr/include/machine/pmap.h to determine 
USRSTACK, but kernel configuration of KVA_PAGES is defined in
/usr/src/sys/compile/XXXX/opt_global.h, so libc_r use wrong value of
USRSTACK.

Once after KVA_PAGES is documented in LINT, a note on this would be helpful
for those who need pthread and have large memory.

By the way, FreeBSD prior to 4.4-STABLE, including 4.4-RC0, seem to have
wrong pmap.h that can't produce correct USRSTACK from KVA_PAGES macro. If a
person want to upgrade to 4.4-STABLE and change KVA_PAGES, maybe he needs to
make world twice to have correct pmap.h in /usr/include.

On Tue, 16 Oct 2001, Jason Evans wrote:

>  I'm not sure I understand what you're doing, so let me try to summarize my
>  understanding before guessing what the problem is.  It sounds like you are
>  changing the amount of address space reserved for the kernel, which affects
>  where the userland stack is mapped.  Look in
>  src/lib/libc_r/uthread/uthread_init.c for the mmap() call that creates the
>  red zone:
>
>  		/*
>  		 * Create a red zone below the main stack.  All other stacks are
>  		 * constrained to a maximum size by the paramters passed to
>  		 * mmap(), but this stack is only limited by resource limits, so
>  		 * this stack needs an explicitly mapped red zone to protect the
>  		 * thread stack that is just beyond.
>  		 */
>  		if (mmap((void *) USRSTACK - PTHREAD_STACK_INITIAL -
>  		    PTHREAD_STACK_GUARD, PTHREAD_STACK_GUARD, 0, MAP_ANON,
>  		    -1, 0) == MAP_FAILED)
>  			PANIC("Cannot allocate red zone for initial thread");
>
>  The code depends on the USRSTACK macro when determining where to create the
>  red zone, and I suspect that its value is somewhere in kernel memory in
>  your case.
>
>  From src/sys/i386/include/vmparam.h:
>
>  #define	VM_MAXUSER_ADDRESS	VADDR(UMAXPTDI, UMAXPTEOFF)
>
>  #define	USRSTACK		VM_MAXUSER_ADDRESS
>
>  Please let me know if after further diagnostic effort you discover this to
>  be an actual problem in libc_r.

Code like this (actually mainly bogus definitions of non-constants as
constant in <sys/param.h>) is very broken.  USRSTACK depends on the
kernel option KVA_PAGES.  Userland can't see kernel options headers,
so USRSTACK should not be visible in userland.  But i386/include/param.h
provides a bogus default for KVA_PAGES and consequently USRSTACK to
move the breakage from compile time to run time.

Some other bogus non-constant constants:
- OPEN_MAX and CHILD_MAX.
- PAGE_SIZE and pgtok() leaked out to userland a long time ago, desipite
  the existence of syscall and a sysctl to insulate them.  PAGE_SIZE is
  now an option on ia64's.
- UPAGES.
- MAXDSIZE, etc.  There are now tunables.  Anyone tuning them had better
  know all related magic addresses used deep in places like the above.
  There was some discussion of making them (variable) rlimits.  This stalled
  on the complications for knowing all the magic addresses and keeping them
  separate.

Bruce

Comment 4 Peter Wemm 2001-12-17 22:58:12 UTC

State Changed
From-To: open->closed

Committed to -current on 2001/10/26, and -stable on 2001/11/03. 
(See src/lib/libc_r/uthread/uthread_init.c and uthread_stack.c etc)