Bug 193465

Comment 1 Sean Bruno 2014-09-08 17:57:48 UTC

Created attachment 147073 [details]
HEAD vmstat -z output on my mips32 router.

Comment 2 Sean Bruno 2014-09-08 18:05:18 UTC

Specifically, this is what I am doing to work around this issue:

Index: sys/kern/kern_malloc.c
===================================================================
--- sys/kern/kern_malloc.c	(revision 271278)
+++ sys/kern/kern_malloc.c	(working copy)
@@ -151,10 +151,21 @@
 	{1024, "1024", },
 	{2048, "2048", },
 	{4096, "4096", },
+#if PAGE_SIZE > 4096
 	{8192, "8192", },
+#if PAGE_SIZE > 8192
 	{16384, "16384", },
+#if PAGE_SIZE > 16384
 	{32768, "32768", },
+#if PAGE_SIZE > 32768
 	{65536, "65536", },
+#if PAGE_SIZE > 65536
+#error "Unsupported PAGE_SIZE"
+#endif
+#endif
+#endif
+#endif
+#endif
 	{0, NULL},
 };

Comment 3 Sean Bruno 2014-09-08 18:53:51 UTC

(In reply to Sean Bruno from comment #2)
> Specifically, this is what I am doing to work around this issue:
> 
> Index: sys/kern/kern_malloc.c
> ===================================================================
> --- sys/kern/kern_malloc.c	(revision 271278)
> +++ sys/kern/kern_malloc.c	(working copy)
> @@ -151,10 +151,21 @@
>  	{1024, "1024", },
>  	{2048, "2048", },
>  	{4096, "4096", },
> +#if PAGE_SIZE > 4096
>  	{8192, "8192", },
> +#if PAGE_SIZE > 8192
>  	{16384, "16384", },
> +#if PAGE_SIZE > 16384
>  	{32768, "32768", },
> +#if PAGE_SIZE > 32768
>  	{65536, "65536", },
> +#if PAGE_SIZE > 65536
> +#error "Unsupported PAGE_SIZE"
> +#endif
> +#endif
> +#endif
> +#endif
> +#endif
>  	{0, NULL},
>  };

This is incorrect.  I had to revert r266964 and r266963 in order to get the system to work again.

Just using this change results in a panic:
Copyright (c) 1992-2014 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
	The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.0-CURRENT #0 r271278M: Mon Sep  8 11:36:16 PDT 2014
    sbruno@alice:/home/sbruno/bsd/obj/mips/mips.mips/home/sbruno/bsd/fbsd_head/sys/WZR-300HP mips
gcc version 4.2.1 20070831 patched [FreeBSD]
WARNING: WITNESS option enabled, expect reduced performance.
MEMGUARD DEBUGGING ALLOCATOR INITIALIZED:
	MEMGUARD map base: 0xc0800000
	MEMGUARD map size: 104860 KBytes
Preloaded elf kernel "kernel" at 0x80646970.
real memory  = 67108864 (65536K bytes)
Physical memory chunk(s):
0x006e1000 - 0x03ebcfff, 58572800 bytes (14300 pages)
avail memory = 57790464 (55MB)
Trap cause = 2 (TLB miss (load or instr. fetch) - kernel mode)
[ thread pid 0 tid 0 ]
Stopped at      strncmp+0x8:    lb      v1,0(a0)
db> reset

Comment 4 Adrian Chadd 2014-09-12 19:38:12 UTC

Those two commits (266963/266964) aren't relevant. :P

Comment 5 Sean Bruno 2014-09-12 19:45:13 UTC

(In reply to Adrian Chadd from comment #4)
> Those two commits (266963/266964) aren't relevant. :P

This should read "I had to revert 269964 and 269963"

Comment 6 Sean Bruno 2014-09-13 22:36:39 UTC

Xin provided a patch on freebsd-src and I'm moving it here with results.

Patch:

Index: sys/kern/kern_malloc.c
===================================================================
--- sys/kern/kern_malloc.c      (revision 271494)
+++ sys/kern/kern_malloc.c      (working copy)
@@ -717,6 +717,8 @@ kmeminit(void)
         * a given architecture.
         */
        mem_size = vm_cnt.v_page_count;
+       if (mem_size <= 32768) /* delphij XXX 128MB */
+               kmem_zmax = PAGE_SIZE;

        if (vm_kmem_size_scale < 1)
                vm_kmem_size_scale = VM_KMEM_SIZE_SCALE;

Comment 7 Sean Bruno 2014-09-13 22:37:56 UTC

Created attachment 147287 [details]
vmstat -z with applied patch from Xin

Hrm ... we can boot again, however there's still availability of "stuff" above 4k.

That seems bad.

Comment 8 Adrian Chadd 2014-09-19 06:17:04 UTC

Hi,

So what's the verdict?

Can we only create those UMA zones entirely if we have substantially more RAM (say, 256MB) ?


-a

Comment 9 Sean Bruno 2014-09-19 15:26:45 UTC

(In reply to Adrian Chadd from comment #8)
> Hi,
> 
> So what's the verdict?
> 
> Can we only create those UMA zones entirely if we have substantially more
> RAM (say, 256MB) ?
> 
> 
> -a

Look at the vmstat -z that I posted with Xin's patch.

It clearly does *not* allocate above 4k now, but the buckets are there in the output of vmstat.  I don't know if that's a bad thing or not.

Comment 10 Adrian Chadd 2014-09-19 22:59:38 UTC

Ok, let's commit that for now and worry about the details after?

Comment 11 commit-hook 2014-09-22 05:08:00 UTC

A commit references this bug:

Author: sbruno
Date: Mon Sep 22 05:07:23 UTC 2014
New revision: 271944
URL: http://svnweb.freebsd.org/changeset/base/271944

Log:
  svn revisions r269964 and r269963 seemed to have impaired small memory
  footprint systems(32M/64M) and didn't leave enough free memory to load modules
  when it was setting up page tables that for sizes that are never used on
  these smallish boards.

  Set kmem_zmax to PAGE_SIZE on these smaller systems (< 128M) to keep this
  from happening. Verified on mips32 h/w.

  PR:             193465
  Submitted by:   delphij
  Reviewed by:    adrian

Changes:
  head/sys/kern/kern_malloc.c

Comment 12 Sean Bruno 2014-09-22 05:09:10 UTC

Yell at me if I didn't do this right.