stable/11 got lots of kernel level code that abuses small default stack of i386 kernel threads: ZFS, IPSEC, SCTP, device drivers etc. Overflow of kernel stack produces "double fault" panics. kern.kstack_pages is loader tunnable now for i386. The system should be stable out of the box, so we should increase default for kern.kstack_pages. Loader tunnable can serve users of i386 systems that are unhappy with new default, they can decrease it with /boot/loader.conf Some examples: * sys/netinet/sctp_pcb.c, function sctp_load_addresses_from_init() allocates 2184 bytes on stack (disassemble: sub $0x888,%esp); * sys/netinet/sctp_auth.c: sctp_auth_get_cookie_params(): 1592 bytes on stack; * src/sys/contrib/dev/ath/ath_hal/ar9300/ar9300_eeprom.c: ar9300_eeprom_restore_internal_address(): 2092 bytes on stack; * sys/contrib/dev/ath/ath_hal/ar9300/ar9300_paprd.c: create_pa_curve(): 1416 bytes; * sys/libkern/zlib.c: huft_build(): 1420 bytes; And so on. Here is "top-list" for my home router custom kernel (1GB RAM, no swap): Bytes-on-stack In-module 2184 sctp_pcb.o 2092 ar9300_eeprom.o 2080 kern_linker.o 1664 cryptosoft.o 1592 sctp_auth.o 1536 glxsb_hash.o 1420 zlib.o 1416 ar9300_paprd.o 1352 scsi_da.o 1344 nfs_nfsdport.o 1328 vm_object.o 1312 fortuna.o 1232 cam_periph.o 1224 zlib.o 1192 cam_xpt.o 1192 ata_da.o 1184 cam_xpt.o 1168 ata_da.o 1160 sctp_output.o Some of "network hot path" subroutines not shown here do abuse kernel stack too, for example SHA256_Transform() from sys/crypto/sha2/sha256c.c that may be actively used with IPSEC processing. ae@'s https://reviews.freebsd.org/D10869 deals with IPSEC in part and helps in my case, at least while there is no ZFS involved. However, it is impossible to perform similar cleanup for each and every kernel subsystem in near future and double panics with current defaults are bad too.
This should probably have Hardware=i386, yes?
(In reply to Ed Maste from comment #1) Fixed, thanks!
A commit references this bug: Author: ae Date: Mon May 29 09:30:39 UTC 2017 New revision: 319118 URL: https://svnweb.freebsd.org/changeset/base/319118 Log: Disable IPsec debugging code by default when IPSEC_DEBUG kernel option is not specified. Due to the long call chain IPsec code can produce the kernel stack exhaustion on the i386 architecture. The debugging code usually is not used, but it requires a lot of stack space to keep buffers for strings formatting. This patch conditionally defines macros to disable building of IPsec debugging code. IPsec currently has two sysctl variables to configure debug output: * net.key.debug variable is used to enable debug output for PF_KEY protocol. Such debug messages are produced by KEYDBG() macro and usually they can be interesting for developers. * net.inet.ipsec.debug variable is used to enable debug output for DPRINTF() macro and ipseclog() function. DPRINTF() macro usually is used for development debugging. ipseclog() function is used for debugging by administrator. The patch disables KEYDBG() and DPRINTF() macros, and formatting buffers declarations when IPSEC_DEBUG is not present in kernel config. This reduces stack requirement for up to several hundreds of bytes. The net.inet.ipsec.debug variable still can be used to enable ipseclog() messages by administrator. PR: 219476 Reported by: eugen No objection from: #network MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D10869 Changes: head/sys/netipsec/ipsec.h head/sys/netipsec/ipsec_input.c head/sys/netipsec/ipsec_output.c head/sys/netipsec/key_debug.h head/sys/netipsec/xform_ah.c head/sys/netipsec/xform_esp.c head/sys/netipsec/xform_ipcomp.c
A commit references this bug: Author: ae Date: Mon Jun 5 11:11:08 UTC 2017 New revision: 319599 URL: https://svnweb.freebsd.org/changeset/base/319599 Log: MFC r319118: Disable IPsec debugging code by default when IPSEC_DEBUG kernel option is not specified. Due to the long call chain IPsec code can produce the kernel stack exhaustion on the i386 architecture. The debugging code usually is not used, but it requires a lot of stack space to keep buffers for strings formatting. This patch conditionally defines macros to disable building of IPsec debugging code. IPsec currently has two sysctl variables to configure debug output: * net.key.debug variable is used to enable debug output for PF_KEY protocol. Such debug messages are produced by KEYDBG() macro and usually they can be interesting for developers. * net.inet.ipsec.debug variable is used to enable debug output for DPRINTF() macro and ipseclog() function. DPRINTF() macro usually is used for development debugging. ipseclog() function is used for debugging by administrator. The patch disables KEYDBG() and DPRINTF() macros, and formatting buffers declarations when IPSEC_DEBUG is not present in kernel config. This reduces stack requirement for up to several hundreds of bytes. The net.inet.ipsec.debug variable still can be used to enable ipseclog() messages by administrator. PR: 219476 MFC r319412: Build kdebug_secreplay() function only when IPSEC_DEBUG is defined. This should fix the build on sparc. Approved by: re (kib) Changes: _U stable/11/ stable/11/sys/netipsec/ipsec.h stable/11/sys/netipsec/ipsec_input.c stable/11/sys/netipsec/ipsec_output.c stable/11/sys/netipsec/key_debug.c stable/11/sys/netipsec/key_debug.h stable/11/sys/netipsec/xform_ah.c stable/11/sys/netipsec/xform_esp.c stable/11/sys/netipsec/xform_ipcomp.c
ae@ did its job and this PR not longer concerns him but needs general discussion.
A commit references this bug: Author: cem Date: Mon Dec 11 04:32:37 UTC 2017 New revision: 326758 URL: https://svnweb.freebsd.org/changeset/base/326758 Log: i386: Bump KSTACK_PAGES default to match amd64 Logically, extend r286288 to cover all threads, by default. The world has largely moved on from i386. Most FreeBSD users and developers test on amd64 hardware. For better or worse, we have written a non-trivial amount of kernel code that relies on stacks larger than 8 kB, and it "just works" on amd64, so there has been little incentive to shrink it. amd64 had its KSTACK_PAGES bumped to 4 back in Peter's initial AMD64 commit, r114349, in 2003. Since that time, i386 has limped along on a stack half the size. We've even observed the stack overflows years ago, but neglected to fix the issue; see the 20121223 and 20150728 entries in UPDATING. If anyone is concerned with this change, I suggest they configure their AMD64 kernels with KSTACK_PAGES 2 and fix the fallout there first. Eugene has identified a list of high stack usage functions in the first PR below. PR: 219476, 224218 Reported by: eugen@, Shreesh Holla <hshreesh AT yahoo.com> Relnotes: maybe Sponsored by: Dell EMC Isilon Changes: head/sys/i386/conf/NOTES head/sys/i386/include/param.h
*** Bug 185427 has been marked as a duplicate of this bug. ***
Fixed and MFC'd.