Bug 174822 - [tcp] Page fault in tcp_discardcb under high traffic
Summary: [tcp] Page fault in tcp_discardcb under high traffic
Status: Closed Overcome By Events
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 9.1-RELEASE
Hardware: Any Any
: Normal Affects Only Me
Assignee: freebsd-bugs (Nobody)
Depends on:
Reported: 2012-12-30 08:40 UTC by adam.twardowski
Modified: 2020-11-30 22:13 UTC (History)
1 user (show)

See Also:


Note You need to log in before you can comment on or make changes to this bug.
Description adam.twardowski 2012-12-30 08:40:00 UTC
Random kernel page fault's under high network traffic traffic.  I'm running haproxy 1.4.22 to load balance traffic to a cluster of web servers.  I'm seeing the same error on multiple machines and I already ran memtest to verify that there wasn't any issues with the memory.

[root@ID11278 /var/crash]# kgdb /boot/kernel/kernel.symbols vmcore.0
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "amd64-marcel-freebsd"...

Unread portion of the kernel message buffer:

Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address   = 0x28
fault code              = supervisor read data, page not present
instruction pointer     = 0x20:0xffffffff80a7a951
stack pointer           = 0x28:0xffffff88c1836780
frame pointer           = 0x28:0xffffff88c1836800
code segment            = base rx0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 22494 (haproxy)
trap number             = 12
panic: page fault
cpuid = 0
KDB: stack backtrace:
#0 0xffffffff809208a6 at kdb_backtrace+0x66
#1 0xffffffff808ea8be at panic+0x1ce
#2 0xffffffff80bd8240 at trap_fatal+0x290
#3 0xffffffff80bd857d at trap_pfault+0x1ed
#4 0xffffffff80bd8b9e at trap+0x3ce
#5 0xffffffff80bc315f at calltrap+0x8
#6 0xffffffff80a80ffa at tcp_usr_detach+0x11a
#7 0xffffffff80952f84 at sofree+0x134
#8 0xffffffff80953a81 at soclose+0x3c1
#9 0xffffffff808a9653 at _fdrop+0x23
#10 0xffffffff808aa952 at closef+0x52
#11 0xffffffff808ab192 at kern_close+0x172
#12 0xffffffff80bd7ae6 at amd64_syscall+0x546
#13 0xffffffff80bc3447 at Xfast_syscall+0xf7
Uptime: 3d2h59m9s
Dumping 1980 out of 32726 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%

Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /boot/kernel/geom_mirror.ko.symbols...done.
Loaded symbols for /boot/kernel/geom_mirror.ko
Reading symbols from /boot/modules/dscp_rewrite.ko...done.
Loaded symbols for /boot/modules/dscp_rewrite.ko
Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from /boot/kernel/ipfw.ko.symbols...done.
Loaded symbols for /boot/kernel/ipfw.ko
#0  doadump (textdump=Variable "textdump" is not available.
) at pcpu.h:224
224     pcpu.h: No such file or directory.
        in pcpu.h
(kgdb) bt
#0  doadump (textdump=Variable "textdump" is not available.
) at pcpu.h:224
#1  0xffffffff808ea3a1 in kern_reboot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:448
#2  0xffffffff808ea897 in panic (fmt=0x1 <Address 0x1 out of bounds>) at /usr/src/sys/kern/kern_shutdown.c:636
#3  0xffffffff80bd8240 in trap_fatal (frame=0xc, eva=Variable "eva" is not available.
) at /usr/src/sys/amd64/amd64/trap.c:857
#4  0xffffffff80bd857d in trap_pfault (frame=0xffffff88c18366d0, usermode=0) at /usr/src/sys/amd64/amd64/trap.c:773
#5  0xffffffff80bd8b9e in trap (frame=0xffffff88c18366d0) at /usr/src/sys/amd64/amd64/trap.c:456
#6  0xffffffff80bc315f in calltrap () at /usr/src/sys/amd64/amd64/exception.S:228
#7  0xffffffff80a7a951 in tcp_discardcb (tp=0x0) at /usr/src/sys/netinet/tcp_subr.c:839
#8  0xffffffff80a80ffa in tcp_usr_detach (so=Variable "so" is not available.
) at /usr/src/sys/netinet/tcp_usrreq.c:205
#9  0xffffffff80952f84 in sofree (so=0xfffffe05e6dbe7f8) at /usr/src/sys/kern/uipc_socket.c:647
#10 0xffffffff80953a81 in soclose (so=0xfffffe05e6dbe7f8) at /usr/src/sys/kern/uipc_socket.c:741
#11 0xffffffff808a9653 in _fdrop (fp=0xfffffe01d5f18280, td=Variable "td" is not available.
) at file.h:316
#12 0xffffffff808aa952 in closef (fp=0xfffffe01d5f18280, td=0xfffffe000fcd28e0) at /usr/src/sys/kern/kern_descrip.c:2250
#13 0xffffffff808ab192 in kern_close (td=0xfffffe000fcd28e0, fd=Variable "fd" is not available.
) at /usr/src/sys/kern/kern_descrip.c:1236
#14 0xffffffff80bd7ae6 in amd64_syscall (td=0xfffffe000fcd28e0, traced=0) at subr_syscall.c:135
#15 0xffffffff80bc3447 in Xfast_syscall () at /usr/src/sys/amd64/amd64/exception.S:387
#16 0x0000000800ff1cfc in ?? ()
Previous frame inner to this frame (corrupt stack?)

How-To-Repeat: Not easily repeatable, but it seems to happen to me every couple of days at least.
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2012-12-31 00:58:21 UTC
Responsible Changed
From-To: freebsd-bugs->freebsd-net

Over to maintainer(s).
Comment 2 Eitan Adler freebsd_committer freebsd_triage 2017-12-31 07:59:21 UTC
For bugs matching the following criteria:

Status: In Progress Changed: (is less than) 2014-06-01

Reset to default assignee and clear in-progress tags.

Mail being skipped
Comment 3 Michael Tuexen freebsd_committer 2020-11-30 22:13:10 UTC
FreeBSD 9.1 is not supported anymore. Please reopen if the problem still exists on a supported version of FreeBSD.