Bug 121485 - [vm] panic with 7.0-RELEASE [regression]
Summary: [vm] panic with 7.0-RELEASE [regression]
Status: Open
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 7.0-RELEASE
Hardware: Any Any
: Normal Affects Only Me
Assignee: Volker Werth
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-03-08 01:20 UTC by Rory Arms
Modified: 2018-05-28 19:49 UTC (History)
0 users

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Rory Arms 2008-03-08 01:20:00 UTC
7.0-RELEASE panic'd. I had the system configured to do a minidump. So,
I used kgdb to get a backtrace. This is a dual processor Pentium II 375 Mhz.
It is on a Tyan Thunder100 motherboard. It has 1 GiB of memory and ACPI is
disabled both in the BIOS and in FreeBSD, as with it on, FreeBSD considers
it to be blacklisted. This system has used FreeBSD since at least 4.1-RELEASE.
Previous to 7.0, it was running 6.3 and seemed to be stable, there were no
crashes. I think the last time it crashed was an early 6 or a 5.x release.
The issue back them seemed to relate to a bug in networking when running
out of GIANT, as the only way I could keep the system stable is to boot with
debug.mpsafenet="0" in loader.conf(5). As long as it wasn't allowed to run as
MP safe, it was stable. With this 7.0-RELEASE I've turned off that setting,
in hopes that the bug is now resolved.

Here's the backtrace from kgdb(1). Keep in mind this is the first time I've
had a crash after turning on minidump support, so I'm assuming the procedures
to analyze the core file are the same with this setting. The system was in
the process of recompiling ports with portupgrade(1)

> sudo kgdb /boot/kernel/kernel /usr/local/crash/vmcore.0 
Password:
[GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd".

Unread portion of the kernel message buffer:
pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 17859 (ruby18)
trap number             = 12
panic: page fault
cpuid = 0
Uptime: 1d4h38m24s
Physical memory: 1015 MB
Dumping 206 MB: 191 175 159 143 127 111 95 79 63 47 31 15

#0  doadump () at pcpu.h:195
195     pcpu.h: No such file or directory.
        in pcpu.h
(kgdb) bt
#0  doadump () at pcpu.h:195
#1  0xc059f866 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:409
#2  0xc059fb3e in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:563
#3  0xc080bf9c in trap_fatal (frame=0xe66e8968, eva=0)
    at /usr/src/sys/i386/i386/trap.c:899
#4  0xc080c20b in trap_pfault (frame=0xe66e8968, usermode=0, eva=0)
    at /usr/src/sys/i386/i386/trap.c:812
#5  0xc080cc02 in trap (frame=0xe66e8968) at /usr/src/sys/i386/i386/trap.c:490
#6  0xc07f384b in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#7  0xc0807689 in pmap_clear_modify (m=0xc1fa5ca8)
    at /usr/src/sys/i386/i386/pmap.c:3309
#8  0xc07ae99c in vm_page_set_validclean (m=0xc1fa5ca8, base=0, size=4096)
    at /usr/src/sys/vm/vm_page.c:1806
#9  0xc0600a77 in vfs_page_set_valid (bp=Variable "bp" is not available.
) at /usr/src/sys/kern/vfs_bio.c:3391
#10 0xc06072c7 in bdwrite (bp=0xd7d81074) at /usr/src/sys/kern/vfs_bio.c:3507
#11 0xc07852c9 in ffs_write (ap=0xe66e8bbc)
    at /usr/src/sys/ufs/ffs/ffs_vnops.c:780
#12 0xc08222e4 in VOP_WRITE_APV (vop=0xc08b8920, a=0xe66e8bbc)
    at vnode_if.c:691
#13 0xc06294c1 in vn_write (fp=0xc86e3a20, uio=0xe66e8c60, 
    active_cred=0xc46e5500, flags=1, td=0xc469c630) at vnode_if.h:373
#14 0xc05d3657 in dofilewrite (td=0xc469c630, fd=3, fp=0xc86e3a20, 
    auio=0xe66e8c60, offset=4189184, flags=1) at file.h:254
---Type <return> to continue, or q <return> to quit---
#15 0xc05d3825 in kern_pwritev (td=0xc469c630, fd=3, auio=0xe66e8c60, 
    offset=4189184) at /usr/src/sys/kern/sys_generic.c:449
#16 0xc05d38ad in pwrite (td=0xc469c630, uap=0xe66e8cfc)
    at /usr/src/sys/kern/sys_generic.c:350
#17 0xc080c595 in syscall (frame=0xe66e8d38)
    at /usr/src/sys/i386/i386/trap.c:1035
#18 0xc07f38b0 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:196
#19 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb) 

Loaded KLDs:

> kldstat 
Id Refs Address    Size     Name
 1    8 0xc0400000 5b8030   kernel
 2    1 0xc09b9000 5844     if_tap.ko
 3    1 0xc3fcc000 9000     if_bridge.ko
 4    1 0xc3fd5000 6000     bridgestp.ko
 5    2 0xc4092000 d000     ipfw.ko
 6    1 0xc40be000 4000     ipdivert.ko
 7    1 0xc4514000 2000     green_saver.ko

the dmesg(1):

> dmesg 
Copyright (c) 1992-2008 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 7.0-RELEASE #1: Wed Mar  5 17:52:52 EST 2008
    root@Tserver.TrueStep.com:/mnt/obj/usr/src/sys/TSERVER-70
Timecounter "i8254" frequency 1193182 Hz quality 0
CPU: Pentium II/Pentium II Xeon/Celeron (375.04-MHz 686-class CPU)
  Origin = "GenuineIntel"  Id = 0x652  Stepping = 2
  Features=0x183fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR>
real memory  = 1073741824 (1024 MB)
avail memory = 1041379328 (993 MB)
MPTable: <INTEL    440GX       >
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
ioapic0: Assuming intbase of 0
ioapic0 <Version 1.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413)
cpu0 on motherboard
cpu1 on motherboard
pcib0: <MPTable Host-PCI bridge> pcibus 0 on motherboard
pci0: <PCI bus> on pcib0
agp0: <Intel 82443GX host to PCI bridge> on hostb0
pcib1: <PCI-PCI bridge> at device 1.0 on pci0
pci1: <PCI bus> on pcib1
vgapci0: <VGA-compatible display> port 0xc800-0xc8ff mem 0xfd000000-0xfdffffff,0xfe1ff000-0xfe1fffff at device 0.0 on pci1
isab0: <PCI-ISA bridge> at device 7.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX4 UDMA33 controller> port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xffa0-0xffaf at device 7.1 on pci0
ata0: <ATA channel 0> on atapci0
ata0: [ITHREAD]
ata1: <ATA channel 1> on atapci0
ata1: [ITHREAD]
uhci0: <Intel 82371AB/EB (PIIX4) USB controller> port 0xef80-0xef9f irq 19 at device 7.2 on pci0
uhci0: [GIANT-LOCKED]
uhci0: [ITHREAD]
usb0: <Intel 82371AB/EB (PIIX4) USB controller> on uhci0
usb0: USB revision 1.0
uhub0: <Intel UHCI root hub, class 9/0, rev 1.00/1.00, addr 1> on usb0
uhub0: 2 ports with 2 removable, self powered
piix0: <PIIX Timecounter> port 0x440-0x44f at device 7.3 on pci0
Timecounter "PIIX" frequency 3579545 Hz quality 0
pcib2: <MPTable PCI-PCI bridge> at device 16.0 on pci0
pci2: <PCI bus> on pcib2
ath0: <Atheros 5212> mem 0xfe6f0000-0xfe6fffff irq 16 at device 4.0 on pci2
ath0: [ITHREAD]
ath0: using obsoleted if_watchdog interface
ath0: Ethernet address: 00:0f:3d:ad:b9:f2
ath0: mac 5.9 phy 4.3 radio 4.6
fxp0: <Intel 82559 Pro/100 Ethernet> port 0xdf00-0xdf3f mem 0xfe6ef000-0xfe6effff,0xfe500000-0xfe5fffff irq 17 at device 5.0 on pci2
miibus0: <MII bus> on fxp0
inphy0: <i82555 10/100 media interface> PHY 1 on miibus0
inphy0:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp0: Ethernet address: 00:90:27:ee:02:97
fxp0: [ITHREAD]
fxp1: <Intel 82558 Pro/100 Ethernet> port 0xef40-0xef5f mem 0xffaff000-0xffafffff,0xfea00000-0xfeafffff irq 19 at device 17.0 on pci0
miibus1: <MII bus> on fxp1
inphy1: <i82555 10/100 media interface> PHY 1 on miibus1
inphy1:  10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
fxp1: Ethernet address: 00:e0:81:10:22:27
fxp1: [ITHREAD]
ahc0: <Adaptec aic7895 Ultra SCSI adapter> port 0xe400-0xe4ff mem 0xfebfe000-0xfebfefff irq 16 at device 18.0 on pci0
ahc0: [ITHREAD]
aic7895C: Ultra Wide Channel A, SCSI Id=7, 32/253 SCBs
ahc1: <Adaptec aic7895 Ultra SCSI adapter> port 0xe800-0xe8ff mem 0xfebff000-0xfebfffff irq 16 at device 18.1 on pci0
ahc1: [ITHREAD]
aic7895C: Ultra Wide Channel B, SCSI Id=7, 32/253 SCBs
ahc2: <Adaptec 2902/04/10/15/20C/30C SCSI adapter> port 0xe000-0xe0ff mem 0xfebfd000-0xfebfdfff irq 16 at device 19.0 on pci0
ahc2: [ITHREAD]
aic7850: Single Channel A, SCSI Id=7, 3/253 SCBs
pmtimer0 on isa0
orm0: <ISA Option ROMs> at iomem 0xc0000-0xcbfff,0xcc000-0xd07ff pnpid ORM0000 on isa0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
atkbd0: [ITHREAD]
fdc0: <Enhanced floppy controller> at port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on isa0
fdc0: [FILTER]
fd0: <1440-KB 3.5" drive> on fdc0 drive 0
ppc0: <Parallel port> at port 0x378-0x37f irq 7 on isa0
ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode
ppc0: FIFO with 16/16/8 bytes threshold
ppbus0: <Parallel port bus> on ppc0
ppbus0: [ITHREAD]
plip0: <PLIP network interface> on ppbus0
lpt0: <Printer> on ppbus0
lpt0: Interrupt-driven port
ppi0: <Parallel I/O> on ppbus0
ppc0: [GIANT-LOCKED]
ppc0: [ITHREAD]
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
sio0 at port 0x3f8-0x3ff irq 4 flags 0x10 on isa0
sio0: type 16550A, console
sio0: [FILTER]
sio1 at port 0x2f8-0x2ff irq 3 on isa0
sio1: type 16550A
sio1: [FILTER]
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
unknown: <PNP0c01> can't assign resources (memory)
unknown: <PNP0303> can't assign resources (port)
unknown: <PNP0c02> can't assign resources (port)
unknown: <PNP0501> can't assign resources (port)
unknown: <PNP0501> can't assign resources (port)
unknown: <PNP0401> can't assign resources (port)
unknown: <PNP0700> can't assign resources (port)
ugen0: <American Power Conversion Back-UPS RS 1500 FW:8.g9 .D USB FW:g9, class 0/0, rev 1.10/1.06, addr 2> on uhub0
Timecounters tick eahc2: Someone reset channel A
very 1.000 msec
Waiting 5 seconds for SCSI devices to settle
ad0: 156334MB <Maxtor 6Y160P0 YAR41BW0> at ata0-master UDMA33
GEOM_LABEL: Label for provider ad0s1a is ufs/Network.
da0 at ahc0 bus 0 target 2 lun 0
da0: <SEAGATE ST410800N 7117> Fixed Direct Access SCSI-2 device 
da0: 10.000MB/s transfers (10.000MHz, offset 15)
da0: 8347MB (17096357 512 byte sectors: 255H 63S/T 1064C)
da1 at ahc0 bus 0 target 6 lun 0
da1: <QUANTUM FIREBALL SE8.4S PJ0A> Fixed Direct Access SCSI-2 device 
da1: 20.000MB/s transfers (20.000MHz, offset 15)
da1: Command Queueing Enabled
da1: 8191MB (16777215 512 byte sectors: 255H 63S/T 1044C)
SMP: AP CPU #1 Launched!
Trying to mount root from ufs:/dev/da0s1a
WARNING: / was not properly dismounted
WARNING: /usr was not properly dismounted
/usr: mount pending error: blocks 200 files 4
WARNING: /var was not properly dismounted
/var: mount pending error: blocks 968 files 1
GWARNING: /Network was not properly dismounted
EOM_LABEL: Label ufs/Network removed.
bridge0: Ethernet address: b2:4f:63:a9:da:42
ath0: ath_chan_set: unable to reset channel 6 (2437 Mhz, flags 0x490 hal flags 0x150)
ipfw2 (+ipv6) initialized, divert loadable, rule-based forwarding disabled, default to deny, logging disabled
pid 1444 (mu-conference), uid 1012: exited on signal 11


Lastly the KERNCONF that I used to compile TSERVER-70:

cpu             I686_CPU
ident           TSERVER-70


makeoptions     DEBUG=-g                # Build kernel with gdb(1) debug symbols

options         SCHED_4BSD              # 4BSD scheduler
options         PREEMPTION              # Enable kernel thread preemption
options         INET                    # InterNETworking
options         INET6                   # IPv6 communications protocols
options         SCTP                    # Stream Control Transmission Protocol
options         FFS                     # Berkeley Fast Filesystem
options         SOFTUPDATES             # Enable FFS soft updates support
options         UFS_ACL                 # Support for access control lists
options         UFS_DIRHASH             # Improve performance on big directories
options         UFS_GJOURNAL            # Enable gjournal-based UFS journaling
options         NFSCLIENT               # Network Filesystem Client
options         NFSSERVER               # Network Filesystem Server
options         MSDOSFS                 # MSDOS Filesystem
options         CD9660                  # ISO 9660 Filesystem
options         PROCFS                  # Process filesystem (requires PSEUDOFS)
options         PSEUDOFS                # Pseudo-filesystem framework
options         GEOM_PART_GPT           # GUID Partition Tables.
options         GEOM_LABEL              # Provides labelization
options         COMPAT_43TTY            # BSD 4.3 TTY compat [KEEP THIS!]
options         COMPAT_FREEBSD6         # Compatible with FreeBSD6
options         SCSI_DELAY=5000         # Delay (in ms) before probing SCSI
options         KTRACE                  # ktrace(1) support
options         SYSVSHM                 # SYSV-style shared memory
options         SYSVMSG                 # SYSV-style message queues
options         SYSVSEM                 # SYSV-style semaphores
options         _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions
options         KBD_INSTALL_CDEV        # install a CDEV entry in /dev
options         ADAPTIVE_GIANT          # Giant mutex is adaptive.
options         STOP_NMI                # Stop CPUS using NMI instead of IPI
options         AUDIT                   # Security event auditing

options         SMP                     # Symmetric MultiProcessor Kernel
device          apic                    # I/O APIC

device          cpufreq

device          pci

device          fdc

device          ata
device          atadisk         # ATA disk drives
device          ataraid         # ATA RAID drives
device          atapicd         # ATAPI CDROM drives
options         ATA_STATIC_ID   # Static device numbering

device          ahc             # AHA2940 and onboard AIC7xxx devices
options         AHC_REG_PRETTY_PRINT    # Print register bitfields in debug
                                        # output.  Adds ~128k to driver.
                                        # output.  Adds ~215k to driver.



device          scbus           # SCSI bus (required for SCSI)
device          ch              # SCSI media changers
device          da              # Direct Access (disks)
device          sa              # Sequential Access (tape etc)
device          cd              # CD
device          pass            # Passthrough device (direct SCSI access)
device          ses             # SCSI Environmental Services (and SAF-TE)



device          atkbdc          # AT keyboard controller
device          atkbd           # AT keyboard
device          psm             # PS/2 mouse

device          kbdmux          # keyboard multiplexer

device          vga             # VGA video card driver

device          splash          # Splash screen and screen saver support

device          sc

device          agp             # support several AGP chipsets

device          pmtimer

device          cbb             # cardbus (yenta) bridge
device          pccard          # PC Card (16-bit) bus
device          cardbus         # CardBus (32-bit) bus

device          sio             # 8250, 16[45]50 based serial ports
device          uart            # Generic UART driver

device          ppc
device          ppbus           # Parallel port bus (required)
device          lpt             # Printer
device          plip            # TCP/IP over parallel
device          ppi             # Parallel port interface device



device          miibus          # MII bus support
device          fxp             # Intel EtherExpress PRO/100B (82557, 82558)


device          wlan            # 802.11 support
device          wlan_wep        # 802.11 WEP support
device          wlan_ccmp       # 802.11 CCMP support
device          wlan_tkip       # 802.11 TKIP support
device          wlan_amrr       # AMRR transmit rate control algorithm
device          wlan_scan_ap    # 802.11 AP mode scanning
device          wlan_scan_sta   # 802.11 STA mode scanning
device          ath             # Atheros pci/cardbus NIC's
device          ath_hal         # Atheros HAL (Hardware Access Layer)
device          ath_rate_sample # SampleRate tx rate control for ath

device          loop            # Network loopback
device          random          # Entropy device
device          ether           # Ethernet support
device          sl              # Kernel SLIP
device          ppp             # Kernel PPP
device          tun             # Packet tunnel.
device          pty             # Pseudo-ttys (telnet etc)
device          md              # Memory "disks"
device          gif             # IPv6 and IPv4 tunneling
device          faith           # IPv6-to-IPv4 relaying (translation)
device          firmware        # firmware assist module

device          bpf             # Berkeley packet filter

device          uhci            # UHCI PCI->USB interface
device          ohci            # OHCI PCI->USB interface
device          usb             # USB Bus (required)
device          ugen            # Generic
device          uhid            # "Human Interface Devices"
device          ukbd            # Keyboard
device          ulpt            # Printer
device          umass           # Disks/Mass storage - Requires scbus and da
device          ums             # Mouse
device          ural            # Ralink Technology RT2500USB wireless NICs
device          rum             # Ralink Technology RT2501USB wireless NICs

How-To-Repeat: None, this has only happened once so far. The system has been running
7.0-RELEASE for about 2 days though.
Comment 1 Volker Werth freebsd_committer 2008-03-11 03:53:01 UTC
State Changed
From-To: open->feedback


Rory, 
can you please enter post mortem debugger (kgdb) again with your core file and 
see what "p panicstr" gives you?
Comment 2 Volker Werth freebsd_committer 2008-03-11 05:13:52 UTC
State Changed
From-To: feedback->suspended


feedback received, suspend for now until further information is available when reproduceable again 


Comment 3 Volker Werth freebsd_committer 2008-03-11 05:13:52 UTC
Responsible Changed
From-To: freebsd-bugs->vwe


track
Comment 4 rorya 2008-05-16 17:52:50 UTC
Ok, I have a new panic to follow this up with. However the code path looks 
different than in the one originally reported. But this is with the 
same version of FreeBSD as the original report, nothing has changed 
ather than applying some security updates and port updates. This was after 
58 days of uptime.

# kgdb -c vmcore.0 /boot/kernel/kernel
[GDB will not be able to debug user-mode threads: 
/usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you 
are
welcome to change it and/or distribute copies of it under certain 
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for 
details.
This GDB was configured as "i386-marcel-freebsd".

Unread portion of the kernel message buffer:
08
fault code              = supervisor write, page not present
instruction pointer     = 0x20:0xc08152c9
stack pointer           = 0x28:0xe6802bc4
frame pointer           = 0x28:0xe6802c04
code segment            = base rx0, limit 0xfffff, type 0x1b
                         = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 81696 (sendmail)
trap number             = 12
panic: page fault
cpuid = 1
Uptime: 68d7h20m21s
Physical memory: 1015 MB
Dumping 253 MB: 238 222 206 190 174 158 142 126 110 94 78 62 46 30 14

#0  doadump () at pcpu.h:195
195     pcpu.h: No such file or directory.
         in pcpu.h
(kgdb) bt
#0  doadump () at pcpu.h:195
#1  0xc059fbd6 in boot (howto=260) at 
/usr/src/sys/kern/kern_shutdown.c:409
#2  0xc059feae in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:563
#3  0xc08190cc in trap_fatal (frame=0xe6802b84, eva=1048584)
     at /usr/src/sys/i386/i386/trap.c:899
#4  0xc081933b in trap_pfault (frame=0xe6802b84, usermode=0, eva=1048584)
     at /usr/src/sys/i386/i386/trap.c:812
#5  0xc0819d32 in trap (frame=0xe6802b84) at 
/usr/src/sys/i386/i386/trap.c:490
#6  0xc080097b in calltrap () at /usr/src/sys/i386/i386/exception.s:139
#7  0xc08152c9 in pmap_remove_pages (pmap=0xc4667514)
     at /usr/src/sys/i386/i386/pmap.c:3124
#8  0xc07b573c in vmspace_exit (td=0xc5193c60) at 
/usr/src/sys/vm/vm_map.c:404
#9  0xc057dbe4 in exit1 (td=0xc5193c60, rv=0)
     at /usr/src/sys/kern/kern_exit.c:294
#10 0xc057ef1d in sys_exit (td=Could not find the frame base for 
"sys_exit".
) at /usr/src/sys/kern/kern_exit.c:98
#11 0xc08196c5 in syscall (frame=0xe6802d38)
     at /usr/src/sys/i386/i386/trap.c:1035
#12 0xc08009e0 in Xint0x80_syscall () at 
/usr/src/sys/i386/i386/exception.s:196
#13 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb) p panicstr
$1 = 0xc08f3e00 "page fault"
(kgdb)
Comment 5 Alan Cox 2008-08-02 05:18:18 UTC
The second panic with pmap_remove_pages() in the call stack almost 
always indicates that the machine has flakey memory.  It would be worth 
running memtest86 or memtest86+ on this machine.

Regards,
Alan
Comment 6 John Baldwin freebsd_committer freebsd_triage 2009-03-19 21:25:54 UTC
Based on my experience, this is almost certainly a hardware issue.

-- 
John Baldwin
Comment 7 pwsouth 2009-08-13 14:04:28 UTC
I have what appears to be a similar crash using FreeBSD 7.2-RELEASE-p3
(GENERIC).  The crash occurs reliably when running a full dump of
/dev/twed0s1f and writing the output to a filesystem on /dev/ad8s1c
which is only used for backups.  However if I run the same backup and
pipe the output via ssh to a remote machine over the network, it
doesn't crash.  I did newfs /dev/ad8s1c again and it still crashes
(that is vmcore.4 appended below the first dump).  If this is bad
hardware, can you suggest a way to identify the faulty part?   Thanks,
 --Paul


[root@f2 /usr/obj/usr/src/sys/GENERIC]# kgdb kernel.debug /var/crash/vmcore.3
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...

Unread portion of the kernel message buffer:


Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address   = 0x10002c30
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0xc08496e8
stack pointer           = 0x28:0xe91d681c
frame pointer           = 0x28:0xe91d6838
code segment            = base rx0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 98027 (split)
trap number             = 12
panic: page fault
cpuid = 1
Uptime: 2d22h51m11s
Physical memory: 3179 MB
Dumping 309 MB:twe0: completion event for nonbusy command
 294 278 262 246 230 214 198 182 166 150 134 118 102 86 70 54 38 22 6

Reading symbols from /boot/kernel/acpi.ko...Reading symbols from
/boot/kernel/acpi.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/acpi.ko
Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from
/boot/kernel/ipfw.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ipfw.ko
#0  doadump () at pcpu.h:196
196             __asm __volatile("movl %%fs:0,%0" : "=r" (td));


(kgdb) p panicstr
$1 = 0xc0caff60 "page fault"


(kgdb) list *0xc08496e8
0xc08496e8 is in vfs_page_set_valid (/usr/src/sys/kern/vfs_bio.c:3386).
3381             * page boundry or cross the end of the buffer.  The end of the
3382             * buffer, in this case, is our file EOF, not the
allocation size
3383             * of the buffer.
3384             */
3385            soff = off;
3386            eoff = (off + PAGE_SIZE) & ~(off_t)PAGE_MASK;
3387            if (eoff > bp->b_offset + bp->b_bcount)
3388                    eoff = bp->b_offset + bp->b_bcount;
3389
3390            /*


(kgdb) backtrace
#0  doadump () at pcpu.h:196
#1  0xc07e25c7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
#2  0xc07e2899 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:574
#3  0xc0ae3efc in trap_fatal (frame=0xe91d67dc, eva=268446768) at
/usr/src/sys/i386/i386/trap.c:939
#4  0xc0ae4180 in trap_pfault (frame=0xe91d67dc, usermode=0,
eva=268446768) at /usr/src/sys/i386/i386/trap.c:852
#5  0xc0ae4b2c in trap (frame=0xe91d67dc) at /usr/src/sys/i386/i386/trap.c:530
#6  0xc0ac923b in calltrap () at /usr/src/sys/i386/i386/exception.s:159
#7  0xc08496e8 in vfs_page_set_valid (bp=0xdaa62bfc, off=-67293184,
pageno=3, m=0xc2669038) at /usr/src/sys/kern/vfs_bio.c:3386
#8  0xc084fdf7 in bdwrite (bp=0xdaa62bfc) at /usr/src/sys/kern/vfs_bio.c:3511
#9  0xc09e0e52 in ffs_balloc_ufs2 (vp=0xc8413228, startoffset=Variable
"startoffset" is not available.
) at /usr/src/sys/ufs/ffs/ffs_balloc.c:882
#10 0xc09fef05 in ffs_write (ap=0xe91d6bc4) at
/usr/src/sys/ufs/ffs/ffs_vnops.c:724
#11 0xc0afa236 in VOP_WRITE_APV (vop=0xc0c6c160, a=0xe91d6bc4) at vnode_if.c:691
#12 0xc0871507 in vn_write (fp=0xc784ada8, uio=0xe91d6c60,
active_cred=0xc6e4be00, flags=0, td=0xc6e9faf0) at vnode_if.h:373
#13 0xc081b647 in dofilewrite (td=0xc6e9faf0, fd=1, fp=0xc784ada8,
auio=0xe91d6c60, offset=-1, flags=0) at file.h:257
#14 0xc081b928 in kern_writev (td=0xc6e9faf0, fd=1, auio=0xe91d6c60)
at /usr/src/sys/kern/sys_generic.c:402
#15 0xc081b99f in write (td=0xc6e9faf0, uap=0xe91d6cfc) at
/usr/src/sys/kern/sys_generic.c:318
#16 0xc0ae44d5 in syscall (frame=0xe91d6d38) at
/usr/src/sys/i386/i386/trap.c:1090
#17 0xc0ac92a0 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:255
#18 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)


(kgdb) up
#1  0xc07e25c7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
418                     doadump();
(kgdb) up
#2  0xc07e2899 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:574
574             boot(bootopt);
(kgdb) up
#3  0xc0ae3efc in trap_fatal (frame=0xe91d67dc, eva=268446768) at
/usr/src/sys/i386/i386/trap.c:939
939                     panic("%s", trap_msg[type]);
(kgdb) up
#4  0xc0ae4180 in trap_pfault (frame=0xe91d67dc, usermode=0,
eva=268446768) at /usr/src/sys/i386/i386/trap.c:852
852                     trap_fatal(frame, eva);
(kgdb) up
#5  0xc0ae4b2c in trap (frame=0xe91d67dc) at /usr/src/sys/i386/i386/trap.c:530
530                             (void) trap_pfault(frame, FALSE, eva);
(kgdb) up
#6  0xc0ac923b in calltrap () at /usr/src/sys/i386/i386/exception.s:159
159             call    trap
Current language:  auto; currently asm
(kgdb) up
#7  0xc08496e8 in vfs_page_set_valid (bp=0xdaa62bfc, off=-67293184,
pageno=3, m=0xc2669038) at /usr/src/sys/kern/vfs_bio.c:3386
3386            eoff = (off + PAGE_SIZE) & ~(off_t)PAGE_MASK;
Current language:  auto; currently c
(kgdb) up
#8  0xc084fdf7 in bdwrite (bp=0xdaa62bfc) at /usr/src/sys/kern/vfs_bio.c:3511
3511                    vfs_page_set_valid(bp, foff, i, m);
(kgdb) up
#9  0xc09e0e52 in ffs_balloc_ufs2 (vp=0xc8413228, startoffset=Variable
"startoffset" is not available.
) at /usr/src/sys/ufs/ffs/ffs_balloc.c:882
882                             bdwrite(bp);
(kgdb) up
#10 0xc09fef05 in ffs_write (ap=0xe91d6bc4) at
/usr/src/sys/ufs/ffs/ffs_vnops.c:724
724                     error = UFS_BALLOC(vp, uio->uio_offset, xfersize,
(kgdb) up
#11 0xc0afa236 in VOP_WRITE_APV (vop=0xc0c6c160, a=0xe91d6bc4) at vnode_if.c:691
691                     rc = vop->vop_write(a);
(kgdb) up
#12 0xc0871507 in vn_write (fp=0xc784ada8, uio=0xe91d6c60,
active_cred=0xc6e4be00, flags=0, td=0xc6e9faf0) at vnode_if.h:373
373             return (VOP_WRITE_APV(vp->v_op, &a));
(kgdb) up
#13 0xc081b647 in dofilewrite (td=0xc6e9faf0, fd=1, fp=0xc784ada8,
auio=0xe91d6c60, offset=-1, flags=0) at file.h:257
257             return ((*fp->f_ops->fo_write)(fp, uio, active_cred,
flags, td));
(kgdb) up
#14 0xc081b928 in kern_writev (td=0xc6e9faf0, fd=1, auio=0xe91d6c60)
at /usr/src/sys/kern/sys_generic.c:402
402             error = dofilewrite(td, fd, fp, auio, (off_t)-1, 0);
(kgdb) up
#15 0xc081b99f in write (td=0xc6e9faf0, uap=0xe91d6cfc) at
/usr/src/sys/kern/sys_generic.c:318
318             error = kern_writev(td, uap->fd, &auio);
(kgdb) up
#16 0xc0ae44d5 in syscall (frame=0xe91d6d38) at
/usr/src/sys/i386/i386/trap.c:1090
1090                    error = (*callp->sy_call)(td, args);
(kgdb) up
#17 0xc0ac92a0 in Xint0x80_syscall () at /usr/src/sys/i386/i386/exception.s:255
255             call    syscall
Current language:  auto; currently asm
(kgdb) up
#18 0x00000033 in ?? ()
(kgdb) up
Initial frame selected; you cannot go up.


(kgdb) [root@f2 /usr/obj/usr/src/sys/GENERIC]# kgdb kernel.debug
/var/crash/vmcore.4
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-marcel-freebsd"...

Unread portion of the kernel message buffer:


Fatal trap 12: page fault while in kernel mode
cpuid = 1; apic id = 01
fault virtual address   = 0xfdfff890
fault code              = supervisor read, page not present
instruction pointer     = 0x20:0xc084fdfa
stack pointer           = 0x28:0xe92df840
frame pointer           = 0x28:0xfdfff888
code segment            = base rx0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, def32 1, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
current process         = 5689 (split)
trap number             = 12
panic: page fault
cpuid = 1
Uptime: 4h32m29s
Physical memory: 3179 MB
Dumping 264 MB: 249 233 217 201 185 169 153 137 121 105 89 73 57 41 25 9

Reading symbols from /boot/kernel/acpi.ko...Reading symbols from
/boot/kernel/acpi.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/acpi.ko
Reading symbols from /boot/kernel/ipfw.ko...Reading symbols from
/boot/kernel/ipfw.ko.symbols...done.
done.
Loaded symbols for /boot/kernel/ipfw.ko
#0  doadump () at pcpu.h:196
196             __asm __volatile("movl %%fs:0,%0" : "=r" (td));


(kgdb) p panicstr
$1 = 0xc0caff60 "page fault"


(kgdb) list *0xc084fdfa
0xc084fdfa is in bdwrite (/usr/src/sys/kern/vfs_bio.c:3504).
3499            foff = bp->b_offset;
3500            KASSERT(bp->b_offset != NOOFFSET,
3501                ("vfs_clean_pages: no buffer offset"));
3502            VM_OBJECT_LOCK(bp->b_bufobj->bo_object);
3503            vm_page_lock_queues();
3504            for (i = 0; i < bp->b_npages; i++) {
3505                    m = bp->b_pages[i];
3506                    noff = (foff + PAGE_SIZE) & ~(off_t)PAGE_MASK;
3507                    eoff = noff;
3508


(kgdb) backtrace
#0  doadump () at pcpu.h:196
#1  0xc07e25c7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
#2  0xc07e2899 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:574
#3  0xc0ae3efc in trap_fatal (frame=0xe92df800, eva=4261410960) at
/usr/src/sys/i386/i386/trap.c:939
#4  0xc0ae4180 in trap_pfault (frame=0xe92df800, usermode=0,
eva=4261410960) at /usr/src/sys/i386/i386/trap.c:852
#5  0xc0ae4b2c in trap (frame=0xe92df800) at /usr/src/sys/i386/i386/trap.c:530
#6  0xc0ac923b in calltrap () at /usr/src/sys/i386/i386/exception.s:159
#7  0xc084fdfa in bdwrite (bp=Cannot access memory at address 0xfdfff890
) at /usr/src/sys/kern/vfs_bio.c:3504
Cannot access memory at address 0xfdfff88c


(kgdb) up
#1  0xc07e25c7 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:418
418                     doadump();
(kgdb) up
#2  0xc07e2899 in panic (fmt=Variable "fmt" is not available.
) at /usr/src/sys/kern/kern_shutdown.c:574
574             boot(bootopt);
(kgdb) up
#3  0xc0ae3efc in trap_fatal (frame=0xe92df800, eva=4261410960) at
/usr/src/sys/i386/i386/trap.c:939
939                     panic("%s", trap_msg[type]);
(kgdb) up
#4  0xc0ae4180 in trap_pfault (frame=0xe92df800, usermode=0,
eva=4261410960) at /usr/src/sys/i386/i386/trap.c:852
852                     trap_fatal(frame, eva);
(kgdb) up
#5  0xc0ae4b2c in trap (frame=0xe92df800) at /usr/src/sys/i386/i386/trap.c:530
530                             (void) trap_pfault(frame, FALSE, eva);
(kgdb) up
#6  0xc0ac923b in calltrap () at /usr/src/sys/i386/i386/exception.s:159
159             call    trap
Current language:  auto; currently asm
(kgdb) up
#7  0xc084fdfa in bdwrite (bp=Cannot access memory at address 0xfdfff890
) at /usr/src/sys/kern/vfs_bio.c:3504
3504            for (i = 0; i < bp->b_npages; i++) {
Current language:  auto; currently c
(kgdb) up
Cannot access memory at address 0xfdfff88c
Comment 8 Eitan Adler freebsd_committer freebsd_triage 2018-05-28 19:49:09 UTC
batch change:

For bugs that match the following
-  Status Is In progress 
AND
- Untouched since 2018-01-01.
AND
- Affects Base System OR Documentation

DO:

Reset to open status.


Note:
I did a quick pass but if you are getting this email it might be worthwhile to double check to see if this bug ought to be closed.