Bug 258932 - panic: vm_page_free_prep: freeing mapped page
Summary: panic: vm_page_free_prep: freeing mapped page
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: amd64 Any
: --- Affects Only Me
Assignee: Mark Johnston
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-10-04 17:49 UTC by Bryan Drewery
Modified: 2022-09-21 20:11 UTC (History)
5 users (show)

See Also:

Attachments
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Bryan Drewery freebsd_committer freebsd_triage 2021-10-04 17:49:07 UTC 
Bug 256806 looks similar.

Tracing pid 3521 tid 915586 td 0xfffffe24f38d9740
kdb_enter() at kdb_enter+0x37/frame 0xfffffe25039bd460
vpanic() at vpanic+0x1b8/frame 0xfffffe25039bd4c0
panic() at panic+0x43/frame 0xfffffe25039bd520
vm_page_free_prep() at vm_page_free_prep+0x20c/frame 0xfffffe25039bd540
vm_page_free_toq() at vm_page_free_toq+0x12/frame 0xfffffe25039bd570
vm_object_page_remove() at vm_object_page_remove+0xb7/frame 0xfffffe25039bd5c0
vm_map_entry_delete() at vm_map_entry_delete+0x120/frame 0xfffffe25039bd610
vm_map_delete() at vm_map_delete+0xc8/frame 0xfffffe25039bd680
vm_map_remove() at vm_map_remove+0x81/frame 0xfffffe25039bd6b0
exec_new_vmspace() at exec_new_vmspace+0x1b7/frame 0xfffffe25039bd710
exec_elf64_imgact() at exec_elf64_imgact+0xa14/frame 0xfffffe25039bd800
kern_execve() at kern_execve+0x66d/frame 0xfffffe25039bdb70
sys_execve() at sys_execve+0x5a/frame 0xfffffe25039bdc00
amd64_syscall() at amd64_syscall+0x5db/frame 0xfffffe25039bdd30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe25039bdd30
--- syscall (59, FreeBSD ELF64, sys_execve), rip = 0x80045a62a, rsp = 0x7fffffff98e8, rbp = 0x7fffffff99f0 ---
db> x/s version
version:        FreeBSD 14.0-CURRENT #15 local-main-n248808-2390217a45d9: Wed Aug 18 15:26:33 PDT 2021\012    root@c1100-1.shatow.net:/usr/obj/usr/src/amd64.amd64/sys/LOCAL\012
                                         [This hash is likely useless but the date is close. I can't get the proper upstream hash until I exit ddb.]

db> show proc 3521
Process 3521 (tmux) at 0xfffff80757808a70:
 state: NORMAL
 uid: 0  gids: 0, 0, 5, 65531, 65532
 parent: pid 1634 at 0xfffff809dd32f000
 ABI: FreeBSD ELF64
 flag: 0x14000000  flag2: 0
 arguments: tmux: server (/tmp/tmux-0/default)
 reaper: 0xfffff801140cf538 reapsubtree: 1
 sigparent: 20
 vmspace: 0xfffffe2622d6c9f0
   (map 0xfffffe2622d6c9f0)
   (map.pmap 0xfffffe2622d6cab0)
   (pmap 0xfffffe2622d6cb10)
 threads: 1
915586                   Run     CPU 13                      tmux

db> show thread 915586
Thread 915586 at 0xfffffe24f38d9740:
 proc (pid 3521): 0xfffff80757808a70
 name: tmux
 pcb: 0xfffffe24f38d9c50
 stack: 0xfffffe25039ba000-0xfffffe25039bdfff
 flags: 0x20804  pflags: 0
 state: RUNNING (CPU 13)
 priority: 128
 container lock: sched lock 13 (0xfffffe2385f240c0)
 last voluntary switch: 0.026 s ago

db> show pginfo 0xfffffe006b2d04d8
page 0xfffffe006b2d04d8 obj 0xfffff802509ca210 pidx 0x0 phys 0x107d157000 q 0 ref 0x80000000
  af 0x9 of 0x0 f 0x1 act 0 busy f38d9742 valid 0xff dirty 0xff


db> show object 0xfffff802509ca210
Object 0xfffff802509ca210: type=1, size=0x4e9, res=368, ref=2, flags=0x3310 ruid 0 charge 4e9000
 sref=1, backing_object(2)=(0xfffff80e2bbab210)+0xf7000
  memory:=(off=0x0,page=0x107d157000),(off=0x1,page=0xa41695000),(off=0xf,page=0xe24eec000),(off=0x10,page=0xc5d709000),(off=0x11,page=0xbd53a2000),(off=0x16,page=0x10de242000)
   ...(off=0x17,page=0xd73f6f000),(off=0x18,page=0xf0f275000),(off=0x19,page=0xdfb393000),(off=0x1a,page=0x1138aba000),(off=0x1b,page=0xcdcf5d000),(off=0x1e,page=0x11e5959000)
   ...(off=0x1f,page=0x115bbfa000),(off=0x20,page=0x10e3f5e000),(off=0x21,page=0xd2f9cf000),(off=0x22,page=0xae2205000),(off=0x23,page=0xa364c9000),(off=0x24,page=0x113441a000)
   ...(off=0x25,page=0xa8e398000),(off=0x26,page=0x9d2ef1000),(off=0x27,page=0xeabf18000),(off=0x28,page=0xd38937000),(off=0x29,page=0x9e1704000),(off=0x2a,page=0x9ee01b000)
   ...(off=0x2b,page=0xa2c3e1000),(off=0x2c,page=0x1056386000),(off=0x2d,page=0x9b8a6f000),(off=0x2e,page=0x9d5a12000),(off=0x2f,page=0xadb6aa000),(off=0x30,page=0x10ab439000)
   ...(off=0x31,page=0xcac8b4000),(off=0x32,page=0xbaf3b5000),(off=0x33,page=0xd722e0000),(off=0x37,page=0xb35e19000),(off=0x38,page=0xf8d50d000),(off=0x39,page=0xf93d95000)
   ...(off=0x3a,page=0x11933a9000),(off=0x3b,page=0x11ea86a000),(off=0x3c,page=0x10eecbe000),(off=0x3d,page=0x1143f71000),(off=0x3e,page=0xe7f801000),(off=0x3f,page=0xed01d3000)
   ...(off=0x40,page=0xf2ee31000),(off=0x41,page=0x10113e8000),(off=0x42,page=0x119db7c000),(off=0x43,page=0x1005c96000),(off=0x44,page=0x10cebd8000),(off=0x45,page=0xb7ed4e000)
   ...(off=0x46,page=0xcdd31b000),(off=0x47,page=0x97cd8b000),(off=0x48,page=0xa7cde5000),(off=0x49,page=0x11c6cbe000),(off=0x4a,page=0xaea6eb000),(off=0x4b,page=0xe2281b000)
   ...(off=0x4c,page=0x10e9764000),(off=0x4d,page=0x98a424000),(off=0x4e,page=0x10cd3df000),(off=0x4f,page=0xa97c5d000),(off=0x50,page=0xcb0055000),(off=0x51,page=0xcccffa000)
   ...(off=0x52,page=0x111dd71000),(off=0x53,page=0xee4af8000),(off=0x54,page=0xacd187000),(off=0x55,page=0xc306ee000),(off=0x56,page=0xc2a56d000),(off=0x57,page=0xb7b45d000)
   ...(off=0x58,page=0x11c06d0000),(off=0x59,page=0x99b389000),(off=0x5a,page=0x1074300000),(off=0x5b,page=0x8af26e000),(off=0x5c,page=0x3625f5000),(off=0x5d,page=0x7337e3000)
   ...(off=0x5e,page=0xf18756000),(off=0x5f,page=0xeec35c000),(off=0x60,page=0xc99f4c000),(off=0x61,page=0xd84a9a000),(off=0x62,page=0xb425e9000),(off=0x63,page=0xf768ba000)
   ...(off=0x6b,page=0xebd3f9000),(off=0x6c,page=0xf999e6000),(off=0x6d,page=0x11f373c000),(off=0x71,page=0xefd3bd000),(off=0x72,page=0xe2766a000),(off=0x73,page=0xa1eeb8000)
   ...(off=0x74,page=0xf748d1000),(off=0x75,page=0xce943c000),(off=0x76,page=0x10b1133000),(off=0x77,page=0xea714c000),(off=0x78,page=0xf7e274000),(off=0x79,page=0x11077ee000)
   ...(off=0x7a,page=0x1000cfa000),(off=0x7b,page=0xa439ae000),(off=0x7c,page=0xa31f9b000),(off=0x81,page=0xc53a86000),(off=0x82,page=0xabefce000),(off=0x83,page=0xd1a038000)
   ...(off=0x84,page=0xf5e20b000),(off=0x85,page=0xfdf220000),(off=0x86,page=0xadd8b0000),(off=0x8b,page=0x1112832000),(off=0x8c,page=0x10b4298000),(off=0x8d,page=0xc70a59000)
   ...(off=0x8e,page=0xfd8991000),(off=0x8f,page=0xf3c8af000),(off=0x90,page=0xb57c36000),(off=0x91,page=0xfd3f16000),(off=0x92,page=0xe40552000),(off=0x93,page=0xf5de4f000)
   ...(off=0x94,page=0xa2f435000),(off=0x95,page=0xd41edc000),(off=0x96,page=0x115278f000),(off=0x97,page=0xba0b42000),(off=0x98,page=0x10526c3000),(off=0x99,page=0x101f247000)
   ...(off=0x9a,page=0xcbe20c000),(off=0x9b,page=0x114fbab000),(off=0x9c,page=0xf9af71000),(off=0xa0,page=0xf2c226000),(off=0xa1,page=0x9a520d000),(off=0xa2,page=0x9e0623000)
   ...(off=0xa3,page=0xebcdb4000),(off=0xa4,page=0xf33373000),(off=0xa5,page=0x110b364000),(off=0xa6,page=0x10f49b0000),(off=0xa7,page=0xb38db1000),(off=0xa8,page=0xaae059000)
   ...(off=0xa9,page=0xd23fb9000),(off=0xaa,page=0xd8768b000),(off=0xab,page=0x1008bca000),(off=0xb0,page=0xd1f3ff000),(off=0xb1,page=0xc38c2f000),(off=0xb2,page=0xc597f1000)
   ...(off=0xb3,page=0x10390af000),(off=0xb4,page=0xbde840000),(off=0xb5,page=0xaeb5c4000),(off=0xbc,page=0xcec70d000),(off=0xbd,page=0x1005853000),(off=0xbe,page=0x1081828000)
   ...(off=0xc6,page=0xf139b4000),(off=0xc7,page=0xb6b3bb000),(off=0xc8,page=0xad30c3000),(off=0xc9,page=0xd3a06c000),(off=0xca,page=0xf6a23d000),(off=0xcb,page=0xf60db8000)
   ...(off=0xcc,page=0x10e373a000),(off=0xcd,page=0xd532d0000),(off=0xce,page=0xef838b000),(off=0xcf,page=0x4128e000),(off=0xd0,page=0x6d1e44000),(off=0xd1,page=0x4f81cf000)
   ...(off=0xd5,page=0xbfdf9a000),(off=0xd6,page=0xfd9a27000),(off=0xd7,page=0xe02545000),(off=0xd8,page=0x10bbd0a000),(off=0xd9,page=0x96dd28000),(off=0xda,page=0x10d5a03000)
   ...(off=0xdb,page=0xa67832000),(off=0xdc,page=0xd76a4a000),(off=0xdd,page=0xd14c75000),(off=0xde,page=0xf0667f000),(off=0xdf,page=0xec0e23000),(off=0xe0,page=0xc737dc000)
   ...(off=0xed,page=0x6ab01a000),(off=0xee,page=0x54fe1a000),(off=0xef,page=0x1aaa54000),(off=0x102,page=0xebc09b000),(off=0x103,page=0x1037c83000),(off=0x104,page=0xf0af0d000)
   ...(off=0x105,page=0xd8ec88000),(off=0x106,page=0xf4f98e000),(off=0x107,page=0xe028f6000),(off=0x108,page=0x34b43d000),(off=0x109,page=0x7fd99c000),(off=0x10a,page=0x472297000)
   ...(off=0x10b,page=0x35bdc0000),(off=0x10c,page=0x57d63d000),(off=0x10d,page=0x3590f1000),(off=0x10e,page=0x133f79000),(off=0x10f,page=0x3efcc8000),(off=0x110,page=0xd7ca08000)
   ...(off=0x111,page=0xd312c9000),(off=0x112,page=0xc51661000),(off=0x113,page=0xceb8e1000),(off=0x114,page=0xb467ed000),(off=0x115,page=0xcee68b000),(off=0x119,page=0x11ac600000)
   ...(off=0x11a,page=0xf90ffe000),(off=0x11b,page=0xef18f4000),(off=0x11c,page=0x10672b3000),(off=0x11d,page=0xa18ba3000),(off=0x11e,page=0x10270c1000),(off=0x11f,page=0xfdf1f0000)
   ...(off=0x120,page=0x10396ce000),(off=0x121,page=0x1170325000),(off=0x122,page=0x11858ab000),(off=0x128,page=0x1155eae000),(off=0x129,page=0xed8f53000),(off=0x12a,page=0x1160722000)
   ...(off=0x12b,page=0x1007383000),(off=0x12c,page=0xc24241000),(off=0x12d,page=0x342b7e000),(off=0x12e,page=0xaea989000),(off=0x12f,page=0x39855e000),(off=0x130,page=0x2370d3000)
   ...(off=0x131,page=0x766d3a000),(off=0x132,page=0x4cd689000),(off=0x133,page=0x485f5b000),(off=0x134,page=0x23d98e000),(off=0x135,page=0x11c9bcb000),(off=0x136,page=0x11e9922000)
   ...(off=0x137,page=0xb31582000),(off=0x138,page=0xa21649000),(off=0x139,page=0xf40889000),(off=0x13a,page=0x11f4278000),(off=0x13b,page=0xf825ac000),(off=0x13c,page=0x6bc800000)
   ...(off=0x13d,page=0x3225e0000),(off=0x13e,page=0x9d755f000),(off=0x13f,page=0xb7893b000),(off=0x140,page=0xf4478d000),(off=0x141,page=0xff5735000),(off=0x142,page=0x112fa8f000)
   ...(off=0x143,page=0xb3282b000),(off=0x144,page=0xceb3c1000),(off=0x145,page=0xf9c67f000),(off=0x146,page=0x3d82a8000),(off=0x147,page=0xfa0b83000),(off=0x148,page=0xf7b81d000)
   ...(off=0x149,page=0x9d7487000),(off=0x14a,page=0xf2b1fd000),(off=0x14b,page=0x3d6f48000),(off=0x14c,page=0xd20e81000),(off=0x14d,page=0xdd73fd000),(off=0x14e,page=0xd1fc22000)
   ...(off=0x14f,page=0xcd48fe000),(off=0x150,page=0xc0a6e4000),(off=0x151,page=0xe6988b000),(off=0x152,page=0x992318000),(off=0x153,page=0xc6bd29000),(off=0x154,page=0x1084004000)
   ...(off=0x155,page=0xe8f816000),(off=0x156,page=0xea3299000),(off=0x157,page=0xb054d5000),(off=0x158,page=0xf339f8000),(off=0x159,page=0xa79f49000),(off=0x15a,page=0xc0c1ee000)
   ...(off=0x15b,page=0xecf83f000),(off=0x15c,page=0xb83486000),(off=0x15d,page=0xdf5e7c000),(off=0x15e,page=0xfa5133000),(off=0x15f,page=0x114611e000),(off=0x160,page=0xd9e1f9000)
   ...(off=0x161,page=0xf8be12000),(off=0x162,page=0x101e335000),(off=0x163,page=0xfd5f60000),(off=0x164,page=0x753d54000),(off=0x178,page=0x3624da000),(off=0x179,page=0x397c6d000)
   ...(off=0x17a,page=0x8edd93000),(off=0x17b,page=0x308c23000),(off=0x17c,page=0x80b9d1000),(off=0x17d,page=0x3a0b38000),(off=0x184,page=0x17432b000),(off=0x187,page=0xe5f42e000)
   ...(off=0x188,page=0xf59ade000),(off=0x189,page=0x1155c4f000),(off=0x18a,page=0xf205c7000),(off=0x18b,page=0x1129b3a000),(off=0x18c,page=0x9400f5000),(off=0x18d,page=0xd2941e000)
   ...(off=0x18e,page=0xf38561000),(off=0x18f,page=0xaf9c79000),(off=0x190,page=0x11604d4000),(off=0x191,page=0x104c26f000),(off=0x192,page=0xca74a5000),(off=0x193,page=0x45eb13000)
   ...(off=0x194,page=0xdfd834000),(off=0x195,page=0xaa9707000),(off=0x196,page=0xea1a29000),(off=0x197,page=0xc4b912000),(off=0x198,page=0x1115b08000),(off=0x199,page=0xb1ea62000)
   ...(off=0x19a,page=0x985be5000),(off=0x19b,page=0xc473b7000),(off=0x19c,page=0xbd8dc9000),(off=0x19d,page=0xb06e1f000),(off=0x19e,page=0xc7d2a0000),(off=0x19f,page=0xea7742000)
   ...(off=0x1a0,page=0xb9cbba000),(off=0x1a1,page=0x9997c5000),(off=0x1a2,page=0xdaf078000),(off=0x1a3,page=0xae7145000),(off=0x1a4,page=0x7a4f9b000),(off=0x1a5,page=0xaa5e18000)
   ...(off=0x1a6,page=0x1093f37000),(off=0x1a7,page=0xbf5d4c000),(off=0x1a8,page=0xda9a15000),(off=0x1a9,page=0xe9b4fe000),(off=0x1aa,page=0xf9a8ee000),(off=0x1ab,page=0xc44b86000)
   ...(off=0x1af,page=0xc41619000),(off=0x1b0,page=0x10372dd000),(off=0x1b1,page=0x9ed991000),(off=0x1b2,page=0xc5e2a8000),(off=0x1b3,page=0x104eea6000),(off=0x1b4,page=0xb42c46000)
   ...(off=0x1b5,page=0xb209c5000),(off=0x1b6,page=0xf8ad8a000),(off=0x1b7,page=0xa30db9000),(off=0x1b8,page=0xd5a6e7000),(off=0x1b9,page=0x104a9d5000),(off=0x1ba,page=0xf813e9000)
   ...(off=0x1c4,page=0x10674f1000),(off=0x1c5,page=0x115d9d0000),(off=0x1c6,page=0xa14737000),(off=0x1c7,page=0xa44a06000),(off=0x1c8,page=0xb4d9d9000),(off=0x1c9,page=0x102cef2000)
   ...(off=0x1e1,page=0x70bb14000),(off=0x1e3,page=0x90dac000),(off=0x1e4,page=0xc5f366000),(off=0x1f1,page=0x27f971000),(off=0x1f2,page=0x211b0b000),(off=0x1f3,page=0x4063ec000)
   ...(off=0x1f4,page=0xb1f749000),(off=0x1f5,page=0xa7895d000),(off=0x1f6,page=0xe3c4bb000),(off=0x1f7,page=0xb41714000),(off=0x26c,page=0xf06784000),(off=0x2c4,page=0xa2b6f7000)
   ...(off=0x2c5,page=0xcb13be000),(off=0x2c6,page=0x9c62d2000),(off=0x2d6,page=0xd5b431000),(off=0x2d7,page=0xcaa4fa000),(off=0x2d8,page=0xad75da000),(off=0x2f6,page=0xe07203000)
   ...(off=0x2f7,page=0xcc67d4000),(off=0x2f8,page=0xfe2119000),(off=0x312,page=0x374520000),(off=0x313,page=0x89f6ab000),(off=0x314,page=0x223ebf000),(off=0x315,page=0x147015000)
   ...(off=0x316,page=0x848651000),(off=0x317,page=0x83e8ad000),(off=0x3cd,page=0x36128b000),(off=0x3ce,page=0x353bda000),(off=0x3cf,page=0x24324c000),(off=0x422,page=0x10ce09a000)
   ...(off=0x423,page=0x7599ea000),(off=0x424,page=0x9b3e16000),(off=0x42f,page=0xf9aa86000),(off=0x430,page=0xdb77fc000),(off=0x431,page=0xfd8c0a000),(off=0x4da,page=0x106f1d8000)
   ...(off=0x4db,page=0x157e71000),(off=0x4dc,page=0x6d0917000),(off=0x4dd,page=0x9a7d5f000),(off=0x4de,page=0xa290b4000),(off=0x4df,page=0xbd9420000),(off=0x4e0,page=0xc1d34e000)
   ...(off=0x4e1,page=0xd4f12a000),(off=0x4e2,page=0xbd50cf000),(off=0x4e3,page=0x3743d1000),(off=0x4e4,page=0x1a3648000),(off=0x4e5,page=0x7fcc01000),(off=0x4e6,page=0x19b657000)
   ...(off=0x4e7,page=0x5b288b000),(off=0x4e8,page=0x117037000)

db> show procvm 3521
p = 0xfffff80757808a70, vmspace = 0xfffffe2622d6c9f0, map = 0xfffffe2622d6c9f0, pmap = 0xfffffe2622d6cb10
Task map 0xfffffe2622d6c9f0: pmap=0xfffffe2622d6cb10, nentries=29, version=15
  map entry 0xfffff801cec64ea0: start=0x8120ce000, end=0x8120d1000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80a0cdd6420, offset=0x0, obj ruid 0 charge 3000, copy (needed)
    Object 0xfffff80a0cdd6420: type=0, size=0x3, res=3, ref=2, flags=0x1010 ruid 0 charge 3000
     sref=0, backing_object(0)=(0)+0x2183000
  map entry 0xfffff8048a5d12a0: start=0x8120d1000, end=0x812141000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80dfc866528, offset=0x0, obj ruid 0 charge 70000, copy (needed)
    Object 0xfffff80dfc866528: type=1, size=0x70, res=78, ref=2, flags=0x1310 ruid 0 charge 70000
     sref=0, backing_object(3)=(0xfffff80208c26e70)+0x5e3000
  map entry 0xfffff805b875a180: start=0x812141000, end=0x812143000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80972b4c420, offset=0x0, obj ruid 0 charge 2000, copy (needed)
    Object 0xfffff80972b4c420: type=0, size=0x2, res=2, ref=2, flags=0x1010 ruid 0 charge 2000
     sref=0, backing_object(0)=(0)+0x0
  map entry 0xfffff801e5c098a0: start=0x812143000, end=0x8121dd000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80dfca47b58, offset=0x0, obj ruid 0 charge 9a000, copy (needed)
    Object 0xfffff80dfca47b58: type=0, size=0x9a, res=43, ref=2, flags=0x1110 ruid 0 charge 9a000
     sref=0, backing_object(2)=(0xfffff8058aebe108)+0x0
  map entry 0xfffff803dc6f8480: start=0x8121dd000, end=0x8121e0000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80dfa54bd68, offset=0x0, obj ruid 0 charge 3000, copy (needed)
    Object 0xfffff80dfa54bd68: type=0, size=0x3, res=3, ref=2, flags=0x1010 ruid 0 charge 3000
     sref=0, backing_object(0)=(0)+0x0
  map entry 0xfffff8011bb2c660: start=0x8121e0000, end=0x8121e2000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80c01965420, offset=0x0, obj ruid 0 charge 2000, copy (needed)
    Object 0xfffff80c01965420: type=0, size=0x2, res=2, ref=2, flags=0x1010 ruid 0 charge 2000
     sref=0, backing_object(0)=(0)+0x2295000
  map entry 0xfffff8011bb2c6c0: start=0x8121e2000, end=0x8121e3000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80e070ff528, offset=0x0, obj ruid 0 charge 1000, copy (needed)
    Object 0xfffff80e070ff528: type=0, size=0x1, res=1, ref=2, flags=0x1010 ruid 0 charge 1000
     sref=0, backing_object(0)=(0)+0x0
  map entry 0xfffff8030c063000: start=0x8121e3000, end=0x8121e4000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80e1a7e8420, offset=0x0, obj ruid 0 charge 1000, copy (needed)
    Object 0xfffff80e1a7e8420: type=0, size=0x1, res=1, ref=2, flags=0x1010 ruid 0 charge 1000
     sref=0, backing_object(0)=(0)+0x2298000
  map entry 0xfffff808e74305a0: start=0x8121e4000, end=0x8121e6000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80b18210a50, offset=0x0, obj ruid 0 charge 2000, copy (needed)
    Object 0xfffff80b18210a50: type=0, size=0x2, res=2, ref=2, flags=0x1010 ruid 0 charge 2000
     sref=0, backing_object(0)=(0)+0x0
  map entry 0xfffff808b4f4d000: start=0x8121e6000, end=0x812415000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80e1fed9b58, offset=0x0, obj ruid 0 charge 22f000, copy (needed)
    Object 0xfffff80e1fed9b58: type=1, size=0x22f, res=135, ref=2, flags=0x1310 ruid 0 charge 22f000
     sref=0, backing_object(2)=(0xfffff8058aebe108)+0xa3000
  map entry 0xfffff804eabc11e0: start=0x812415000, end=0x812418000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff8021d525a50, offset=0x0, obj ruid 0 charge 3000, copy (needed)
    Object 0xfffff8021d525a50: type=1, size=0x3, res=0, ref=2, flags=0x1210 ruid 0 charge 3000
     sref=0, backing_object(0)=(0)+0x24ca000
  map entry 0xfffff80637e9da80: start=0x812418000, end=0x81241b000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff803fb59f630, offset=0x0, obj ruid 0 charge 3000, copy (needed)
    Object 0xfffff803fb59f630: type=1, size=0x3, res=2, ref=2, flags=0x1210 ruid 0 charge 3000
     sref=0, backing_object(0)=(0)+0x0
  map entry 0xfffff8015133bf60: start=0x81241b000, end=0x81241d000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80eb0fbbe70, offset=0x0, obj ruid 0 charge 2000, copy (needed)
    Object 0xfffff80eb0fbbe70: type=0, size=0x2, res=2, ref=2, flags=0x1010 ruid 0 charge 2000
     sref=0, backing_object(0)=(0)+0x0
  map entry 0xfffff802ebe34120: start=0x81241d000, end=0x81248b000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff809d65adc60, offset=0x0, obj ruid 0 charge 6e000, copy (needed)
    Object 0xfffff809d65adc60: type=1, size=0x6e, res=68, ref=2, flags=0x1310 ruid 0 charge 6e000
     sref=0, backing_object(2)=(0xfffff80df4569d68)+0x0
  map entry 0xfffff802f129e600: start=0x81248b000, end=0x81248e000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80d8cc90108, offset=0x0, obj ruid 0 charge 3000, copy (needed)
    Object 0xfffff80d8cc90108: type=0, size=0x3, res=3, ref=2, flags=0x1010 ruid 0 charge 3000
     sref=0, backing_object(0)=(0)+0x6e000
  map entry 0xfffff8046dbe5660: start=0x81248e000, end=0x812490000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80d83677000, offset=0x0, obj ruid 0 charge 2000, copy (needed)
    Object 0xfffff80d83677000: type=0, size=0x2, res=2, ref=2, flags=0x1010 ruid 0 charge 2000
     sref=0, backing_object(0)=(0)+0x71000
  map entry 0xfffff800b75a5660: start=0x812490000, end=0x812493000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80e1f69cc60, offset=0x0, obj ruid 0 charge 3000, copy (needed)
    Object 0xfffff80e1f69cc60: type=0, size=0x3, res=3, ref=2, flags=0x1010 ruid 0 charge 3000
     sref=0, backing_object(0)=(0)+0x73000
  map entry 0xfffff808fe0b7060: start=0x812493000, end=0x8124b6000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff805d2aeab58, offset=0x0, obj ruid 0 charge 23000, copy (needed)
    Object 0xfffff805d2aeab58: type=1, size=0x23, res=33, ref=2, flags=0x1210 ruid 0 charge 23000
     sref=0, backing_object(0)=(0)+0x76000
  map entry 0xfffff806181d6c60: start=0x8124b6000, end=0x8124b9000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80af208a108, offset=0x0, obj ruid 0 charge 3000, copy (needed)
    Object 0xfffff80af208a108: type=0, size=0x3, res=3, ref=2, flags=0x1010 ruid 0 charge 3000
     sref=0, backing_object(0)=(0)+0x99000
  map entry 0xfffff803d79b9120: start=0x8124b9000, end=0x812515000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80e366b6738, offset=0x0, obj ruid 0 charge 5c000, copy (needed)
    Object 0xfffff80e366b6738: type=1, size=0x5c, res=31, ref=2, flags=0x1310 ruid 0 charge 5c000
     sref=0, backing_object(2)=(0xfffff80f95af0210)+0x0
  map entry 0xfffff80014331c00: start=0x812515000, end=0x812517000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80e2b70f738, offset=0x0, obj ruid 0 charge 2000, copy (needed)
    Object 0xfffff80e2b70f738: type=0, size=0x2, res=2, ref=2, flags=0x1010 ruid 0 charge 2000
     sref=0, backing_object(0)=(0)+0xf8000
  map entry 0xfffff802d4406ea0: start=0x812517000, end=0x812523000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80e367ba528, offset=0x0, obj ruid 0 charge c000, copy (needed)
    Object 0xfffff80e367ba528: type=0, size=0xc, res=7, ref=2, flags=0x1110 ruid 0 charge c000
     sref=0, backing_object(2)=(0xfffff80f95af0210)+0x5e000
  map entry 0xfffff803fc486ea0: start=0x812523000, end=0x812526000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80e1aa25420, offset=0x0, obj ruid 0 charge 3000, copy (needed)
    Object 0xfffff80e1aa25420: type=0, size=0x3, res=3, ref=2, flags=0x1010 ruid 0 charge 3000
     sref=0, backing_object(0)=(0)+0x25d8000
  map entry 0xfffff802453256c0: start=0x812526000, end=0x812d00000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff8048db7d528, offset=0x0, obj ruid 0 charge 7da000, copy (needed)
    Object 0xfffff8048db7d528: type=1, size=0x7da, res=1946, ref=2, flags=0x1310 ruid 0 charge 7da000
     sref=1, backing_object(3)=(0xfffff80208c26e70)+0xa38000
  map entry 0xfffff80262ce2540: start=0x812d00000, end=0x816500000, eflags=0xc,
   prot=3/7/copy, ruid 0, object=0xfffff80e2c724420, offset=0x0, obj ruid 0 charge 3800000, copy (needed)
    Object 0xfffff80e2c724420: type=1, size=0x3800, res=1284, ref=2, flags=0x1210 ruid 0 charge 3800000
     sref=0, backing_object(0)=(0)+0x0
  map entry 0xfffff801853422a0: start=0x7fffdfffe000, end=0x7fffdffff000, eflags=0,
   prot=0/7/copy, object=0, offset=0x0
  map entry 0xfffff80871770ae0: start=0x7fffdffff000, end=0x7ffffffdf000, eflags=0x30000,
   prot=0/0/copy, object=0, offset=0x0
  map entry 0xfffff806333c6540: start=0x7ffffffdf000, end=0x7ffffffff000, eflags=0x1004,
   prot=3/7/copy, object=0xfffff80b5e771000, offset=0x0, obj ruid 0 charge 20000, copy (done)
    Object 0xfffff80b5e771000: type=0, size=0x20, res=1, ref=1, flags=0x3110 ruid 0 charge 20000
     sref=0, backing_object(2)=(0xfffff8056d778420)+0x0
  map entry 0xfffff8063066eae0: start=0x7ffffffff000, end=0x800000000000, eflags=0,
   prot=5/5/share, object=0xfffff8094c389e70, offset=0x0
    Object 0xfffff8094c389e70: type=4, size=0x1, res=1, ref=191, flags=0x6 ruid -1 charge 0
     sref=0, backing_object(0)=(0)+0x0
Comment 1 Mark Johnston freebsd_committer freebsd_triage 2021-10-04 18:09:23 UTC 
If the system is still in DDB, can you show output from the following:

db> x/gx 0xfffffe006b2d0510

This should print the address of the first element of the page's pv list, should be an address of the form 0xffff8.*.  Then show the dword at that address:

db> x/gx 0xffff8...

Then do

db> show pte <addr printed by the last command>
Comment 2 Bryan Drewery freebsd_committer freebsd_triage 2021-10-04 18:13:23 UTC 
(In reply to Mark Johnston from comment #1)

> If the system is still in DDB, can you show output from the following:
> 
> db> x/gx 0xfffffe006b2d0510


db> x/gx 0xfffffe006b2d0510
0xfffffe006b2d0510:     fffff80b88576460

> This should print the address of the first element of the page's pv list, should > be an address of the form 0xffff8.*.  Then show the dword at that address:
> 
> db> x/gx 0xffff8...
>

db> x/gx 0xfffff80b88576460
0xfffff80b88576460:     811be5000

> Then do
>
> db> show pte <addr printed by the last command>


db> show pte 811be5000
VA 0x0000000811be5000 pml4e 0x0000000000000000
Comment 3 Mark Johnston freebsd_committer freebsd_triage 2021-10-04 19:01:38 UTC 
It looks like the PV entry indeed belongs to a different pmap:

db> x/xg,8 0xfffff80b88576000                                              
0xfffff80b88576000:     8e741120        fffffe23        9bd10000        fffff801
0xfffff80b88576010:     e7ac008         fffff810        c0000006        7ffc011

db> show procvm                                                          
p = 0xfffff80757808a70, vmspace = 0xfffffe2622d6c9f0, map = 0xfffffe2622d6c9f0, pmap = 0xfffffe2622d6cb10

But it's hard to tell from ddb which process owns that pmap.  I think a kernel dump is needed to diagnose further, though then we will lose the user PTPs.
Comment 4 Bryan Drewery freebsd_committer freebsd_triage 2021-10-04 20:12:39 UTC 
(In reply to Mark Johnston from comment #3)

> But it's hard to tell from ddb which process owns that pmap.  I think a kernel dump is needed to diagnose further, though then we will lose the user PTPs.

Is there a feasible script we could write to determine that? If we procvm every proc and do some parsing... I'd hate to lose any relevant information here. I can do the data gathering and parsing if you think it would be useful before reboot/dumping.
Comment 5 Mark Johnston freebsd_committer freebsd_triage 2021-10-04 22:53:39 UTC 
(In reply to Bryan Drewery from comment #4)
If we can run "show proc <pid>" for each pid shown in "show all procs" output, then that'd be good enough.
Comment 6 Bryan Drewery freebsd_committer freebsd_triage 2021-10-04 23:29:53 UTC 
(In reply to Mark Johnston from comment #5)

Great. I have a script running capturing all of that now.
Comment 7 Bryan Drewery freebsd_committer freebsd_triage 2021-10-05 02:35:31 UTC 
(In reply to Mark Johnston from comment #5)
https://people.freebsd.org/~bdrewery/allproc.txt
Comment 8 Mark Johnston freebsd_committer freebsd_triage 2021-10-05 15:12:08 UTC 
(In reply to Bryan Drewery from comment #7)
Thanks.  So it appears that the referencing pmap belongs to the parent process, another tmux process:

db> show proc 1634
Process 1634 (tmux) at 0xfffff809dd32f000:
 state: NORMAL
 uid: 0  gids: 0, 0, 5, 65531, 65532
 parent: pid 1 at 0xfffff801140cf538
 ABI: FreeBSD ELF64
 flag: 0x10000000  flag2: 0
 arguments: tmux: server (/tmp/tmux-0/default)
 reaper: 0xfffff801140cf538 reapsubtree: 1
 sigparent: 20
 vmspace: 0xfffffe238e741000
   (map 0xfffffe238e741000)
   (map.pmap 0xfffffe238e7410c0)
   (pmap 0xfffffe238e741120)
 threads: 1
101736                   S       select  0xfffff80a3a993740  tmux

Switching to that thread,

db> show pte 0x811be5000                                                                                                                                                                                                                                                                                                      
VA 0x0000000811be5000 pml4e 0x0000000338dad067 pdpe 0x0000000876895067 pde 0x000000077bde3067 pte 0x800000107d157425

So the PTE is indeed valid and the paddr matches that of the page in question.

Proc 1634 has a very very fragmented vm_map for some reason, tons of small map entries.  I need to let "show procvm" run for a while to find the map entry corresponding to 0x811be5000.
Comment 9 Mark Johnston freebsd_committer freebsd_triage 2021-10-05 17:15:31 UTC 
procvm entry:

    map entry 0xfffff80ae7805360: start=0x811be5000, end=0x8120ce000, eflags=0x4,        
     prot=3/7/copy, object=0xfffff805d279b630, offset=0x0, obj ruid 0 charge 4e9000, copy (done)
      Object 0xfffff805d279b630: type=0, size=0x4e9, res=2, ref=1, flags=0x3110 ruid 0 charge 4e9000
       sref=0, backing_object(2)=(0xfffff802509ca210)+0x0

So this map entry shadows the object to which the page belongs.  Both this object and its backing object (the one containing page 0xfffffe006b2d04d8) have ONEMAPPING set.  The shadow chain seems very deep, about 12 VM objects before we reach the root (an anonymous swap object).
Comment 10 Bryan Drewery freebsd_committer freebsd_triage 2021-10-07 02:45:38 UTC 
The build on this system corresponds to https://cgit.freebsd.org/src/commit/?id=c7cf100aafb4cb881e05a5153de152907f6c07f3
Comment 11 Bryan Drewery freebsd_committer freebsd_triage 2022-05-04 17:38:58 UTC 
Just hit it again on a recent kernel from main April 22nd.

panic: vm_page_free_prep: freeing mapped page 0xfffffe0020bccfe8
cpuid = 4
time = 1651629479
KDB: stack backtrace:
db_trace_self_wrapper() at db_trace_self_wrapper+0x2b/frame 0xfffffe01d929c660
vpanic() at vpanic+0x17f/frame 0xfffffe01d929c6b0
panic() at panic+0x43/frame 0xfffffe01d929c710
vm_page_free_prep() at vm_page_free_prep+0x20c/frame 0xfffffe01d929c730
vm_page_free_toq() at vm_page_free_toq+0x12/frame 0xfffffe01d929c760
vm_object_page_remove() at vm_object_page_remove+0xb1/frame 0xfffffe01d929c7c0
vm_map_entry_delete() at vm_map_entry_delete+0x110/frame 0xfffffe01d929c810
vm_map_delete() at vm_map_delete+0xc8/frame 0xfffffe01d929c880
vm_map_remove() at vm_map_remove+0x81/frame 0xfffffe01d929c8b0
exec_new_vmspace() at exec_new_vmspace+0x211/frame 0xfffffe01d929c900
exec_elf64_imgact() at exec_elf64_imgact+0xb59/frame 0xfffffe01d929ca00
kern_execve() at kern_execve+0x76d/frame 0xfffffe01d929cd80
sys_execve() at sys_execve+0x5a/frame 0xfffffe01d929ce00
amd64_syscall() at amd64_syscall+0x6a6/frame 0xfffffe01d929cf30
fast_syscall_common() at fast_syscall_common+0xf8/frame 0xfffffe01d929cf30
--- syscall (59, FreeBSD ELF64, sys_execve), rip = 0x821e0d36a, rsp = 0x82072dd08, rbp = 0x82072de10 ---
KDB: enter: panic
[ thread pid 24172 tid 101498 ]
Stopped at      kdb_enter+0x32: movq    $0,0x127ce23(%rip)
Comment 12 Bryan Drewery freebsd_committer freebsd_triage 2022-05-04 18:15:50 UTC 
tmux (child) again. tmux -> zsh -> screen -> poudriere.
Comment 13 Bryan Drewery freebsd_committer freebsd_triage 2022-05-04 18:25:38 UTC 
Well init -> tmux(panic) -> zsh -> lockf -> time -> screen -> screen -> sh (/root/build) -> [ttyin]

The open file for sh is also open in vim.

This script runs Poudriere. I suspect it is at the end of the build where I have a 'read n' to pause before exiting. So the system is pretty idle.
Comment 14 Bryan Drewery freebsd_committer freebsd_triage 2022-05-04 18:35:16 UTC 
Actually all of that build stuff is a child of the parent tmux process. The current tmux process is a child that I believe is execing some powerline/status update handling.

I'm just thinking out what the repro may be.
Comment 15 Bryan Drewery freebsd_committer freebsd_triage 2022-05-04 19:11:28 UTC 
https://people.freebsd.org/~pho/stress/log/log0110.txt may be the same thing from misc/cmp.sh stress2 test.
Comment 16 Mark Johnston freebsd_committer freebsd_triage 2022-06-12 17:48:24 UTC 
After staring at a vmcore from bdrewery for a long time, I think I see the bug.  It's a race which seems very hard to hit accidentally.

When the kernel crashed, we were removing and freeing pages from a VM object in preparation for execve().  The OBJ_ONEMAPPING flag is set on the object, which means only the current process has mapped the object, which why it's (supposed to be) safe to free the object's pages.  The problem is that there are in fact two mappings of the object: the other is in the parent tmux process, a COW mapping.  So OBJ_ONEMAPPING should absolutely not be set!

When vmspace_fork() copies vm map entries from the parent into the child, it's supposed to clear OBJ_ONEMAPPING.  For anonymous mappings, this happens in

  vmspace_fork() -> vm_map_copy_entry() -> vm_map_copy_swap_object()

vm_map_copy_entry() uses the following test to determine whether it's dealing with a swap object:

  if (src_object->type == OBJT_DEFAULT || (src_object->flags & OBJ_SWAP) != 0)

This test is done racily, i.e., without the object lock, which isn't quite a problem in itself, but with commit 4b8365d752ef4 it can produce false negatives.  In particular, the first time a page in the object gets paged out, swap_pager_put_pages() converts the object to a OBJT_SWAP object:

  object->type = OBJT_SWAP;
  vm_object_set_flag(object, OBJ_SWAP);

So there's a small window where object->type == OBJT_SWAP and OBJ_SWAP is clear.
Comment 17 Mark Johnston freebsd_committer freebsd_triage 2022-06-13 15:50:27 UTC 
https://reviews.freebsd.org/D35470
Comment 18 Mark Johnston freebsd_committer freebsd_triage 2022-06-13 15:51:09 UTC 
IMO this deserves an EN.
Comment 19 commit-hook freebsd_committer freebsd_triage 2022-06-20 17:01:34 UTC 
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=e123264e4dc394602f9fed2f0376204b5998d815

commit e123264e4dc394602f9fed2f0376204b5998d815
Author:     Mark Johnston <markj@FreeBSD.org>
AuthorDate: 2022-06-20 16:18:15 +0000
Commit:     Mark Johnston <markj@FreeBSD.org>
CommitDate: 2022-06-20 16:48:14 +0000

    vm: Fix racy checks for swap objects

    Commit 4b8365d752ef introduced the ability to dynamically register
    VM object types, for use by tmpfs, which creates swap-backed objects.
    As a part of this, checks for such objects changed from

      object->type == OBJT_DEFAULT || object->type == OBJT_SWAP

    to

      object->type == OBJT_DEFAULT || (object->flags & OBJ_SWAP) != 0

    In particular, objects of type OBJT_DEFAULT do not have OBJ_SWAP set;
    the swap pager sets this flag when converting from OBJT_DEFAULT to
    OBJT_SWAP.

    A few of these checks are done without the object lock held.  It turns
    out that this can result in false negatives since the swap pager
    converts objects like so:

      object->type = OBJT_SWAP;
      object->flags |= OBJ_SWAP;

    Fix the problem by adding explicit tests for OBJT_SWAP objects in
    unlocked checks.

    PR:             258932
    Fixes:          4b8365d752ef ("Add OBJT_SWAP_TMPFS pager")
    Reported by:    bdrewery
    Reviewed by:    kib
    MFC after:      2 weeks
    Sponsored by:   The FreeBSD Foundation
    Differential Revision:  https://reviews.freebsd.org/D35470

 sys/vm/vm_map.c     | 9 ++++++---
 sys/vm/vm_mmap.c    | 5 ++---
 sys/vm/vm_pageout.c | 5 +++--
 3 files changed, 11 insertions(+), 8 deletions(-)
Comment 20 Bryan Drewery freebsd_committer freebsd_triage 2022-06-22 14:53:53 UTC 
(In reply to Mark Johnston from comment #16)

Great job Mark! Thanks for the perseverance.
Comment 21 commit-hook freebsd_committer freebsd_triage 2022-07-04 13:07:38 UTC 
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=de0b1239dfa49011f76369551838d31c00e5daa1

commit de0b1239dfa49011f76369551838d31c00e5daa1
Author:     Mark Johnston <markj@FreeBSD.org>
AuthorDate: 2022-06-20 16:18:15 +0000
Commit:     Mark Johnston <markj@FreeBSD.org>
CommitDate: 2022-07-04 13:06:55 +0000

    vm: Fix racy checks for swap objects

    Commit 4b8365d752ef introduced the ability to dynamically register
    VM object types, for use by tmpfs, which creates swap-backed objects.
    As a part of this, checks for such objects changed from

      object->type == OBJT_DEFAULT || object->type == OBJT_SWAP

    to

      object->type == OBJT_DEFAULT || (object->flags & OBJ_SWAP) != 0

    In particular, objects of type OBJT_DEFAULT do not have OBJ_SWAP set;
    the swap pager sets this flag when converting from OBJT_DEFAULT to
    OBJT_SWAP.

    A few of these checks are done without the object lock held.  It turns
    out that this can result in false negatives since the swap pager
    converts objects like so:

      object->type = OBJT_SWAP;
      object->flags |= OBJ_SWAP;

    Fix the problem by adding explicit tests for OBJT_SWAP objects in
    unlocked checks.

    PR:             258932
    Fixes:          4b8365d752ef ("Add OBJT_SWAP_TMPFS pager")
    Reported by:    bdrewery
    Reviewed by:    kib
    Sponsored by:   The FreeBSD Foundation

    (cherry picked from commit e123264e4dc394602f9fed2f0376204b5998d815)

 sys/vm/vm_map.c     | 9 ++++++---
 sys/vm/vm_mmap.c    | 5 ++---
 sys/vm/vm_pageout.c | 5 +++--
 3 files changed, 11 insertions(+), 8 deletions(-)