Bug 255041 - panic: system freeze/panic with D29125 patch
Summary: panic: system freeze/panic with D29125 patch
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Only Me
Assignee: freebsd-bugs (Nobody)
URL: https://reviews.freebsd.org/D29772
Keywords:
Depends on:
Blocks:
 
Reported: 2021-04-14 03:49 UTC by Masachika ISHIZUKA
Modified: 2021-04-28 15:22 UTC (History)
6 users (show)

See Also:


Attachments
core.txt.2.xz (29.36 KB, application/octet-stream)
2021-04-14 16:13 UTC, Masachika ISHIZUKA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Masachika ISHIZUKA 2021-04-14 03:49:19 UTC
I had trouble with recent 14.0-CURRENT.
It works well on recent 14.0-CURRENT until starting firefox.
If I start firefox (v87.0), system freeze or reboot but no core dumps.
If it rollbacked to e80e371d79d825712a9078e0d1a14b62567291c4, firefox works fine.
After or equatl to 1ae20f7c70ea16fa8b702e409030e170df4f5c13, this panic occured.
I reverted 1ae20f7c70ea16fa8b702e409030e170df4f5c13 to 8cca7b7f28feaf0c5e2dfedb985ae334a4013ef6, it works fine.
Please revert 1ae20f7c70ea16fa8b702e409030e170df4f5c13 on main stream.

My machines are dell XPS 12 (9Q33) notebook (core i7-4500U) and dell Vostro 3267 desktop (core i5-7500).
Comment 1 Mark Millard 2021-04-14 11:26:47 UTC
Looks like 1ae20f7c70e is a detection of inappropriate use
of M_WAITOK in a context where it is not allowed because
it might sleep when sleeping is not allowed:

diff --git a/sys/kern/kern_malloc.c b/sys/kern/kern_malloc.c
index 48383358e3ad..0d6f9dcfcab7 100644
--- a/sys/kern/kern_malloc.c
+++ b/sys/kern/kern_malloc.c
@@ -537,7 +537,7 @@ malloc_dbg(caddr_t *vap, size_t *sizep, struct malloc_type *mtp,
 #ifdef EPOCH_TRACE
 			epoch_trace_list(curthread);
 #endif
-			KASSERT(1, 
+			KASSERT(0,
 			    ("malloc(M_WAITOK) with sleeping prohibited"));
 		}
 	}

Letting the inappropriate activity happen by reverting the change
does not look reasonable from what I can tell. (Not that I'm
expert, however.)

More likely the context needs to be analyzed for what allocation
is happening in what kind of restricted context and then, with
that information, a fix that avoids the bad type of activity
would be developed. It is not obvious what all code would need
to change at this point.
Comment 2 Li-Wen Hsu freebsd_committer 2021-04-14 13:53:32 UTC
CC author of https://reviews.freebsd.org/D29125
Comment 3 Kyle Evans freebsd_committer 2021-04-14 13:58:12 UTC
(In reply to Masachika ISHIZUKA from comment #0)

Hi,

Can you please provide some information about the panic you received? We'll need at least a backtrace to have even some remote idea of what's happening.
Comment 4 Masachika ISHIZUKA 2021-04-14 16:13:55 UTC
Created attachment 224109 [details]
core.txt.2.xz

crash dump
Comment 5 Kyle Evans freebsd_committer 2021-04-14 16:21:14 UTC
(In reply to Masachika ISHIZUKA from comment #4)

CC'ing Andrey, looks like b99a6823204f0ecbefef21af585dde03c4d76ba1 ("Rework ipfw dynamic states implementation to be lockless on fast path.") started doing M_WAITOK allocations in a callout (dyn_tick -> dyn_grow_hashtable), which is a non-sleepable context.
Comment 6 Masachika ISHIZUKA 2021-04-14 16:23:31 UTC
(In reply to Masachika ISHIZUKA from comment #4)
# cat info.2
Dump header from device: /dev/gpt/fbswap
  Architecture: amd64
  Architecture Version: 2
  Dump Length: 525697024
  Blocksize: 512
  Compression: none
  Dumptime: 2021-04-15 00:44:38 +0900
  Hostname: carrot.ish.org
  Magic: FreeBSD Kernel Dump
  Version String: FreeBSD 14.0-CURRENT #28 0c80ad2dc: Thu Apr 15 00:32:41 JST 2021
    ishizuka@carrot.ish.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC
  Panic String: malloc(M_WAITOK) with sleeping prohibited
  Dump Parity: 4198134333
  Bounds: 2
  Dump Status: good
carrot# cat /var/crash/info.2
Dump header from device: /dev/gpt/fbswap
  Architecture: amd64
  Architecture Version: 2
  Dump Length: 525697024
  Blocksize: 512
  Compression: none
  Dumptime: 2021-04-15 00:44:38 +0900
  Hostname: carrot.ish.org
  Magic: FreeBSD Kernel Dump
  Version String: FreeBSD 14.0-CURRENT #28 0c80ad2dc: Thu Apr 15 00:32:41 JST 2021
    ishizuka@carrot.ish.org:/usr/obj/usr/src/amd64.amd64/sys/GENERIC
  Panic String: malloc(M_WAITOK) with sleeping prohibited
  Dump Parity: 4198134333
  Bounds: 2
  Dump Status: good
#

vmcore is too large, so it put to https://www.ish.org/files/vmcore.2.xz
Comment 7 Masachika ISHIZUKA 2021-04-14 16:26:16 UTC
(In reply to Masachika ISHIZUKA from comment #6)
> vmcore is too large, so it put to https://www.ish.org/files/vmcore.2.xz

Sorry, 
https://www.ish.org/files/vmcore.2-0c80ad2dc.xz
Comment 8 Andrey V. Elsukov freebsd_committer 2021-04-14 16:33:43 UTC
(In reply to Kyle Evans from comment #5)

I think the problem was here in earlier implementation too, I'll fix this next tomorrow. Thanks for the report.
Comment 9 Kyle Evans freebsd_committer 2021-04-14 16:44:47 UTC
(In reply to Andrey V. Elsukov from comment #8)

Ah, sorry, I did not look much further than `git blame` and the message. Thanks!
Comment 10 commit-hook freebsd_committer 2021-04-16 07:25:15 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=9bacbf1ae243de43d642c3ac0b7318ae0e5c5235

commit 9bacbf1ae243de43d642c3ac0b7318ae0e5c5235
Author:     Andrey V. Elsukov <ae@FreeBSD.org>
AuthorDate: 2021-04-16 07:22:44 +0000
Commit:     Andrey V. Elsukov <ae@FreeBSD.org>
CommitDate: 2021-04-16 07:22:44 +0000

    ipfw: do not use sleepable malloc in callout context.

    Use M_NOWAIT flag when hash growing is called from callout.

    PR:             255041
    Reviewed by:    kevans
    MFC after:      10 days
    Differential Revision: https://reviews.freebsd.org/D29772

 sys/netpfil/ipfw/ip_fw_dynamic.c | 68 ++++++++++++++++++++++++++++++----------
 1 file changed, 51 insertions(+), 17 deletions(-)
Comment 11 commit-hook freebsd_committer 2021-04-28 10:07:00 UTC
A commit in branch stable/13 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=29246f2cb0c5c5c06c73e99dc0d6c947f62c7dc4

commit 29246f2cb0c5c5c06c73e99dc0d6c947f62c7dc4
Author:     Andrey V. Elsukov <ae@FreeBSD.org>
AuthorDate: 2021-04-16 07:22:44 +0000
Commit:     Andrey V. Elsukov <ae@FreeBSD.org>
CommitDate: 2021-04-28 10:05:03 +0000

    ipfw: do not use sleepable malloc in callout context.

    Use M_NOWAIT flag when hash growing is called from callout.

    PR:             255041
    Reviewed by:    kevans
    Differential Revision: https://reviews.freebsd.org/D29772

    (cherry picked from commit 9bacbf1ae243de43d642c3ac0b7318ae0e5c5235)

 sys/netpfil/ipfw/ip_fw_dynamic.c | 68 ++++++++++++++++++++++++++++++----------
 1 file changed, 51 insertions(+), 17 deletions(-)
Comment 12 commit-hook freebsd_committer 2021-04-28 10:08:02 UTC
A commit in branch stable/12 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=5ebfa80f7543d5cda828ee15ed20c9fc03f99208

commit 5ebfa80f7543d5cda828ee15ed20c9fc03f99208
Author:     Andrey V. Elsukov <ae@FreeBSD.org>
AuthorDate: 2021-04-16 07:22:44 +0000
Commit:     Andrey V. Elsukov <ae@FreeBSD.org>
CommitDate: 2021-04-28 10:06:33 +0000

    ipfw: do not use sleepable malloc in callout context.

    Use M_NOWAIT flag when hash growing is called from callout.

    PR:             255041
    Reviewed by:    kevans
    Differential Revision: https://reviews.freebsd.org/D29772

    (cherry picked from commit 9bacbf1ae243de43d642c3ac0b7318ae0e5c5235)

 sys/netpfil/ipfw/ip_fw_dynamic.c | 68 ++++++++++++++++++++++++++++++----------
 1 file changed, 51 insertions(+), 17 deletions(-)
Comment 13 Andrey V. Elsukov freebsd_committer 2021-04-28 15:22:24 UTC
Fixed in head/, stable/13 and stable/12. Thanks!