Bug 292495 - Kernel panic in 15.0 when using inotify from podman with ZFS encrypted datasets
Summary: Kernel panic in 15.0 when using inotify from podman with ZFS encrypted datasets
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: kern (show other bugs)
Version: 15.0-CURRENT
Hardware: Any Any
: --- Affects Some People
Assignee: Mark Johnston
URL:
Keywords: crash, regression
Depends on:
Blocks:
 
Reported: 2026-01-16 06:14 UTC by Jed Laundry
Modified: 2026-05-08 13:40 UTC (History)
2 users (show)

See Also:


Attachments
Core file (202.04 KB, text/plain)
2026-01-16 06:14 UTC, Jed Laundry
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jed Laundry 2026-01-16 06:14:28 UTC
Created attachment 267185 [details]
Core file

(This is my first FreeBSD bug report, apologies if I've missed a step)

After upgrading from 14.3 to 15.0, I found that trying to use my Visual Studio Code podman containers resulted in a kernel panic. I was able to reproduce this issue in a fresh 15.0 install.

It seems to be something to do with the new inotify implementation, and only seems to happen when mapping an encrypted ZFS dataset into the container, where not all files are owned by root:wheel.

I've attached the core.txt.6 file, and I can provide the full vmcore.6 if required.

Steps to reproduce:

1. Install FreeBSD 15 with ZFS root
2. Install and configure podman:

```bash
pkg install podman

echo "fdesc   /dev/fd         fdescfs         rw      0       0" >> /etc/fstab

default_if=$(netstat -rn | awk '/^default/ {print $NF}' | tail -n1)
cat /usr/local/etc/containers/pf.conf.sample | sed "s/if = \"ix0\"/if = \"$default_if\"/g" > /etc/pf.conf

service linux enable
service podman enable
service pf enable
reboot
```

3. Create an encrypted home:

```bash
zfs destroy zroot/home/jlaundry
zfs create -o encryption=on -o keyformat=passphrase zroot/home/jlaundry
```

4. Create a Linux container to run vscode - I use Tunnels as the FreeBSD VM is headless, but I don’t think that’s strictly necessary:

```bash
podman pull --os=linux docker.io/library/ubuntu:24.04

mkdir vscode
cat << 'EOF' > vscode/Dockerfile
FROM ubuntu:24.04

# fix for error "E: Dynamic MMap ran out of room. Please increase the size of APT::Cache-Start. Current value: 25165824. (man 5 apt.conf)"
RUN echo 'APT::Cache-Start "100000000";' >> /etc/apt/apt.conf.d/70debconf

RUN apt update && \
    apt install curl git build-essential python3 python3-venv -y && \
    apt clean all && \
    curl -sL "https://code.visualstudio.com/sha/download?build=stable&os=cli-alpine-x64" --output /tmp/vscode-cli.tar.gz && \
    tar -xf /tmp/vscode-cli.tar.gz -C /usr/bin && \
    rm /tmp/vscode-cli.tar.gz && \
    mkdir /src

VOLUME /src
VOLUME /root

CMD [ "code", "tunnel", "--accept-server-license-terms", "--no-sleep" ]
EOF

podman build --os=linux -t localhost/vscode ./vscode

mkdir /root/testcode

podman run --replace -v /home/jlaundry/src:/src:rw -v /root/testcode:/root:rw --os=linux --name f15test -h f15test vscode
```

5. In my case I followed the steps to authenticate the Tunnel, and then, inside the container:

```bash
# Any repo will do, as long as you can push to it
git clone https://github.com/jlaundry/test.git
```

6. Stop the container, and change the ownership of the mapped encrypted volume:

```bash
chmod -R jlaundry:jlaundry /home/jlaundry/src
```

7. Start the container again, then from vscode make a change to a file, commit, and try to push - the crash happens immediately on push.
Comment 1 Mark Johnston freebsd_committer freebsd_triage 2026-04-25 21:35:08 UTC
https://reviews.freebsd.org/D56639
Comment 2 commit-hook freebsd_committer freebsd_triage 2026-04-26 01:56:41 UTC
A commit in branch main references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=a02d794f5acd12ba3cf1de5c204a8dd56af47edd

commit a02d794f5acd12ba3cf1de5c204a8dd56af47edd
Author:     Mark Johnston <markj@FreeBSD.org>
AuthorDate: 2026-04-26 01:35:37 +0000
Commit:     Mark Johnston <markj@FreeBSD.org>
CommitDate: 2026-04-26 01:56:14 +0000

    nullfs: Clear inotify flags during reclaim

    The inotify flags are copied from the lower vnode into the nullfs vnode
    so that the INOTIFY() macro will invoke VOP_INOTIFY on the nullfs vnode;
    this is then bypassed to the lower vnode.  However, when a nullfs vnode
    is reclaimed we should clear these flags, as the vnode is now doomed and
    no longer forwards VOPs to the lower vnode.

    Add regression tests.  Remove a test in vn_inotify_revoke() which is no
    longer needed after this change.

    PR:             292495
    Reviewed by:    kib
    Reported by:    Jed Laundry <jlaundry@jlaundry.com>
    Fixes:          f1f230439fa4 ("vfs: Initial revision of inotify")
    MFC after:      1 week
    Differential Revision:  https://reviews.freebsd.org/D56639

 sys/fs/nullfs/null_vnops.c    |  12 +++++
 sys/kern/vfs_inotify.c        |   4 --
 tests/sys/kern/inotify_test.c | 112 ++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 124 insertions(+), 4 deletions(-)
Comment 3 commit-hook freebsd_committer freebsd_triage 2026-05-03 15:46:44 UTC
A commit in branch stable/15 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=10567c80b6845717306c5301d104e7440033c4f5

commit 10567c80b6845717306c5301d104e7440033c4f5
Author:     Mark Johnston <markj@FreeBSD.org>
AuthorDate: 2026-04-26 01:35:37 +0000
Commit:     Mark Johnston <markj@FreeBSD.org>
CommitDate: 2026-05-03 14:58:57 +0000

    nullfs: Clear inotify flags during reclaim

    The inotify flags are copied from the lower vnode into the nullfs vnode
    so that the INOTIFY() macro will invoke VOP_INOTIFY on the nullfs vnode;
    this is then bypassed to the lower vnode.  However, when a nullfs vnode
    is reclaimed we should clear these flags, as the vnode is now doomed and
    no longer forwards VOPs to the lower vnode.

    Add regression tests.  Remove a test in vn_inotify_revoke() which is no
    longer needed after this change.

    PR:             292495
    Reviewed by:    kib
    Reported by:    Jed Laundry <jlaundry@jlaundry.com>
    Fixes:          f1f230439fa4 ("vfs: Initial revision of inotify")
    MFC after:      1 week
    Differential Revision:  https://reviews.freebsd.org/D56639

    (cherry picked from commit a02d794f5acd12ba3cf1de5c204a8dd56af47edd)

 sys/fs/nullfs/null_vnops.c    |  12 +++++
 sys/kern/vfs_inotify.c        |   4 --
 tests/sys/kern/inotify_test.c | 112 ++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 124 insertions(+), 4 deletions(-)
Comment 4 commit-hook freebsd_committer freebsd_triage 2026-05-07 20:41:15 UTC
A commit in branch releng/15.1 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=fdd8f30b9f314d04698f864c4b2a80b43a71114a

commit fdd8f30b9f314d04698f864c4b2a80b43a71114a
Author:     Mark Johnston <markj@FreeBSD.org>
AuthorDate: 2026-04-26 01:35:37 +0000
Commit:     Colin Percival <cperciva@FreeBSD.org>
CommitDate: 2026-05-07 20:40:04 +0000

    nullfs: Clear inotify flags during reclaim

    The inotify flags are copied from the lower vnode into the nullfs vnode
    so that the INOTIFY() macro will invoke VOP_INOTIFY on the nullfs vnode;
    this is then bypassed to the lower vnode.  However, when a nullfs vnode
    is reclaimed we should clear these flags, as the vnode is now doomed and
    no longer forwards VOPs to the lower vnode.

    Add regression tests.  Remove a test in vn_inotify_revoke() which is no
    longer needed after this change.

    Approved by:    re (cperciva)
    PR:             292495
    Reviewed by:    kib
    Reported by:    Jed Laundry <jlaundry@jlaundry.com>
    Fixes:          f1f230439fa4 ("vfs: Initial revision of inotify")
    MFC after:      1 week
    Differential Revision:  https://reviews.freebsd.org/D56639

    (cherry picked from commit a02d794f5acd12ba3cf1de5c204a8dd56af47edd)
    (cherry picked from commit 10567c80b6845717306c5301d104e7440033c4f5)

 sys/fs/nullfs/null_vnops.c    |  12 +++++
 sys/kern/vfs_inotify.c        |   4 --
 tests/sys/kern/inotify_test.c | 112 ++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 124 insertions(+), 4 deletions(-)
Comment 5 Mark Johnston freebsd_committer freebsd_triage 2026-05-08 13:40:29 UTC
This will be fixed in 15.1.  Thank you for the report.