Bug 250870 - vnet jail with samba can take down my node
Summary: vnet jail with samba can take down my node
Status: Closed FIXED
Alias: None
Product: Base System
Classification: Unclassified
Component: misc (show other bugs)
Version: 12.1-RELEASE
Hardware: amd64 Any
: --- Affects Some People
Assignee: Kristof Provost
URL: https://reviews.freebsd.org/D27378
Keywords:
Depends on:
Blocks:
 
Reported: 2020-11-04 19:43 UTC by Sebastian S
Modified: 2021-01-29 01:21 UTC (History)
6 users (show)

See Also:
kp: mfc-stable12+


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sebastian S 2020-11-04 19:43:58 UTC
Hey Devs , 

I'm running samba in one of my jails and this jail is possible to take down my small little server. 

At the moment I'm not sure if this only happens if I start the jail or when I stop it. 


Nov  2 22:33:52 freebsd kernel: ng_ether_ifnet_arrival_event: can't re-name node eth0
Nov  2 22:33:52 freebsd kernel: nd6_dad_timer: cancel DAD on epair2a because of ND6_IFF_IFDISABLED.
Nov  2 22:33:53 freebsd kernel: ng_ether_ifnet_arrival_event: can't re-name node eth0
Nov  2 22:33:53 freebsd kernel: ng_ether_ifnet_arrival_event: can't re-name node eth1
Nov  2 22:33:58 freebsd kernel: nd6_dad_timer: cancel DAD on epair4a because of ND6_IFF_IFDISABLED.
Nov  2 22:33:58 freebsd kernel: ng_ether_ifnet_arrival_event: can't re-name node eth0
Nov  2 22:34:33 freebsd kernel: nd6_dad_timer: cancel DAD on tap4 because of ND6_IFF_IFDISABLED.
Nov  2 22:36:02 freebsd kernel: ng_ether_ifnet_arrival_event: can't re-name node epair91b
Nov  2 22:36:08 freebsd kernel: ng_ether_ifnet_arrival_event: can't re-name node epair90b
Nov  2 22:36:10 <3.3> smb nmbd[15833]: [2020/11/02 22:36:10.942003,  0] ../../lib/util/become_daemon.c:147(daemon_status)
Nov  2 22:36:10 <3.3> smb nmbd[15833]:   daemon_status: daemon 'nmbd' : No local IPv4 non-loopback interfaces available, waiting for interface ...
Nov  2 22:36:10 <3.3> smb nmbd[15833]: [2020/11/02 22:36:10.942168,  0] ../../source3/nmbd/nmbd_subnetdb.c:254(create_subnets)
Nov  2 22:36:10 <3.3> smb nmbd[15833]:   NOTE: NetBIOS name resolution is not supported for Internet Protocol Version 6 (IPv6).
Nov  2 22:36:11 <3.3> smb smbd[15837]: [2020/11/02 22:36:11.266743,  0] ../../lib/util/become_daemon.c:136(daemon_ready)
Nov  2 22:36:11 <3.3> smb smbd[15837]:   daemon_ready: daemon 'smbd' finished starting up and ready to serve connections
Nov  2 22:36:14 freebsd kernel: ng_ether_ifnet_arrival_event: can't re-name node epair89b
Nov  2 22:36:15 <3.3> smb nmbd[15833]: [2020/11/02 22:36:15.955385,  0] ../../lib/util/become_daemon.c:136(daemon_ready)
Nov  2 22:36:15 <3.3> smb nmbd[15833]:   daemon_ready: daemon 'nmbd' finished starting up and ready to serve connections
Nov  2 22:36:19 <3.3> smb nmbd[15833]: [2020/11/02 22:36:19.968272,  0] ../../source3/libsmb/nmblib.c:924(send_udp)
Nov  2 22:36:19 <3.3> smb nmbd[15833]:   Packet send failed to 192.168.10.255(137) ERRNO=Host is down
Nov  2 22:36:19 <3.3> smb nmbd[15833]: [2020/11/02 22:36:19.968510,  0] ../../source3/nmbd/nmbd_packets.c:1639(retransmit_or_expire_response_records)
Nov  2 22:36:19 <3.3> smb nmbd[15833]:   retransmit_or_expire_response_records: Failed to resend packet id 15156 to IP 192.168.10.255 on subnet 192.168.10.150
Nov  2 22:36:19 <3.3> smb nmbd[15833]: [2020/11/02 22:36:19.968539,  0] ../../source3/libsmb/nmblib.c:924(send_udp)
Nov  2 22:36:19 <3.3> smb nmbd[15833]:   Packet send failed to 192.168.10.255(137) ERRNO=Host is down
Nov  2 22:36:19 <3.3> smb nmbd[15833]: [2020/11/02 22:36:19.968562,  0] ../../source3/nmbd/nmbd_packets.c:1639(retransmit_or_expire_response_records)
Nov  2 22:36:19 <3.3> smb nmbd[15833]:   retransmit_or_expire_response_records: Failed to resend packet id 15157 to IP 192.168.10.255 on subnet 192.168.10.150
Nov  2 22:36:19 <3.3> smb nmbd[15833]: [2020/11/02 22:36:19.968596,  0] ../../source3/libsmb/nmblib.c:924(send_udp)
Nov  2 22:36:19 <3.3> smb nmbd[15833]:   Packet send failed to 192.168.10.255(137) ERRNO=Host is down
Nov  2 22:36:19 <3.3> smb nmbd[15833]: [2020/11/02 22:36:19.968618,  0] ../../source3/nmbd/nmbd_packets.c:1639(retransmit_or_expire_response_records)
Nov  2 22:36:19 <3.3> smb nmbd[15833]:   retransmit_or_expire_response_records: Failed to resend packet id 15158 to IP 192.168.10.255 on subnet 192.168.10.150
Nov  2 22:36:19 <3.3> smb nmbd[15833]: [2020/11/02 22:36:19.968648,  0] ../../source3/libsmb/nmblib.c:924(send_udp)
Nov  2 22:36:19 <3.3> smb nmbd[15833]:   Packet send failed to 192.168.10.255(137) ERRNO=Host is down
Nov  2 22:36:19 <3.3> smb nmbd[15833]: [2020/11/02 22:36:19.968671,  0] ../../source3/nmbd/nmbd_packets.c:1639(retransmit_or_expire_response_records)
Nov  2 22:36:19 <3.3> smb nmbd[15833]:   retransmit_or_expire_response_records: Failed to resend packet id 15159 to IP 192.168.10.255 on subnet 192.168.10.150
Nov  2 22:36:19 <3.3> smb nmbd[15833]: [2020/11/02 22:36:19.968696,  0] ../../source3/libsmb/nmblib.c:924(send_udp)
Nov  2 22:36:19 <3.3> smb nmbd[15833]:   Packet send failed to 192.168.10.255(137) ERRNO=Host is down
Nov  2 22:36:19 <3.3> smb nmbd[15833]: [2020/11/02 22:36:19.968727,  0] ../../source3/nmbd/nmbd_packets.c:1639(retransmit_or_expire_response_records)
Nov  2 22:36:19 <3.3> smb nmbd[15833]:   retransmit_or_expire_response_records: Failed to resend packet id 15161 to IP 192.168.10.255 on subnet 192.168.10.150
Nov  2 22:36:20 <3.3> smb nmbd[15833]: [2020/11/02 22:36:20.988941,  0] ../../source3/libsmb/nmblib.c:924(send_udp)
Nov  2 22:36:20 <3.3> smb nmbd[15833]:   Packet send failed to 192.168.10.255(137) ERRNO=Host is down
Nov  2 22:36:20 <3.3> smb nmbd[15833]: [2020/11/02 22:36:20.989031,  0] ../../source3/nmbd/nmbd_packets.c:1639(retransmit_or_expire_response_records)
Nov  2 22:36:20 <3.3> smb nmbd[15833]:   retransmit_or_expire_response_records: Failed to resend packet id 15161 to IP 192.168.10.255 on subnet 192.168.10.150
Nov  2 22:36:31 <3.3> smb nmbd[15833]: [2020/11/02 22:36:31.995786,  0] ../../source3/libsmb/nmblib.c:924(send_udp)
Nov  2 22:36:31 <3.3> smb nmbd[15833]:   Packet send failed to 192.168.10.255(138) ERRNO=Host is down
Nov  2 22:36:33 <3.3> smb nmbd[15833]: [2020/11/02 22:36:33.001826,  0] ../../source3/libsmb/nmblib.c:924(send_udp)
Nov  2 22:36:33 <3.3> smb nmbd[15833]:   Packet send failed to 192.168.10.255(138) ERRNO=Host is down
Nov  2 22:36:34 <3.3> smb nmbd[15833]: [2020/11/02 22:36:34.013256,  0] ../../source3/libsmb/nmblib.c:924(send_udp)
Nov  2 22:36:34 <3.3> smb nmbd[15833]:   Packet send failed to 192.168.10.255(138) ERRNO=Host is down
Nov  2 22:36:36 <3.3> smb nmbd[15833]: [2020/11/02 22:36:36.025470,  0] ../../source3/libsmb/nmblib.c:924(send_udp)
Nov  2 22:36:36 <3.3> smb nmbd[15833]:   Packet send failed to 192.168.10.255(138) ERRNO=Host is down
Nov  2 22:36:38 <3.3> smb nmbd[15833]: [2020/11/02 22:36:38.043048,  0] ../../source3/libsmb/nmblib.c:924(send_udp)
Nov  2 22:36:38 <3.3> smb nmbd[15833]:   Packet send failed to 192.168.10.255(138) ERRNO=Host is down
Nov  2 22:36:38 <3.3> smb nmbd[15833]: [2020/11/02 22:36:38.043164,  0] ../../source3/libsmb/nmblib.c:924(send_udp)
Nov  2 22:36:38 <3.3> smb nmbd[15833]:   Packet send failed to 192.168.10.255(137) ERRNO=Host is down
Nov  2 22:36:38 <3.3> smb nmbd[15833]: [2020/11/02 22:36:38.043189,  0] ../../source3/nmbd/nmbd_packets.c:180(send_netbios_packet)
Nov  2 22:36:38 <3.3> smb nmbd[15833]:   send_netbios_packet: send_packet() to IP 192.168.10.255 port 137 failed
Nov  2 22:36:38 <3.3> smb nmbd[15833]: [2020/11/02 22:36:38.043231,  0] ../../source3/nmbd/nmbd_nameregister.c:582(register_name)
Nov  2 22:36:38 <3.3> smb nmbd[15833]:   register_name: Failed to send packet trying to register name ^A^B__MSBROWSE__^B<01>
Nov  2 22:36:52 <5.3> unifi syslogd: exiting on signal 15
Nov  2 22:36:54 <3.3> smb nmbd[15833]: [2020/11/02 22:36:54.472914,  0] ../../source3/nmbd/nmbd.c:59(terminate)
Nov  2 22:36:54 <3.3> smb nmbd[15833]:   Got SIGTERM: going down...
Nov  2 22:36:54 <3.3> smb nmbd[15833]: [2020/11/02 22:36:54.473029,  0] ../../source3/libsmb/nmblib.c:924(send_udp)
Nov  2 22:36:54 <3.3> smb nmbd[15833]:   Packet send failed to 192.168.10.255(138) ERRNO=Host is down
Nov  2 22:36:54 <5.3> smb sysl

atal trap 12: page fault while in kernel mode
cpuid = 4; apic id = 04
fault virtual address   = 0x40
fault code              = supervisor read data, page not present
instruction pointer     = 0x20:0xffffffff83253f5f
stack pointer           = 0x28:0xfffffe0075d9b800
frame pointer           = 0x28:0xfffffe0075d9b840
code segment            = base rx0, limit 0xfffff, type 0x1b
                        = DPL 0, pres 1, long 1, def32 0, gran 1
processor eflags        = interrupt enabled, resume, IOPL = 0
currentogd: exiFting on signal 15
 process                = 0 (thread taskq)
trap number             = 12
panic: page fault
cpuid = 4
time = 1604353014
KDB: stack backtrace:
#0 0xffffffff80c1d2f7 at kdb_backtrace+0x67
#1 0xffffffff80bd062d at vpanic+0x19d
#2 0xffffffff80bd0483 at panic+0x43
#3 0xffffffff810a8dcc at trap_fatal+0x39c
#4 0xffffffff810a8e19 at trap_pfault+0x49
#5 0xffffffff810a840f at trap+0x29f
#6 0xffffffff81081c9c at calltrap+0x8
#7 0xffffffff80cd041d at if_down+0x14d
#8 0xffffffff80ccdc5a at if_detach_internal+0x87a
#9 0xffffffff80cd496c at if_vmove+0x3c
#10 0xffffffff80cd4918 at vnet_if_return+0x48
#11 0xffffffff80cfe334 at vnet_destroy+0x124
#12 0xffffffff80b988d0 at prison_deref+0x2a0
#13 0xffffffff80c2fad4 at taskqueue_run_locked+0x154
#14 0xffffffff80c30e08 at taskqueue_thread_loop+0x98
#15 0xffffffff80b90c43 at fork_exit+0x83
#16 0xffffffff81082cde at fork_trampoline+0xe
Uptime: 4m38s
Dumping 1937 out of 32617 MB:..1%..11%..21%..31%..41%..51%..61%..71%..81%..91%
Dump complete
Comment 1 Kristof Provost freebsd_committer freebsd_triage 2020-11-04 20:25:55 UTC
See PR#244703, PR#238870. It's a know issue destroying vnet jails (and you'll only see it on destroy).

12.2 contains a mitigation that doesn't fully fix the problem but does make it less likely.
Comment 2 Sebastian S 2020-11-16 17:24:30 UTC
thank you !
Comment 3 commit-hook freebsd_committer freebsd_triage 2020-12-01 16:24:31 UTC
A commit references this bug:

Author: kp
Date: Tue Dec  1 16:24:00 UTC 2020
New revision: 368237
URL: https://svnweb.freebsd.org/changeset/base/368237

Log:
  if: Fix panic when destroying vnet and epair simultaneously

  When destroying a vnet and an epair (with one end in the vnet) we often
  panicked. This was the result of the destruction of the epair, which destroys
  both ends simultaneously, happening while vnet_if_return() was moving the
  struct ifnet to its home vnet. This can result in a freed ifnet being re-added
  to the home vnet V_ifnet list. That in turn panics the next time the ifnet is
  used.

  Prevent this race by ensuring that vnet_if_return() cannot run at the same time
  as if_detach() or epair_clone_destroy().

  PR:		238870, 234985, 244703, 250870
  MFC after:	2 weeks
  Sponsored by:	Modirum MDPay
  Differential Revision:	https://reviews.freebsd.org/D27378

Changes:
  head/sys/net/if.c
Comment 4 Kubilay Kocak freebsd_committer freebsd_triage 2020-12-08 01:39:40 UTC
^Triage: Assign to committer resolving
Comment 5 commit-hook freebsd_committer freebsd_triage 2020-12-15 15:34:34 UTC
A commit references this bug:

Author: kp
Date: Tue Dec 15 15:33:29 UTC 2020
New revision: 368663
URL: https://svnweb.freebsd.org/changeset/base/368663

Log:
  MFC r368237:

  if: Fix panic when destroying vnet and epair simultaneously

  When destroying a vnet and an epair (with one end in the vnet) we often
  panicked. This was the result of the destruction of the epair, which destroys
  both ends simultaneously, happening while vnet_if_return() was moving the
  struct ifnet to its home vnet. This can result in a freed ifnet being re-added
  to the home vnet V_ifnet list. That in turn panics the next time the ifnet is
  used.

  Prevent this race by ensuring that vnet_if_return() cannot run at the same time
  as if_detach() or epair_clone_destroy().

  PR:		238870, 234985, 244703, 250870
  Sponsored by:	Modirum MDPay

Changes:
_U  stable/12/
  stable/12/sys/net/if.c
Comment 6 commit-hook freebsd_committer freebsd_triage 2021-01-29 01:06:11 UTC
A commit in branch releng/12.1 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=e0c15f45abd4bd5165e11b557a8c90d0faf5cfeb

commit e0c15f45abd4bd5165e11b557a8c90d0faf5cfeb
Author:     Kristof Provost <kp@FreeBSD.org>
AuthorDate: 2021-01-18 21:55:53 +0000
Commit:     Ed Maste <emaste@FreeBSD.org>
CommitDate: 2021-01-29 00:58:55 +0000

    MFC r368237: if: Fix panic when destroying vnet and epair simultaneously

    When destroying a vnet and an epair (with one end in the vnet) we often
    panicked. This was the result of the destruction of the epair, which destroys
    both ends simultaneously, happening while vnet_if_return() was moving the
    struct ifnet to its home vnet. This can result in a freed ifnet being re-added
    to the home vnet V_ifnet list. That in turn panics the next time the ifnet is
    used.

    Prevent this race by ensuring that vnet_if_return() cannot run at the same time
    as if_detach() or epair_clone_destroy().

    PR:             238870, 234985, 244703, 250870
    Sponsored by:   Modirum MDPay
    Approved by:    so

 sys/net/if.c     | 147 +++++++++++++++++++++++++++++++++++++------------------
 sys/net/if_var.h |  24 ++-------
 2 files changed, 104 insertions(+), 67 deletions(-)
Comment 7 commit-hook freebsd_committer freebsd_triage 2021-01-29 01:21:23 UTC
A commit in branch releng/12.2 references this bug:

URL: https://cgit.FreeBSD.org/src/commit/?id=e682b62c96e94c60d830e4414215032e0d4f8dad

commit e682b62c96e94c60d830e4414215032e0d4f8dad
Author:     Kristof Provost <kp@FreeBSD.org>
AuthorDate: 2020-09-12 16:33:05 +0000
Commit:     Ed Maste <emaste@FreeBSD.org>
CommitDate: 2021-01-29 01:14:24 +0000

    MFC r368237: if: Fix panic when destroying vnet and epair simultaneously

    When destroying a vnet and an epair (with one end in the vnet) we often
    panicked. This was the result of the destruction of the epair, which destroys
    both ends simultaneously, happening while vnet_if_return() was moving the
    struct ifnet to its home vnet. This can result in a freed ifnet being re-added
    to the home vnet V_ifnet list. That in turn panics the next time the ifnet is
    used.

    Prevent this race by ensuring that vnet_if_return() cannot run at the same time
    as if_detach() or epair_clone_destroy().

    PR:             238870, 234985, 244703, 250870
    Sponsored by:   Modirum MDPay
    Approved by:    so

 sys/net/if.c     | 147 +++++++++++++++++++++++++++++++++++++------------------
 sys/net/if_var.h |  24 ++-------
 2 files changed, 104 insertions(+), 67 deletions(-)