Bug 239417

Summary: ARP ping fails from the host to bridged vnet jails
Product: Base System Reporter: Yuri Victorovich <yuri>
Component: kernAssignee: freebsd-net (Nobody) <net>
Status: Closed Works As Intended    
Severity: Affects Only Me CC: eugen, jail
Priority: ---    
Version: 12.0-STABLE   
Hardware: Any   
OS: Any   

Description Yuri Victorovich freebsd_committer freebsd_triage 2019-07-24 04:04:00 UTC
I connect vnet jails to the host using netgraph bridge using this script /usr/src/share/examples/jails/jng:
> jng bridge jailnet sk0

I assign the IP address to jailnet from the LAN network after jailnet is transferred into the jail.

Connectivity with jail works in all directions, but arping from host to jail's IP address doesn't receive any response.
arping from jail to hosts's IP and to all other IPs, including other jails' IPs work.

Why arping doesn't receive responses from jail?

ipfw.ko isn't loaded into kernel.
Comment 1 Eugene Grosbein freebsd_committer freebsd_triage 2019-07-24 16:27:42 UTC
If you mean net/arping, the PR should be for Ports & Packages and you should show output of arping -v -v ...
Comment 2 Yuri Victorovich freebsd_committer freebsd_triage 2019-07-24 16:51:19 UTC
arping -v -v ,,, log:

> $ arping -v -v 192.168.5.205
> arping: clock_getres() = 0s 1ns
> arping: libnet_init(<null>)
> arping: Autodetected interface sk0
> arping: libnet_init(sk0)
> Timestamp types:
>   Name               Description
> arping: chroot(/var/empty): Operation not permitted
> arping: setgroups(0, NULL): Operation not permitted
> arping: setgid(): Operation not permitted
> arping: setuid(): Operation not permitted
> arping: pcap_get_selectable_fd(): 5
> This box:   Interface: sk0  IP: 192.168.5.3   MAC address: 00:1c:32:3a:87:32
> ARPING 192.168.5.205
> arping: sending packet at time 145827.861062692
> Timeout
> arping: sending packet at time 145828.867484471
> Timeout
> arping: sending packet at time 145829.873842180
> ^C

> the PR should be for Ports & Packages

But net/arping normally works, except for in this situation.
In this case it doesn't get ARP responses, which seems to be a kernel problem because ARP responses are supposed to come from all bridged peers.
arping of the same jail from another such jail, or from outside hosts works, just not from the host where jail resides.
Comment 3 Eugene Grosbein freebsd_committer freebsd_triage 2019-07-24 21:29:42 UTC
Your log shows that arping does not send packets to the bridge. It sends them to sk0 physical interface. No wonder it gets no answers from physical part of bridge where target IP does not exist.

You should move host's IP address from sk0 to bridge interface so apring and others send traffic through the bridge and not sk0.
Comment 4 Eugene Grosbein freebsd_committer freebsd_triage 2019-07-25 19:03:12 UTC
Yury, had you a chance to try with IP address moved to the bridge?

I'd suggest you create the bridge at startup and assign host's IP to it. Then add/remove members to the brige only.
Comment 5 Yuri Victorovich freebsd_committer freebsd_triage 2019-07-25 19:24:34 UTC
(In reply to Eugene Grosbein from comment #4)

Hi Eugene,

I am doing this in the context of the containerization application that I am working on, yet unannounced (https://github.com/yurivict/crate). It is already functional, but needs networking and configuration management bits to be added or fixed.

It runs programs and services in jails, and connects them to network dynamically using firewall rules or bridges.

The main mode of connectivity is the "blend" mode when it blends with the host IP address using epair(4) and firewall rules. This doesn't involve creation of IP addresses, and I have it working, except for some minor details.

The secondary mode is the "lan" mode which creates a dedicated LAN IP address for each container. I asked this question originally for this mode. Your suggestion about moving the IP address to the bridge makes sense. The process looks a bit more invasive than I originally thought it would be, because it would involve altering the network settings, moving the IP to another interface for the life of container or all containers and then moving it back, as per your advise. I will try this. This is only needed to find a spare LAN IP to allocate, and is not needed for subsequent container operation. Also this "lan" mode isn't strictly necessary because the "blend" mode should satisfy most use cases. I just came up with it, and thought that it would be interesting to implement it too.



Thank you for your help!
Yuri
Comment 6 Eugene Grosbein freebsd_committer freebsd_triage 2019-07-25 22:21:32 UTC
Closing for now. Feel free to re-open if you find it's really kernel problem.