Comment 1 Andrey V. Elsukov 2017-08-12 18:36:40 UTC

ENOBUFS usually means that ND layer can not resolve L2 address. And this is because you incorrectly configured your IPv6.

(In reply to Andrey V. Elsukov from comment #1)

Read the description of the bug again.
I described how I managed to get the ipv6 to work.

Comment 3 Andrey V. Elsukov 2017-08-12 19:48:59 UTC

(In reply to Vladislav V. Prodan from comment #2)
> Read the description of the bug again.
> I described how I managed to get the ipv6 to work.

It is wrong, and this is why it doesn't work.

> ifconfig_em0_ipv6="inet6 2001:41d0:e:XXX::1/128"

What is the reason of /128 prefix length?

> ipv6_static_routes="ipv6gw"
> ipv6_route_ipv6gw="-host 2001:41d0:000e:03ff:ff:ff:ff:ff -iface em0"

This will not work on FreeBSD due to implementation specificity. 

> ipv6_defaultrouter="2001:41d0:000e:03ff:ff:ff:ff:ff"

When the kernel is going to send IPv6 packet, it does L2 lookup to determine L2 address. ND6 code does lookup for destination address only when an address is considered as neighbor. When you have not configured the correct prefix, the ND6 has not any interfaces where the destination address can be considered as neighbor. In your case even the L2 address of default router can not be found.

(In reply to Andrey V. Elsukov from comment #3)

>> Read the description of the bug again.
>> I described how I managed to get the ipv6 to work.

>It is wrong, and this is why it doesn't work.

It's someone who did not quite correctly implement the IPv6 implementation in FreeBSD.
I can give a working example for Debian, there is the same crutch in the destination route for the default router.

>> ifconfig_em0_ipv6="inet6 2001:41d0:e:XXX::1/128"

>What is the reason of /128 prefix length?

So decided marketers in the OVH. I already pointed out to them that there are a number of vulnerabilities related to the substitution of IPv6 addresses.
Personally, I only need one ipv6 address.

>> ipv6_static_routes="ipv6gw"
>> ipv6_route_ipv6gw="-host 2001:41d0:000e:03ff:ff:ff:ff:ff -iface em0"

>This will not work on FreeBSD due to implementation specificity. 

Why not? It works the same.

# netstat -rn6
Routing tables

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
default                           fe80::12bd:18ff:fee5:ff80%em0 UG          em0
::1                               link#2                        UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
2001:41d0:e:XXX::/56              link#1                        U           em0
2001:41d0:e:XXX::1                link#1                        UHS         lo0
...
2001:41d0:e:3ff:ff:ff:ff:ff       00:22:4d:ad:ff:dc             UHS         em0
...
fe80::/10                         ::1                           UGRS        lo0
fe80::%em0/64                     link#1                        U           em0
fe80::222:4dff:fead:ffdc%em0      link#1                        UHS         lo0
fe80::%lo0/64                     link#2                        U           lo0
fe80::1%lo0                       link#2                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0


>When the kernel is going to send IPv6 packet, it does L2 lookup to determine L2 address. ND6 code does lookup for destination address only when an address is considered as neighbor. When you have not configured the correct prefix, the ND6 has not any interfaces where the destination address can be considered as neighbor. In your case even the L2 address of default router can not be found.

Honestly, I do not care how ND6 is implemented in FreeBSD.
I have issued a static IPv6 address with a given mask.
I have an IPv6 router address by default.
So what if he's not in the L2 segment? The network interface, where this Ipv6 is located, I know. Next, with these packages, the switches and routers of the hoster will be sorted out.

I have a typical network scheme for many hosters dedicated servers.
I have a test machine with which I can work for a while to correct this abnormal behavior.
If you have the time and the desire to eliminate this, in personal correspondence I will provide access to the server.

A similar problem.

Hetzner data center.

The server is loaded into rescue mode with FreeBSD 12.0 x64.

[root@rescue ~]# uname -a
FreeBSD core.domain.com 12.0-RELEASE FreeBSD 12.0-RELEASE r341666 GENERIC  amd64

# I turn on ipv6
sysctl net.inet6.ip6.accept_rtadv=1
ifconfig igb0 inet6 2a01:4f8:241:XXXX::2/64
route -6 add default -iface igb0

# ping6 -c 4 ya.ru
PING6(56=40+8+8 bytes) 2a01:4f8:241:XXXX::2 --> 2a02:6b8::2:242
ping6: sendmsg: No buffer space available
ping6: wrote ya.ru 16 chars, ret=-1
ping6: sendmsg: No buffer space available
ping6: wrote ya.ru 16 chars, ret=-1
ping6: sendmsg: No buffer space available
ping6: wrote ya.ru 16 chars, ret=-1
ping6: sendmsg: No buffer space available
ping6: wrote ya.ru 16 chars, ret=-1

--- ya.ru ping6 statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss



[root@rescue ~]# ifconfig
igb0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=e507bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,TSO4,TSO6,LRO,VLAN_HWFILTER,VLAN_HWTSO,RXCSUM_IPV6,TXCSUM_IPV6>
        ether 00:1e:67:b4:f9:e7
        inet 116.202.XXX.XXX netmask 0xffffffc0 broadcast 116.202.XXX.XXX
        inet6 2a01:4f8:241:XXXX::2 prefixlen 64
        inet6 fe80::21e:67ff:feb4:f9e7%igb0 prefixlen 64 scopeid 0x1
        media: Ethernet autoselect (1000baseT <full-duplex>)
        status: active
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
        options=680003<RXCSUM,TXCSUM,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6>
        inet6 ::1 prefixlen 128
        inet6 fe80::1%lo0 prefixlen 64 scopeid 0x2
        inet 127.0.0.1 netmask 0xff000000
        groups: lo
        nd6 options=21<PERFORMNUD,AUTO_LINKLOCAL>

[root@rescue ~]# netstat -rn6
Routing tables

Internet6:
Destination                       Gateway                       Flags     Netif Expire
::/96                             ::1                           UGRS        lo0
default                           00:1e:67:b4:f9:e7             US         igb0
::1                               link#2                        UH          lo0
::ffff:0.0.0.0/96                 ::1                           UGRS        lo0
2a01:4f8:241:XXXX::/64            link#1                        U          igb0
2a01:4f8:241:XXXX::2              link#1                        UHS         lo0
fe80::/10                         ::1                           UGRS        lo0
fe80::%igb0/64                    link#1                        U          igb0
fe80::21e:67ff:feb4:f9e7%igb0     link#1                        UHS         lo0
fe80::%lo0/64                     link#2                        U           lo0
fe80::1%lo0                       link#2                        UHS         lo0
ff02::/16                         ::1                           UGRS        lo0

(In reply to Vladislav V. Prodan from comment #5)

[root@rescue ~]# kldstat
Id Refs Address                Size Name
 1   14 0xffffffff80200000  243cd00 kernel
 2    1 0xffffffff89e21000     81f0 tmpfs.ko
 3    1 0xffffffff89e2a000   247e20 zfs.ko
 4    1 0xffffffff8a072000     7628 opensolaris.ko
 5    1 0xffffffff8a07a000     1fd4 geom_nop.ko

[root@rescue ~]# netstat -m
4196/4669/8865 mbufs in use (current/cache/total)
4103/2671/6774/4000000 mbuf clusters in use (current/cache/total/max)
7/2017 mbuf+clusters out of packet secondary zone in use (current/cache)
0/17/17/1004726 4k (page size) jumbo clusters in use (current/cache/total/max)
0/0/0/297696 9k jumbo clusters in use (current/cache/total/max)
0/0/0/167454 16k jumbo clusters in use (current/cache/total/max)
9255K/6577K/15832K bytes allocated to network (current/cache/total)
0/0/0 requests for mbufs denied (mbufs/clusters/mbuf+clusters)
0/0/0 requests for mbufs delayed (mbufs/clusters/mbuf+clusters)
0/0/0 requests for jumbo clusters delayed (4k/9k/16k)
0/0/0 requests for jumbo clusters denied (4k/9k/16k)
0 sendfile syscalls
0 sendfile syscalls completed without I/O request
0 requests for I/O initiated by sendfile
0 pages read by sendfile as part of a request
0 pages were valid at time of a sendfile request
0 pages were valid and substituted to bogus page
0 pages were requested for read ahead by applications
0 pages were read ahead by sendfile
0 times sendfile encountered an already busy page
0 requests for sfbufs denied
0 requests for sfbufs delayed

(In reply to Vladislav V. Prodan from comment #6)


I apologize, it was necessary to assign the gateway for ipv6 correctly.

route -6 change default fe80::1%igb0

Same issue here with a box at ultravps.eu. The gateway is outside of the /64, set a host route to it and the defaultroute onto the gateway, but it does not work.

Problem still exists, without properly fix. Nice job FreeBSD team!

Workaround for OVH dedicated and cloud server:

ipv6_activate_all_interfaces="YES"
ipv6_default_interface="em0"
ifconfig_em0_ipv6="inet6 accept_rtadv IPV6_ADDRESS prefixlen 128"
ipv6_static_routes="ovh"
ipv6_route_ovh="-host IPV6_GATEWAY -iface em0"
ipv6_defaultrouter="IPV6_GATEWAY"

I managed to reproduce this on a FreeBSD 12.2 box and CentOS 8 box. For short, the OVH guides are broken. They works on Linux but not on FreeBSD 11 and FreeBSD 12.

A workaround to fix it. Register a static NDP entry,

# ndp -n -s IPV6_GATEWAY GATEWAY_LINKLAYER_ADDRESS

to make it permanent,

# sysrc static_ndp_pairs=gw
# sysrc static_ndp_gw="IPV6_GATEWAY GATEWAY_LINKLAYER_ADDRESS"

CC Andrey V. Elsukov to review this workaround.

This workaround will stop to work as soon as the hoster changes the router interface facing your server

(In reply to Alarig Le Lay from comment #11)
Obviously.