Bug 221379 - bsdinstall(8): Installer doesn't support default router outside local subnet
Summary: bsdinstall(8): Installer doesn't support default router outside local subnet
Status: New
Alias: None
Product: Base System
Classification: Unclassified
Component: bin (show other bugs)
Version: CURRENT
Hardware: Any Any
: --- Affects Some People
Assignee: freebsd-sysinstall mailing list
URL:
Keywords: patch
Depends on:
Blocks:
 
Reported: 2017-08-09 23:02 UTC by Nils Steinger
Modified: 2017-08-15 19:35 UTC (History)
1 user (show)

See Also:


Attachments
Give the user an option to configure resolv.conf even when they skipped the configuration of network addresses and gateways (969 bytes, patch)
2017-08-09 23:03 UTC, Nils Steinger
no flags Details | Diff
Detect the gateway-outside-subnet scenario and generate a working rc.conf for it (2.79 KB, patch)
2017-08-09 23:03 UTC, Nils Steinger
no flags Details | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Nils Steinger 2017-08-09 23:02:41 UTC
*Problem:*

When the default gateway isn't contained in the subnet described by the local system's IP address and Subnet mask, bsdinstall(8) fails when it tries to add the default route, throwing the user back to the start of the install process.

*Background:*

IPv4 exhaustion has led some server providers to use rather questionable networking hacks to conserve IP space — namely, using gateway addresses outside the local subnet so the first address on the local subnet (x.x.x.1) doesn't have to be a router and can be used for customer machines instead.

I'm currently seeing this on a KVM virtual machine from OneProvider (an Online.net reseller), but apparently OVH, Hetzner, and a bunch of smaller players do the same thing.
My (slightly anonymized) network config:

    vtnet0:
        inet 195.154.123.234 netmask 0xffffffff broadcast 195.154.123.234

    Destination        Gateway            Flags     Netif Expire
    default            62.210.112.1       UGS      vtnet0
    62.210.112.1       52:54:00:00:82:41  UHS      vtnet0

On a "normal" FreeBSD system, this can be achieved using an rc.conf something like this:

    ifconfig_vtnet0="inet 195.154.123.234 netmask 255.255.255.255"
    gateway_if="vtnet0"
    gateway_ip="62.210.112.1"
    static_routes="gateway default"
    route_gateway="-host $gateway_ip -interface $gateway_if"
    route_default="default $gateway_ip"

Or temporarily using route(8):

    route add -host 62.210.112.1 -iface vtnet0
    route add default 62.210.112.1

*Problem details:*

The installer tries `route add default 62.210.112.1` directly, which of course fails because there is no route to 62.210.112.1 yet (see usr.sbin/bsdinstall/scripts/netconfig_ipv4:94 in base).

Manually configuring the network from a shell doesn't work because `netconfig_ipv4` *always* re-configures (and thereby breaks) the network, even when nothing was changed.
Responding "No" to "Would you like to configure IPv4 for this interface?" also doesn't help because something in bsdinstall(8) deletes the file behind the /etc/resolv.conf symlink every time the installer is launched.

*Possible solutions (as far as I can see):*

1. As a workaround, we could at least stop bsdinstall from deleting/overwriting /etc/resolv.conf unless the user explicitly requests changes to the configuration.
I've attached `always-write-resolvconf.patch` that gives the user an option to configure resolv.conf even when they skipped the configuration of network addresses and gateways.

2. Preferably, `netconfig_ipv4` should detect the gateway-outside-subnet scenario and generate a working rc.conf for it.
This is a bit more involved, mainly because we need to implement a portable method of detecting if a given IP (the gateway) is inside a given network (the IP + subnet mask of the local machine).
For this, I've built a truly awk-ful function — I had to reimplement bitwise operations from scratch because the awk version in FreeBSD base doesn't support the `and()` function…
From there on it's just some additional shell code along the lines of `if ! router_inside_subnet; then route add -host $defaultrouter -iface $interface; fi; route add default $defaultrouter`.
See `router-outside-subnet.patch` for that one.
Probably needs some improvements in terms of readability. Testing should be less of an issue: the only finicky bit is the is-it-inside-subnet check, and if that crashes or gives a false-negative result, the script just adds a superfluous route that doesn't hurt anyone.
Comment 1 Nils Steinger 2017-08-09 23:03:28 UTC
Created attachment 185214 [details]
Give the user an option to configure resolv.conf even when they skipped the configuration of network addresses and gateways
Comment 2 Nils Steinger 2017-08-09 23:03:55 UTC
Created attachment 185215 [details]
Detect the gateway-outside-subnet scenario and generate a working rc.conf for it
Comment 3 Nils Steinger 2017-08-09 23:09:44 UTC
And then…

*Same issue with IPv6:*

`netconfig_ipv6` has the exact same problem and should receive similar treatment.
However, my solution for IPv4 can't be applied verbatim because awk loses accuracy when going up to 2^64:

% awk 'BEGIN {print(2^64)}'
18446744073709551616
% awk 'BEGIN {print(2^64-1)}'
18446744073709551616
% awk 'BEGIN {print(2^64-1000)}'
18446744073709551616

So we would have to find another way to to subnet parsing for IPv6.