Bug 141023

Summary: [carp] CARP arp replays with wrong src mac
Product: Base System Reporter: Alexey <Alexey>
Component: kernAssignee: Gleb Smirnoff <glebius>
Status: Closed FIXED    
Severity: Affects Only Me CC: nbari, rozhuk.im
Priority: Normal    
Version: 7.2-STABLE   
Hardware: Any   
OS: Any   

Description Alexey 2009-11-30 12:10:01 UTC
We have vlan interface configured:

Base:
vlan254: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=3<RXCSUM,TXCSUM>
        ether 00:15:17:bb:5c:db
        inet 10.254.31.253 netmask 0xffffff00 broadcast 10.254.31.255
        media: Ethernet autoselect (1000baseTX <full-duplex>)
        status: active
        vlan: 254 parent interface: em1
carp:
carp254: flags=49<UP,LOOPBACK,RUNNING> metric 0 mtu 1500
        inet 10.254.31.254 netmask 0xffffff00
        carp: MASTER vhid 35 advbase 1 advskew 10

(it will be same result if we do not use vlan, but configure on em, w/o vlans)

remote machine sent broadcars request for 10.254.31.254:
14:26:40.792991 00:22:b0:51:23:03 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 56: arp who-has 10.254.31.254 tell 10.254.31.232

and machive answered:
14:26:40.792997 00:15:17:bb:5c:db > 00:22:b0:51:23:03, ethertype ARP (0x0806), length 42: arp reply 10.254.31.254 is-at 00:00:5e:00:01:23

As we can see this reported mac 00:00:5e:00:01:23.. but source mac for
ARP REPLAY is 00:15:17:bb:5c:db, not 00:00:5e:00:01:23.

And currently I have some devices on network, who checked the SRC macs
carefully and ignored such replays.

Fix: 

No known fixes.. sorry.
How-To-Repeat: configure carp interface and ping such interface from remove machine.
look for tcpdump (-e) for results
Comment 1 Mark Linimon freebsd_committer freebsd_triage 2009-11-30 21:21:50 UTC
Responsible Changed
From-To: freebsd-bugs->freebsd-net

Over to maintainer(s).
Comment 2 Gleb Smirnoff freebsd_committer freebsd_triage 2011-10-15 10:08:48 UTC
State Changed
From-To: open->analyzed

Let's look at this closer. 


Comment 3 Gleb Smirnoff freebsd_committer freebsd_triage 2011-10-15 10:08:48 UTC
Responsible Changed
From-To: freebsd-net->glebius

I'm now working on major rewrite of CARP for FreeBSD 10, and 
I'd like to take all related PRs.
Comment 4 Gleb Smirnoff freebsd_committer freebsd_triage 2011-10-15 14:19:38 UTC
  Alexey,

  as you described in the original PR:

> and machive answered:
> 14:26:40.792997 00:15:17:bb:5c:db > 00:22:b0:51:23:03, ethertype ARP (0x0806), length 42: arp reply 10.254.31.254 is-at 00:00:5e:00:01:23
> 
> As we can see this reported mac 00:00:5e:00:01:23.. but source mac for
> ARP REPLAY is 00:15:17:bb:5c:db, not 00:00:5e:00:01:23.
> 
> And currently I have some devices on network, who checked the SRC macs
> carefully and ignored such replays.

According to RFC826, which is current standard for ARP implementations,
the hardware address in the transmission layer does not need to match
the hardware addrees in the ARP reply packet itself. At least I can't
find any evidence in the RFC that it must or even should. Moreover, it
is mentioned in the RFC that:

"Ethernet transmission layer (not necessarily accessible to the user)"

So, RFC states that data in the Ethernet header isn't crucial for
the operation of address resolution protocol.

Correct me, if I am wrong above.

Thus, any devices that perform such strict checking on ARP replies,
are violating RFC. And I hope these devices are capable to turning
this feature off. From my point of view, this strictness doesn't
bring in any security, since with any modern network card sender
hardware address can be faked with the same small efforts as faking
address in the ARP packet itself.

So, your devices should be fixed, not CARP in FreeBSD. However, we
are open to bringing in compatibility with other non-RFC-compliant
devices if it is possible to do this in non-destructive way, for
example having this feature off by default. If you are interested in
this feature, you can submit a patch, that implements it, it would
be reviewed and committed to source tree.

-- 
Totus tuus, Glebius.
Comment 5 Alexey Kouznetsov 2011-10-16 10:04:30 UTC
-----Original Message-----
From: Gleb Smirnoff [mailto:glebius@FreeBSD.org] 
  Alexey,

  as you described in the original PR:

>> and machive answered:
>> 14:26:40.792997 00:15:17:bb:5c:db > 00:22:b0:51:23:03, ethertype ARP
>>(0x0806), length 42: arp reply 10.254.31.254 is-at 00:00:5e:00:01:23
>> 
>> As we can see this reported mac 00:00:5e:00:01:23.. but source mac for
>> ARP REPLAY is 00:15:17:bb:5c:db, not 00:00:5e:00:01:23.

>> And currently I have some devices on network, who checked the SRC macs
>> carefully and ignored such replays.

> According to RFC826, which is current standard for ARP implementations,
> the hardware address in the transmission layer does not need to match
> the hardware addrees in the ARP reply packet itself. At least I can't
> find any evidence in the RFC that it must or even should. Moreover, it
> is mentioned in the RFC that:

> "Ethernet transmission layer (not necessarily accessible to the user)"

> So, RFC states that data in the Ethernet header isn't crucial for
> the operation of address resolution protocol.

> Correct me, if I am wrong above.

> Thus, any devices that perform such strict checking on ARP replies,
> are violating RFC. And I hope these devices are capable to turning
> this feature off. From my point of view, this strictness doesn't
> bring in any security, since with any modern network card sender
> hardware address can be faked with the same small efforts as faking
> address in the ARP packet itself.

> So, your devices should be fixed, not CARP in FreeBSD. However, we
> are open to bringing in compatibility with other non-RFC-compliant
> devices if it is possible to do this in non-destructive way, for
> example having this feature off by default. If you are interested in
> this feature, you can submit a patch, that implements it, it would
> be reviewed and committed to source tree.

Yes, it looks to be true. But real world is little bit more complex. Last
device I saw was Dlink DES 3028 with modern firmware, who also did filtering
as described above. (devices at examples above was not our, we are ISP and
devices was client's, not remember who was manufacturer for them. But
sometime clients did not understand why device used to be work at other ISP
and will not work here.)

Another related problem: only traffic with L2 src of carp interface
(00:00:5e:00:01:xx) is multicast traffic with CARP announces. So if you have
long network and block multicast at user's end, there are possible and
devices will not know where router live at and will flood over the subnet
(until traffic reach core, where we have enabled such multicast).

I did  some Q&D solution, I do arping from all carp interfaces with L2 SRC
00:00:5e:00:01:xx once a minute. In general it solved all our problems ;)

Btw I leave such ISP and currently I do not use carps, but I home I'll use
it in future.

With best regards
/Alexey
Comment 6 Gleb Smirnoff freebsd_committer freebsd_triage 2011-12-16 13:01:32 UTC
State Changed
From-To: analyzed->closed

We came to an agreement that CARP in FreeBSD doesn't violate ARP 
RFC, while some equipment does. 

Originator had been informed that a compatibility with weird equipment 
can be brought into FreeBSD, just send patches :) 

Since originator no longer has access to this eqiupment, close the PR.
Comment 7 nbari 2021-10-28 19:40:54 UTC
Any chance to allow CARP announce the mac address of the interface (VLAN -
 https://forums.FreeBSD.org/threads/carp-setup-hetzner-vswitch.82604/post-538993)

This will prevent having to use arping every minute