Bug 236105

Summary:	carp MASTER/MASTER status when adding or removing carp IP addresses
Product:	Base System	Reporter:	Kajetan Staszkiewicz <vegeta>
Component:	kern	Assignee:	freebsd-net (Nobody) <net>
Status:	New ---
Severity:	Affects Only Me	CC:	eugen, ozkan.kirik
Priority:	---
Version:	11.2-RELEASE
Hardware:	Any
OS:	Any

Description Kajetan Staszkiewicz 2019-02-28 12:27:54 UTC

I'm not sure how much of a bug this is, rather a feature request for special cases. 

carp HMAC is calculated using a hash of carp IP addresses for a given VHID. This means that if two routes are configured with different carp IP addresses, verifying signatures will fail and routers will ignore carp annoucements from eachother resulting in MASTER/MASTER operation.

However I understand this is a feature for enforcing configuration check, there are cases when this is a bad idea: on some routers I have carp addresses reconfigured tens of times a day. As it is impossible to configure both routers exactly at the same time, there is often a difference big enough to triger MASTER/MASTER operation which is then soon resolved. I value network stability more than configuration check enforced on me and if configuration was wrong, things s would break anyway and I would be notified by other means.

Please see https://github.com/innogames/freebsd/commit/a1b1ff410c0f532b67660c86593ef7bda2398be8 for my first attempt. It still lacks sysctl value set hook, so works fine only after carp addresses are configured. If you like the idea and believe it could be merged into kernel, I will add a set hook recalculating HMAC and post the patch here.

Comment 1 Ozkan KIRIK 2020-09-30 03:15:41 UTC

It will be a good option.

Comment 2 Eugene Grosbein freebsd_committer

2020-09-30 07:50:57 UTC

Have you tried setting sysctl net.inet.carp.demotion=240 before changing carp IP at backup and setting it back there after both master and backup configuraion changed? You would not need patching sources.

Comment 3 Ozkan KIRIK 2020-09-30 08:14:27 UTC

While master and backup configuration is updating, HMACs are changing also. So that, incrementing net.inet.carp.demotion don't help.
Both of master and backup, dismiss the recevied multicast messages because of different HMACs. By the way, MASTER/MASTER state occured.

Comment 4 Ozkan KIRIK 2020-09-30 08:15:23 UTC

FreeBSD stable/12 also same.

Comment 5 Kajetan Staszkiewicz 2020-09-30 09:04:26 UTC

Playing with demotion configuration won't help, as there is no demotion happening. It's a MASTER/MASTER operation due to configuration difference.

Comment 6 Eugene Grosbein freebsd_committer

2020-09-30 09:17:32 UTC

(In reply to Kajetan Staszkiewicz from comment #5)

Right, but why do you need to apply configuration to both master and backup in parallel? Do it sequentially, first remove vhid from the backup, then reconfigure the master and, at last, re-add vhid to the backup.

Comment 7 Ozkan KIRIK 2020-09-30 09:51:01 UTC

There are too many assigned IP addresses. It takes much time to remove/add ip addresses.

 # ifconfig | grep vhid\  | wc -l
    1075

--

Is there a way for removing/adding bulk ip addresses? Such as prepare and then swap atomically ifconfig configuration like ipfw does (ipfw sets can be swapped atomically)?

Comment 8 Eugene Grosbein freebsd_committer

2020-09-30 13:19:02 UTC

(In reply to Ozkan KIRIK from comment #7)

I don't think there is a way for bulk replacing other than writing your own binary performing same syscalls as ifconfig binary does. Maybe new libifconfig would help, it exists in 12-STABLE and will present in upcoming 12.2-RELEASE.

However, you still can process different vhid(s) in parallel but reconfigure master/backup of single vhid in sync.