<info><title>Common Address Redundancy Protocol

    <info>

      (<acronym>CARP</acronym>)</title>

      <title>Common Address Redundancy Protocol

	(<acronym>CARP</acronym>)</title>

      same <acronym>IP</acronym> address.  In some configurations,

      same <acronym>IP</acronym> address(es) and can be used to

      this may be used for availability or load balancing.  Hosts

      achieve high availability.  Hosts will usually also have a

      may use separate <acronym>IP</acronym> addresses, as in the

      unique <acronym>IP</acronym> address for management and

      example provided here.</para>

      configuration, as in the example provided here.</para>

    <para>To enable support for <acronym>CARP</acronym>, the &os;

    <sect2 xml:id="carp-ha">

      kernel can be rebuilt as described in <xref linkend="kernelconfig"/> with the following option:</para>

      <title>Using <acronym>CARP</acronym> for High

	Availability</title>

    <programlisting>device	carp</programlisting>

      <para>One use of <acronym>CARP</acronym> is to provide

	high availability for one or more services.  This example

	configures failover support with three hosts, all with

	unique <acronym>IP</acronym> addresses and providing the same

	web content.  These machines are load balanced with a Round

	Robin <acronym>DNS</acronym> configuration.  The master and

	backup machines should preferrably be configured identically,

	other than their hostnames and management

	<acronym>IP</acronym> addresses.  These servers need to run

	the same services, such as the web server, with the same

	configuration.  When the failover occurs, requests to the

	service on the shared <acronym>IP</acronym> address can only

	be answered correctly if the backup server has access to the

	same content.  The backup machine has two additional

	<acronym>CARP</acronym> interfaces, one for each of the

	master content server's <acronym>IP</acronym> addresses.  When

	a failure occurs, the backup server will pick up the failed

	master machine's <acronym>IP</acronym> address, and the

	failure would go completely unnoticed by the users.  This

	example has two different masters named

	<systemitem>hosta.example.org</systemitem> and

	<systemitem>hostb.example.org</systemitem>, with

	a shared backup named

	<systemitem>hostc.example.org</systemitem>.</para>

    <para>Alternatively, the <filename>if_carp.ko</filename> module

      <para>Each virtual <acronym>IP</acronym> address has a unique

      can be loaded at boot time.  Add the following line to

	identification number known as a Virtual Host Identification

      <filename>/boot/loader.conf</filename>:</para>

	(<acronym>VHID</acronym>).  The <acronym>VHID</acronym> is

	used to indicate which machines are to share a common address.

	The <acronym>VHID</acronym> for each virtual

	<acronym>IP</acronym> address must be unique across the

	broadcast domain of the network interface.</para>

    </sect2>

    <programlisting>if_carp_load="YES"</programlisting>

    <sect2 xml:id="carp-10x">

      <title>Using <acronym>CARP</acronym> on &os; 10 and

	Later</title>

    <para><acronym>CARP</acronym> functionality should now be

      <para>Enable support for <acronym>CARP</acronym> by loading the

      available and may be tuned via several &man.sysctl.8;

	<filename>carp.ko</filename> kernel module in

      variables:</para>

	<filename>/boot/loader.conf</filename>:</para>

    <informaltable frame="none" pgwide="1">

      <programlisting>carp_load="YES"</programlisting>

      <tgroup cols="2">

	<thead>

	  <row>

	    <entry>OID</entry>

	    <entry>Description</entry>

	  </row>

	</thead>

	<tbody>

      <para>The <acronym>CARP</acronym> can also be built into the

	  <row>

	&os; kernel as described in <xref linkend="kernelconfig"/>

	    <entry><varname>net.inet.carp.allow</varname></entry>

	by enabling the option:</para>

	    <entry>Accept incoming <acronym>CARP</acronym> packets.

	      Enabled by default.</entry>

	  </row>

	  <row>

      <programlisting>device	carp</programlisting>

	    <entry><varname>net.inet.carp.preempt</varname></entry>

	    <entry>This option downs all of the

	      <acronym>CARP</acronym> interfaces on the host when one

	      goes down.  Disabled by default.</entry>

	  </row>

	  <row>

      <para>Set the hostname, configure the management

	    <entry><varname>net.inet.carp.log</varname></entry>

	<acronym>IP</acronym> address, then configure

	    <entry>A value of <literal>0</literal> disables any

	<acronym>CARP</acronym> and the <acronym>IP</acronym> address

	      logging.  A value of <literal>1</literal> enables

	to be shared by adding the required lines to

	      logging of bad <acronym>CARP</acronym> packets.  Values

	<filename>/etc/rc.conf</filename>.  Here are example lines for

	      greater than <literal>1</literal> enable logging of

	<systemitem>hosta.example.org</systemitem>:</para>

	      state changes for the <acronym>CARP</acronym>

	      interfaces.  The default value is

	      <literal>1</literal>.</entry>

	  </row>

	  <row>

      <programlisting>hostname="hosta.example.org"

	    <entry><varname>net.inet.carp.arpbalance</varname></entry>

ifconfig_em0="inet <systemitem class="ipaddress">192.168.1.3</systemitem> netmask 255.255.255.0"

	    <entry>Balance local network traffic using

ifconfig_em0_alias0="vhid 1 pass testpass alias <systemitem class="ipaddress">192.168.1.50</systemitem>/32"</programlisting>

	      <acronym>ARP</acronym>.  Disabled by default.</entry>

	  </row>

	  <row>

      <para>On <systemitem>hostb.example.org</systemitem>:</para>

	    <entry><varname>net.inet.carp.suppress_preempt</varname></entry>

	    <entry>A read-only variable showing the status of

	      preemption suppression.  Preemption can be suppressed

	      if the link on an interface is down.  A value of

	      <literal>0</literal> means that preemption is not

	      suppressed.  Every problem increments this

	      variable.</entry>

	  </row>

	</tbody>

      </tgroup>

    </informaltable>

    <para>The <acronym>CARP</acronym> devices themselves may be

      <programlisting>hostname="hostb.example.org"

      created using &man.ifconfig.8;:</para>

ifconfig_em0="inet <systemitem class="ipaddress">192.168.1.4</systemitem> netmask 255.255.255.0"

ifconfig_em0_alias0="vhid 2 pass testpass alias <systemitem class="ipaddress">192.168.1.51</systemitem>/32"</programlisting>

    <screen>&prompt.root; <userinput>ifconfig carp0 create</userinput></screen>

      <note>

	<para>The passwords specified with &man.ifconfig.8;

	  <option>pass</option> must be identical.

	  <acronym>CARP</acronym> will only listen to and accept

	  advertisements from machines with the correct password.  The

	  <acronym>VHID</acronym> must also be unique for each virtual

	  <acronym>IP</acronym> address.</para>

      </note>

    <para>In a real environment, each interface has a unique

      <para>The third machine,

      identification number known as a Virtual Host IDentification

	<systemitem>hostc.example.org</systemitem>,

      (<acronym>VHID</acronym>) which is used to distinguish the

	needs to be prepared so that it can handle failover from

      host on the network.</para>

	either of the previous host.  This machine is configured

	with two <acronym>CARP</acronym> <acronym>VHID</acronym>s, one

	to handle the virtual <acronym>IP</acronym> address of each

	of the master hosts.  Setting the <option>advskew</option>

	controls the <acronym>CARP</acronym> advertising skew.  The

	skew ensuring that the backup hosts advertises later than the

	master, and controls the order of precedence when there

	are multiple backup servers.  Set the configuration in

	<filename>/etc/rc.conf</filename>:</para>

    <sect2>

      <programlisting>hostname="hostc.example.org"

      <title>Using <acronym>CARP</acronym> for Server

ifconfig_em0="inet <systemitem class="ipaddress">192.168.1.5</systemitem> netmask 255.255.255.0"

	Availability</title>

ifconfig_em0_alias0="vhid 1 advskew 100 pass testpass alias <systemitem class="ipaddress">192.168.1.50</systemitem>/32"

ifconfig_em0_alias1="vhid 2 advskew 100 pass testpass alias <systemitem class="ipaddress">192.168.1.51</systemitem>/32"</programlisting>

      <para>One use of <acronym>CARP</acronym> is to provide server

      <para>Having the two <acronym>CARP</acronym>

	availability.  This example configures failover support for

	<acronym>VHID</acronym>s configured means that

	three hosts, all with unique <acronym>IP</acronym>

	<systemitem>hostc.example.org</systemitem> will notice if

	addresses and providing the same web content.  These machines

	either of the master servers becomes unavailable.  If a master

	act in conjunction with a Round Robin

	fails to advertise before the backup server, the backup server

	<acronym>DNS</acronym> configuration.  The failover machine

	will pick up the shared <acronym>IP</acronym> address until

	has two additional <acronym>CARP</acronym> interfaces, one

	the master becomes available again.</para>

	for each of the content server's

	<acronym>IP</acronym> addresses.  When a

	failure occurs, the failover server will pick up the failed

	machine's <acronym>IP</acronym> address.

	This means that the failure should go completely unnoticed

	by the user.  The failover server requires identical content

	and services as the other content servers it is expected to

	pick up load for.</para>

      <para>The two machines should be configured identically other

      <note>

	than their hostnames and <acronym>VHID</acronym>s.  This

	<para>Preemption is disabled by default.  If preemption has

	example calls these machines

	  been enabled, <systemitem>hostc.example.org</systemitem>

	<systemitem>hosta.example.org</systemitem> and

	  might not release the virtual <acronym>IP</acronym> address

	<systemitem>hostb.example.org</systemitem> respectively.  First, the

	  back to the original master server.  The administrator

	required lines for a <acronym>CARP</acronym> configuration

	  can force the backup server to return the

	have to be added to <filename>/etc/rc.conf</filename>.  Here

	  <acronym>IP</acronym> address to the master with the

	are the lines for

	  command:</para>

	<screen>&prompt.root; <command>ifconfig em0 vhid 1 state backup</command></screen>

      </note>

      <para>At this point, either networking must be restarted or the

	machine rebooted, then <acronym>CARP</acronym> is

	enabled.</para>

      <para><acronym>CARP</acronym> functionality can be controlled

	via several &man.sysctl.8; variables:</para>

      <informaltable frame="none" pgwide="1">

	<tgroup cols="2">

	  <thead>

	    <row>

	      <entry>OID</entry>

	      <entry>Description</entry>

	    </row>

	  </thead>

	  <tbody>

	    <row>

	      <entry><varname>net.inet.carp.allow</varname></entry>

	      <entry>Accept incoming <acronym>CARP</acronym> packets.

		Enabled by default.</entry>

	    </row>

	    <row>

	      <entry><varname>net.inet.carp.preempt</varname></entry>

	      <entry>Allow virtual hosts to preempt each other.  For

		firewalls and routers with multiple interfaces, it is

		desirable to failover all of the addresses running

		carp together when one of the physical interfaces

		goes down.  This is achieved with

		<varname>preempt</varname>.  When one of the physical

		interfaces of the master fails,

		<option>advskew</option> is demoted to a configured

		value on all its <acronym>CARP</acronym>

		<acronym>VHID</acronym>s.  Due to the preempt option,

		the backup host would start announcing itself, and

		thus preempt the master host on both interfaces

		instead of just the failed one.  Disabled by

		default.</entry>

	    </row>

	    <row>

	      <entry><varname>net.inet.carp.log</varname></entry>

	      <entry>Determines what events relating to

		<acronym>CARP</acronym> <acronym>VHID</acronym>s are

		logged.  A value of 0 disables any logging.  A value

		of 1 enables logging state changes of

		<acronym>CARP</acronym> <acronym>VHID</acronym>s.

		Values above 1 enable logging of bad

		<acronym>CARP</acronym> packets.  The default value is

		1.</entry>

	    </row>

	    <row>

	      <entry><varname>net.inet.carp.demotion</varname></entry>

	      <entry>This value shows the current level of

		<acronym>CARP</acronym> demotion.  The value is added

		to the actual <option>advskew</option> sent in

		announcements for all <acronym>VHID</acronym>s.

		During normal system operation the demotion factor is

		zero.  Problematic conditions automatically raise this

		level: when <acronym>CARP</acronym> experiences a

		problem with sending announcements, when an interface

		running a <acronym>VHID</acronym> goes down, or while

		the &man.pfsync.4; interface is not synchronized.  The

		demotion factor can also be adjusted manually by

		writing to this &man.sysctl.8; <acronym>OID</acronym>.

		The signed value set via &man.sysctl.8; is added to

		the current demotion factor.  This allows the

		behavior of <acronym>CARP</acronym> to be controlled

		depending on external conditions, like the status of

		some daemon utility.</entry>

	    </row>

	    <row>

	      <entry><varname>net.inet.carp.ifdown_demotion_factor</varname></entry>

	      <entry>This value is added to

		<varname>net.inet.carp.demotion</varname> when an

		interface running a <acronym>VHID</acronym> goes down.

		The default value is 240 (the maximum

		<option>advskew</option> value).</entry>

	    </row>

	    <row>

	      <entry><varname>net.inet.carp.senderr_demotion_factor</varname></entry>

	      <entry>This value is added to

		<varname>net.inet.carp.demotion</varname> when

		<acronym>CARP</acronym> experiences errors sending its

		announcements.  The default value is 240 (the maximum

		<option>advskew</option> value).</entry>

	    </row>

	  </tbody>

	</tgroup>

      </informaltable>

      <para>Other actions can be triggered from

	<acronym>CARP</acronym> events by using &man.devd.8;.  More

	information is available in &man.carp.4;.</para>

    </sect2>

    <sect2 xml:id="carp-9x">

      <title>Using <acronym>CARP</acronym> on &os; 9 and

	Earlier</title>

      <para>Enable support for <acronym>CARP</acronym> by loading the

	<filename>if_carp.ko</filename> kernel module in

	<filename>/boot/loader.conf</filename>:</para>

      <programlisting>if_carp_load="YES"</programlisting>

      <para>The <acronym>CARP</acronym> can also be built into the

	&os; kernel as described in <xref linkend="kernelconfig"/>

	by enabling the option:</para>

      <programlisting>device	carp</programlisting>

      <para>The <acronym>CARP</acronym> devices themselves may be

	created using &man.ifconfig.8;:</para>

      <screen>&prompt.root; <command>ifconfig carp0 create</command></screen>

      <para>Set the hostname, configure the management

	<acronym>IP</acronym> address, then configure

	<acronym>CARP</acronym> and the <acronym>IP</acronym> address

	to be shared by adding the required lines to

	<filename>/etc/rc.conf</filename>.  Here are example lines for

ifconfig_fxp0="inet 192.168.1.3 netmask 255.255.255.0"

ifconfig_fxp0="inet <systemitem class="ipaddress">192.168.1.3</systemitem> netmask 255.255.255.0"

ifconfig_carp0="vhid 1 pass testpass 192.168.1.50/24"</programlisting>

ifconfig_carp0="vhid 1 pass testpass <systemitem class="ipaddress">192.168.1.50</systemitem>/24"</programlisting>

      <para>On <systemitem>hostb.example.org</systemitem>, use the following

      <para>On <systemitem>hostb.example.org</systemitem>:</para>

	lines:</para>

ifconfig_fxp0="inet 192.168.1.4 netmask 255.255.255.0"

ifconfig_fxp0="inet <systemitem class="ipaddress">192.168.1.4</systemitem> netmask 255.255.255.0"

ifconfig_carp0="vhid 2 pass testpass 192.168.1.51/24"</programlisting>

ifconfig_carp0="vhid 2 pass testpass <systemitem class="ipaddress">192.168.1.51</systemitem>/24"</programlisting>

	<para>It is very important that the passwords, specified by

	<para>The passwords specified with &man.ifconfig.8;

	  the <option>pass</option> option to &man.ifconfig.8;, are

	  <option>pass</option> must be identical.

	  identical.  The <filename>carp</filename> devices will

	  <acronym>CARP</acronym> will only listen to and accept

	  only listen to and accept advertisements from machines

	  advertisements from machines with the correct password.  The

	  with the correct password.  The <acronym>VHID</acronym>

	  <acronym>VHID</acronym> must also be unique for each virtual

	  must also be unique for each machine.</para>

	  <acronym>IP</acronym> address.</para>

      <para>The third machine, <systemitem>provider.example.org</systemitem>,

      <para>The third machine,

	should be prepared so that it may handle failover from either

	<systemitem>hostc.example.org</systemitem>,

	host.  This machine will require two

	needs to be prepared so that it can handle failover from

	<filename>carp</filename> devices, one to handle each

	either of the previous host.  This machine is configured

	host.  The appropriate <filename>/etc/rc.conf</filename>

	with two <acronym>CARP</acronym> devicess, one

	configuration lines will be similar to the following:</para>

	to handle the virtual <acronym>IP</acronym> address of each

	of the master hosts.  Setting the <option>advskew</option>

	controls the <acronym>CARP</acronym> advertising skew.  The

	skew ensuring that the backup hosts advertises later than the

	master, and controls the order of precedence when there

	are multiple backup servers.  Set the configuration in

	<filename>/etc/rc.conf</filename>:</para>

      <programlisting>hostname="provider.example.org"

      <programlisting>hostname="hostc.example.org"

ifconfig_fxp0="inet 192.168.1.5 netmask 255.255.255.0"

ifconfig_fxp0="inet <systemitem class="ipaddress">192.168.1.5</systemitem> netmask 255.255.255.0"

ifconfig_carp0="vhid 1 advskew 100 pass testpass 192.168.1.50/24"

ifconfig_carp0="vhid 1 advskew 100 pass testpass <systemitem class="ipaddress">192.168.1.50</systemitem>/24"

ifconfig_carp1="vhid 2 advskew 100 pass testpass 192.168.1.51/24"</programlisting>

ifconfig_carp1="vhid 2 advskew 100 pass testpass <systemitem class="ipaddress">192.168.1.51</systemitem>/24"</programlisting>

      <para>Having the two <filename>carp</filename> devices will

      <para>Having the two <acronym>CARP</acronym> devicess configured

	allow <systemitem>provider.example.org</systemitem> to notice and pick

	means that <systemitem>hostc.example.org</systemitem> will

	up the <acronym>IP</acronym> address of either machine, should

	notice if either of the master servers becomes unavailable.

	it stop responding.</para>

	If a master fails to advertise before the backup server, the

	backup server will pick up the shared <acronym>IP</acronym>

	address until the master becomes available again.</para>

	<para>The default &os; kernel <emphasis>may</emphasis> have

	<para>Preemption is disabled in the GENERIC &os; kernel.

	  preemption enabled.  If so,

	  If Preemption has been enabled with a custom kernel,

	  <systemitem>provider.example.org</systemitem> may not relinquish the

	  <systemitem>hostc.example.org</systemitem> may not release

	  <acronym>IP</acronym> address back to the original content

	  the <acronym>IP</acronym> address back to the original

	  server.  In this case, an administrator may have to manually

	  content server.  The administrator can force the backup

	  force the <acronym>IP</acronym> back to the master.  The

	  server to return the <acronym>IP</acronym> address to the

	  following command should be issued on

	  master with the command:</para>

	  <systemitem>provider.example.org</systemitem>:</para>

	<screen>&prompt.root; <userinput>ifconfig carp0 down &amp;&amp; ifconfig carp0 up</userinput></screen>

	<screen>&prompt.root; <command>ifconfig carp0 down &amp;&amp; ifconfig carp0 up</command></screen>

      <para>At this point, <acronym>CARP</acronym> should be enabled

      <para>At this point, either networking must be restarted or the

	and available for testing.  For testing, either networking

	machine rebooted, then <acronym>CARP</acronym> is

	has to be restarted or the machines rebooted.</para>

	enabled.</para>