2007-05-13 19:42:52

Netfilter clusterip: reinventing the wheel

One of the lectures at the 2007 edition of ChaosDocks was about the Netfilter extension «clusterip». This extension aims at providing an interface to Netfilter which allows for router redundancy.

Router reduncancy per se involves two major tasks. The first task is, of course, to listen to a common IP. Under OpenBSD, the protocol carp (Common Address Redundancy Protocol) is used for this. It is a versatile protocol which allows connections to be established and related to multiple hosts which share the same IP. The IETF standard protocol VRRP, a standardized variant of the Cisco protocol HSRP (Hot Standby Router Protocol), also specifies this functionality via a master/slave system inside its protocol.

The solution of the Netfilter people however was not to port the Carp protocol to Linux, but instead they reinvented address reduncancy in a rather uncommon way. First of all, all of the routers need an alias interface to the physical interface which will be connected to the corresponding network. This is to ensure that they all receive the entire traffic. It is thereby the duty of the network administrator to ensure that the packets arrive at both hosts.

For this purpose, Netfilter clusterip defines a new multicast protocol which is used to negociate between the nodes who is supposed to take which connections. This protocol then hashes the source and target IPs to assure that the connection sticks to the same router.

The other task is to ensure that the rules and state tables are equal on all redundant routers. In VRRP, the master controller regularly copies his rules and state tables to all of the slave nodes. OpenBSD defines a second primitive protocol to solve this task separately. This protocol is called pfsync.

pfsync itself is a cryptographically authenticated protocol. Only nodes which can authenticate themselves in a challenge-response protocol are allowed to push rules and states. While the carp protocol assures that only one router processes the arriving packets, the pfsync protocol takes care of the connection states (for e.g. NAT) so the packets don't get rejected in case the connection ends up on a router it has not been initiated on.

Thus, pfsync permits full load balancing and redundancy as well as error recovery in case one of the routers goes offline. It does not have any dependency on a special master server, and is cryptographically secure (to our current knowledge).

Nevertheless, the Netfilter architects chose to create an addition to clusterip which takes care of the states. It is part of clusterip even though its purpose is not related to address redundancy in any way. Also, it is incompatible with any of the three preexisting protocols, HSRP, VRRP and CARP/pfsync.

Inquired about his reasons to create his own protocol rather than to port existing protocols to Linux, the clusterip author responded that it would probably involve the same effort. Challenged about the interoperability of his solution, he answered that he didn't look at the existing solutions but had the feeling beforehand that they would not suit his requirements.

He wasn't sure about it though.

Posted by Tonnerre Lombard | Permanent link | File under: chaos, standards