Stream Control Transmission Protocol (SCTP) Associations

where flags can be one of SCTP_BINDX_ADD_ADDR or SCTP_BINDX_REM_ADDR, and the second parameter is a packed list of IPv4 and IPv6 socket address structures.

It is relatively easy to use this on a server to allow clients to connect on any of the server's bound interfaces. But, how do you do this on a client? Where is the bind() operation? Well, just like TCP, this is hidden under the covers. In TCP, if a call to connect() is made and the socket is not yet bound (the usual case for a client), the TCP stack will choose one of the interfaces and an ephemeral port. You can make an explicit call to bind() that will let you choose the interface, but usually the port is left as zero, so an ephemeral port is still chosen.

You can do exactly the same with SCTP—don't call bind() and leave it to the SCTP stack. This will choose an ephemeral port like TCP, but instead of using a single interface, it will choose a set of interfaces (probably all that are available). So calling connect() without an initial bind() or sctp_bindx() will give you multihoming on the client side automatically.

If you call bind() with a specified interface before connect() in the client, you get only that single client-side interface, losing one of the advantages of SCTP! If you call bind() with the wildcard address INADDR_ANY, SCTP will choose a set of interfaces for you. So, SCTP will try to give you multihoming unless you pin it down to a single address using bind() or to a specific set of addresses using sctp_bindx().

With SCTP, I would expect a call to sctp_bindx() with all ports set to zero to choose the same ephemeral port for all addresses. Instead, the current Linux implementation (up to kernel 2.6.21) gets an ephemeral port for the first address and then throws an error, because the ports in the later addresses are still zero instead of this ephemeral value. The workaround is to call bind() with one address with port zero, see what the system set the port to, and then call bindx() on the remaining addresses with this new port number. Listing 1 (multi_homed_client.c) shows an example of this. This workaround probably will become unnecessary in the next specification of SCTP following discussion on the SCTP mailing list.

You can set the local interfaces to be used by sctp_bindx(). A client also can specify the subset of addresses that it wants to use to connect to the server by using the call sctp_connectx(), which takes a list of socket address structures just like sctp_bindx(). Why do this? Well, using connect() with a single address is a possible point of failure at the time that the initial connection is done. This is what the function sctp_connectx() solves. It allows the client to try multiple addresses in order to connect to the server.

The set of addresses in sctp_connectx() is used just to make the initial connection. But, after the connection is established, an interchange of information takes place between the two endpoints. In that exchange, the remote peer tells the local peer which addresses it actually wants to use and vice versa. The set of remote addresses that the remote peer will use need not be the same as what the client used in the connection. However, you at least can assume that one (but you don't know which one) of the addresses passed to sctp_connectx() will appear in the list that the remote peer offers, because the local client had to connect to something!

So, if the remote peer chooses the set of addresses it uses, how does the local client find which ones they are? This is done by another function, sctp_getpaddrs(), that gives the set of remote peer addresses. There is also an sctp_getladdrs() function, in case the local peer forgets which addresses it is using!

Once an association is set up between two endpoints, messages can be sent between them. Note that SCTP does not concern itself with QoS (Quality-of-Service) issues, such as real-time delivery, but only with reliability issues. SCTP uses the multihomed capabilities to try as many possible routes as possible to get messages through. So on the sending side, there is no control over which interfaces are used; indeed, the sender might even use a scheme such as round-robin among its interfaces for each message. However, the sending application can indicate to its SCTP stack which of the remote peer's interface it would prefer to use, and it can tell the remote peer on which interfaces it would prefer to receive messages. These are done by using the setsockopt() call with option type as SCTP_PRIMARY_ADDR or SCTP_SET_PEER_PRIMARY_ADDR. Of course, if these particular addresses are not available, SCTP simply will use different addresses in the association.

Once SCTP is told which interfaces to use, it basically looks after things itself. It uses heartbeats to keep track of which interfaces are alive, and it switches interfaces transparently when failure occurs. This is to satisfy the design goals of SCTP for improved reliability over TCP. Applications can give hints to the SCTP stack about which interfaces to use, but the stack will ignore these hints on failure.