The backlog has an effect on the maximum rate at which a server can accept new TCP connections on a socket. The rate is a function of both the backlog value and the time that connections stay on the queue of partially open connections.
When a TCP connection is being set-up, the client and server perform a three-way handshake to ensure the connection is a valid one. The connection is established when sequence numbers have been exchanged and synchronized in both directions. The sequence numbers are used to provide reliable transmission and flow control (i.e., ensuring either side does not transmit too fast for the other).
As illustrated in Figure 3, the connection is initiated by a listening socket receiving a segment containing a synchronize control flag (SYN) and a sequence number. The server side acknowledges this and sends its own sequence number (SYN|ACK). The client then acknowledges this sequence number (ACK). The connection is established once the server process receives the acknowledgement and performs an accept() system call.
The duration that affects the rate at which new connections are accepted is the time spent on the queue of pending incoming connections. This duration is equal to the round trip time for the SYN|ACK message and its ACK response plus the time taken for the client to process the SYN|ACK message plus the delay for the server to process the ACK and call accept().
The rate at which new connections can be accepted is equal to the number of entries which can fit on the listen queue divided by the average length of time each entry spends on the queue. Therefore, the larger the queue, the greater the rate at which new connection requests can be accepted.
Many systems (particularly BSD-derived or influenced) silently truncate this value (the backlog parameter to the listen() system call) to 5—version 1.2.13 of the Linux kernel does this in /usr/src/linux/net/inet/af_inet.c. Using small values for the listen backlog was one of the major causes of poor web server performance with many operating systems up until recently.
When dealing with Internet applications, you can probably assume that the rate at which new connections can be accepted is equal to the listen backlog divided by the round-trip time of the path between client and server.
The backlog parameter is silently truncated to SOMAXCONN in /usr/src/linux/net/ipv4/af_inet.c. SOMAXCONN is defined as 128 in /usr/src/linux/socket.h for 2.x kernels.