TCP Fast Open

tcp fast open

What Is TCP Fast Open?

TCP Fast Open (TFO), is an extension to the transmission control protocol (TCP) that helps reduce network latency by enabling data to be exchanged during the sender’s initial TCP SYN. A traditional TCP handshake is a 3 step process that is carried out as follows.

  1. The sender sends a SYN packet to the receiver to initiate the connection
  2. The receiver sends a SYN-ACK packet back to the sender to let it know that it is ready to start transmitting data
  3. The sender sends an ACK packet to the receiver

 tcp handshake

Once this process is complete, the sender and receiver can both start exchanging data. However, performing these 3 steps in turn increases network latency time which therefore decreases overall page load speed. TCP Fast Open helps reduce the additional network latency time this handshake incurs by having the client send data during the initial SYN, thus allowing connections to take place during the handshake.

How Does It Work?

The TCP fast open extension works by sending data to the receiver upon the initial SYN from the sender. This allows for data transfer to begin immediately instead of waiting for the entire handshake process to take place. However, TFO only works once a normal TCP handshake has taken place as the sender not only sends a SYN packet to the receiver but also sends a cookie request.

Therefore the process which takes place during the first TCP fast open request between a sender and receiver is as follows:

  1. The sender sends a SYN packet along with a cookie request
  2. The receiver generates the requested cookie
  3. The receiver sends a SYN-ACK along with the cookie back to the sender
  4. The client caches the cookie for the particular server’s IP

Now that the client has cached the cookie generated by the server, subsequent sender and receiver handshake communication will look like this:

  1. The sender sends a packet containing a SYN, TFO cookie, and data
  2. The receiver validates the TFO cookie and data is made available to the application
  3. The receiver sends back a SYN-ACK and continues to send more data packets to the client
  4. The client sends an ACK back to the receiver

Once this process is complete, the TCP continues on normally. By using the TFO extension, the client is able to start sending data immediately and communication can begin earlier.

TCP Fast Open Nginx Configuration

In order to use the TCP fast open extension with Nginx, your kernel settings must be configured to support TFO. As of Linux version 3.7, TFO support was included for clients and servers and as of version 3.13, it should be enabled by default (check your version with uname -r). Use the following command to see if TCP fast open is running on your server.

cat /proc/sys/net/ipv4/tcp_fastopen

If this returns a value of 0, it is disabled. Additionally, if it returns a value of 1, this means TFO is only enabled on outgoing connections (client) and a value of 2 indicates it is only available on listening sockets (server). However, you’ll want your tcp_fastopen set to 3, which enables both.

If the tcp_fastopen value is not equal to 3, you can modify this by running the following command.

echo "3" > /proc/sys/net/ipv4/tcp_fastopen

This will modify the tcp_fastopen value until the system reboots. However, if you would like to avoid having to change the value each time the system reboots, you can create a tcp_fastopen.conf file within the /etc/sysctl.d directory as follows:

echo "net.ipv4.tcp_fastopen=3" > /etc/sysctl.d/30-tcp_fastopen.conf

We now have a tcp_fastopen configuration file which specifies a value of 3 stored in the sysctl.d directory. We can verify the contents of this file by running cat /etc/sysctl.d/30-tcp_fastopen.conf

As for Nginx TFO support, most Nginx packages do not include this by default. However, it can be specified if you build Nginx from source by adding the -DTCP_FASTOPEN=23 compiler flag to NGINX’s configure script. Once your Nginx server is properly configured and supports TFO, adding the TCP fast open option is as simple as opening up your nginx.conf file and adding it to listen directive.

listen 80 fastopen=256

TCP Fast Open – In Summary

As previously mentioned, using the TCP fast open extension can help further reduce network latency by allowing data to be sent earlier. This is especially useful for website visitors who are a great distance away from the origin server therefore increasing round trip times. Although there will likely be some configuration required at the server level, this may prove to be beneficial if you are experiencing network latency issues or need another method to help improve page load times for returning users.

If you’re interested in learning more about TCP fast open, take a read through the specification.

  • orrd

    This article mentions that “/proc/sys/net/ipv4/tcp_fastopen” should be 3, but it doesn’t ever explain what to do if it isn’t (and it won’t be for most servers). There are other articles that do a better job of explaining how to enable it.

    • Thanks for the feedback. We’ve just updated the article to better explain how to modify the tcp_fastopen value to 3 if it isn’t already.