TCP Fast Open
What is TCP Fast Open?
TCP Fast Open (TFO), is an extension to the transmission control protocol (TCP) that helps reduce network latency by enabling data to be exchanged during the sender's initial TCP SYN. A traditional TCP handshake is a three step process that is carried out as follows.
- The sender sends a SYN packet to the receiver to initiate the connection
- The receiver sends a SYN-ACK packet back to the sender to let it know that it is ready to start transmitting data
- The sender sends an ACK packet to the receiver
Once this process is complete, the sender and receiver can both start exchanging data. However, performing these three steps in turn increases network latency time which therefore decreases overall page load speed. TCP Fast Open helps reduce the additional network latency time this handshake incurs by having the client send data during the initial SYN, thus allowing connections to take place during the handshake.
How does it work?
The TCP Fast Open extension works by sending data to the receiver upon the initial SYN from the sender. This allows for data transfer to begin immediately instead of waiting for the entire handshake process to take place. However, TFO only works once a normal TCP handshake has taken place as the sender not only sends a SYN packet to the receiver but also sends a cookie request.
Therefore the process which takes place during the first TCP Fast Open request between a sender and receiver is as follows:
- The sender sends a SYN packet along with a cookie request
- The receiver generates the requested cookie
- The receiver sends a SYN-ACK along with the cookie back to the sender
- The client caches the cookie for the particular server's IP
Now that the client has cached the cookie generated by the server, subsequent sender and receiver handshake communication will look like this:
- The sender sends a packet containing a SYN, TFO cookie, and data
- The receiver validates the TFO cookie and data is made available to the application
- The receiver sends back a SYN-ACK and continues to send more data packets to the client
- The client sends an ACK back to the receiver
Once this process is complete, the TCP continues on normally. By using the TFO extension, the client is able to start sending data immediately and communication can begin earlier.
TCP Fast Open Nginx configuration
In order to use the TCP Fast Open extension with Nginx, your kernel settings must be configured to support TFO. As of Linux version 3.7, TFO support was included for clients and servers and as of version 3.13, it should be enabled by default (check your version with
uname -r). Use the following command to see if TCP Fast Open is running on your server.
If this returns a value of
0, it is disabled. Additionally, if it returns a value of
1, this means TFO is only enabled on outgoing connections (client), and a value of
2 indicates it is only available on listening sockets (server). However, you'll want your
tcp_fastopen set to
3, which enables both.
tcp_fastopen value is not equal to
3, you can modify this by running the following command.
echo "3" > /proc/sys/net/ipv4/tcp_fastopen
This will modify the
tcp_fastopen value until the system reboots. However, if you would like to avoid having to change the value each time the system reboots, you can create a
tcp_fastopen.conf file within the
/etc/sysctl.d directory as follows:
echo "net.ipv4.tcp_fastopen=3" > /etc/sysctl.d/30-tcp_fastopen.conf
We now have a
tcp_fastopen configuration file which specifies a value of
3 stored in the
sysctl.d directory. We can verify the contents of this file by running
As for Nginx TFO support, most Nginx packages do not include this by default. However, it can be specified if you build Nginx from source by adding the
-DTCP_FASTOPEN=23 compiler flag to NGINX's configure script. Once your Nginx server is properly configured and supports TFO, adding the TCP Fast Open option is as simple as opening up your nginx.conf file and adding it to listen directive.
listen 80 fastopen=256
As previously mentioned, using the TCP Fast Open extension can help further reduce network latency by allowing data to be sent earlier. This is especially useful for website visitors who are a great distance away from the origin server therefore increasing round trip times. Although there will likely be some configuration required at the server level, this may prove to be beneficial if you are experiencing network latency issues or need another method to help improve page load times for returning users.
If you're interested in learning more about TCP Fast Open, take a read through the specification.