"You thought connect() was fast? TCP_FASTOPEN says no handshake, no waiting – send the payload with the SYN."

TL;DR;

  • TCP_FASTOPEN lets you send data in the TCP SYN packet — before the handshake completes.
  • This skips 1 full round-trip (RTT) — turning a 3-way handshake into a 2-way data-carrying handshake.
  • It works by: 1) Client: sends data in SYN. 2) Server: processes the data before the connection is fully established.
  • It's defined in RFC 7413, and implemented in Linux since kernel 3.7 (client), 3.13 (server).
  • Huge win for web servers, APIs, HTTP/1.1 over TLS, and any latency-sensitive protocol.

History: Why TCP Was Too Slow for the 21st Century

TCP is old. Like, dial-up and BBS-era old. And it's cautious:

Client: ---> [SYN]
Server: <--- [SYN-ACK]
Client: ---> [ACK]
Client: ---> [GET /index.html]

That's 2 RTTs just to say hello. Then you send the actual request. Painful in:

  • Mobile networks (100ms+ RTT)
  • Global cloud hops (200ms+ RTT)
  • High-frequency trading (where 1 RTT = lose)

So engineers asked:

"Can we send data inside the SYN?"

Yes – if you're willing to bend TCP until it screams.

Enter: TCP Fast Open (TFO).

How TCP Fast Open Works (In Theory)

1) Client sends SYN + data + optional TFO cookie
2) Server sees SYN + data
3) If server trusts the cookie:

  • Immediately processes the data
  • Sends SYN-ACK + potential response

4) Client receives SYN-ACK, completes the 3-way handshake
5) If the connection is accepted – the response is already waiting

Result:

Zero-RTT data transmission.

The client gets data back before the TCP handshake even finishes.

This is black magic. TCP is no longer request, then response. It's request inside SYN.

Why It's Not Always Easy

But this breaks everything.

TCP assumes:

  • SYN = connection attempt
  • Data comes after the connection is confirmed
  • Middleboxes, NATs, firewalls, and proxies will drop or mutilate SYN packets with payloads

So Linux has to:

  • Use per-host TCP Fast Open cookies
  • Maintain fallback logic
  • Handle partially established sockets
  • Re-transmit if the fast-open fails silently

Code: Using TCP_FASTOPEN (Client)

Step 1: Enable Fast Open

echo 1 | sudo tee /proc/sys/net/ipv4/tcp_fastopen

Set to:

1: Enable client
2: Enable server
3: Both

Step 2: Setup Client Socket

int sockfd = socket(AF_INET, SOCK_STREAM, 0);

// Send "Hello" along with the connect()
struct sockaddr_in servaddr = ...;
char msg[] = "HELLO!\n";

int ret = sendto(sockfd, msg, sizeof(msg), MSG_FASTOPEN,
                 (struct sockaddr*)&servaddr, sizeof(servaddr));

The sendto() with MSG_FASTOPEN:

  • Triggers the SYN
  • Attaches your data to it
  • Completes the handshake in background

Now you're a latency killer.

Code: Using TCP_FASTOPEN (Server)

Enable:

echo 3 | sudo tee /proc/sys/net/ipv4/tcp_fastopen

Listen with TCP_FASTOPEN:

int qlen = 5; // backlog
setsockopt(listener, IPPROTO_TCP, TCP_FASTOPEN, &qlen, sizeof(qlen));

Then accept as usual:

int conn = accept(listener, ...);
read(conn, buf, sizeof(buf));

The first read() gets data from the SYN if it was a TFO connection. Otherwise, fallback.

Deep Path Through Kernel

TFO touches the entire TCP stack. Key files:

  • net/ipv4/tcp_fastopen.c
  • tcp_v4_conn_request()
  • tcp_check_req()

Key flags:

  • MSG_FASTOPEN
  • TCP_FASTOPEN_NO_COOKIE
  • TCP_FASTOPEN_KEY

Key socket states:

  • TCP_SYN_RECV may receive data
  • req->fastopen_rsk = partially established socket holding payload

Kernel carefully tracks:

  • Has the 3-way handshake completed?
  • Is data safe to process?
  • Is the cookie valid?
  • Should we fallback?

Observe with tcpdump

Want to see it work?

sudo tcpdump -i lo 'tcp[tcpflags] & (tcp-syn) != 0' -X

Look for:

Flags [S], payload (len=X)

If the SYN packet has non-zero data length, you're using TCP Fast Open.

Also verify server sends SYN-ACK + payload.

Quantum Mode: TFO + TLS + QUIC

TFO is great.

But TLS 1.2 handshake kills its gains.

So next-gen stacks do:

  • TCP_FASTOPEN for the SYN
  • 0-RTT TLS 1.3 for app data (HTTP/2, gRPC)
  • Or ditch TCP entirely and use QUIC (UDP + TLS + congestion)

Still – for pre-TLS or optimized APIs, TFO gives the fastest TCP handshake in existence.

Risks and Security

Spoofing

TFO uses cookies to validate that the client owns its IP.

Otherwise, attackers could:

  • Send SYN + fake requests
  • Force the server to perform expensive operations
  • Never complete handshake (half-open DOS)

MITM

Data in SYN isn't encrypted unless your app layer handles it (e.g. TLS).

Silent Fallbacks

Middleboxes may:

  • Drop SYN+data
  • Mangle MSS/Window scaling
  • Break ECN

Linux silently falls back to classic connect() if TFO fails.

Performance

Without TFO:

RTT1: [SYN] → [SYN-ACK]
RTT2: [ACK + GET] → [HTTP 200]
= 2 RTTs

With TFO:

RTT1: [SYN + GET] → [SYN-ACK + HTTP 200]
= 1 RTT

Or with HTTP/1.0: you can send request and receive response before even ACKing the SYN-ACK.

Final Thoughts

TCP_FASTOPEN is the purest form of TCP latency wizardry:

  • It bypasses RTT costs.
  • It requires intimate knowledge of kernel and network stack behavior.
  • It bends the TCP handshake model to your will.
  • It's fully POSIX-compliant, yet underused.

If you:

  • Control both client and server
  • Can tolerate some edge-case messiness
  • Need to own latency

Then use it.

Otherwise, QUIC is your future.

Further Reading

  • RFC 7413: TCP Fast Open
  • Linux Kernel: tcp_input.c, tcp_fastopen.c
  • Google's early Fast Open patches
  • Chromium's TFO client
  • Apple's TCP Fast Open for Siri and iOS push
  • Wireshark: TCP Fast Open filter = tcp.options.fastopen

Closing

Want me to go deeper?

Next up:

  • Implement TFO + TLS 1.3 + QUIC fallback
  • Analyze dropped SYN+data packets with iptables and Netfilter
  • Rebuild your RPC stack for 0-RTT APIs

Just say the word.