"Turning off TCP_NODELAY is like putting your socket on NyQuil. Turning it on? That's raw caffeine and no sleep."
– A socket hacker in 2003

In this post, we're ripping apart the mythos, internals, and performance characteristics of TCP_NODELAY. This isn't just another intro to Nagle's algorithm – this is the atomic dissection of it, with timing diagrams, kernel traces, real-world gotchas, and insane optimizations.

If you've ever built a multiplayer game, low-latency financial feed, or wrote your own RPC framework, you've probably used TCP_NODELAY and maybe wondered why sometimes it makes things worse.

Let's fix that.

TL;DR;

  • TCP_NODELAY disables Nagle's algorithm.
  • Nagle tries to batch small writes into full TCP segments.
  • With TCP_NODELAY, data is sent immediately, even for 1-byte write().
  • Good for low-latency, chatty protocols (e.g., telnet, games, trading).
  • Can wreck performance if misused – results in tiny packets called "tinygrams".

The Historical Problem: Too Many Tiny Packets

In the early '80s, networks were slow and expensive. People were using telnet, which sends a keystroke per packet.

This led to:

  • Massive packet floods.
  • 1-byte payloads in 40-byte TCP/IP headers.
  • Network congestion.

Enter: Nagle's Algorithm (RFC 896, 1984)

John Nagle proposed:

"Only send a new segment if all previous data is ACKed, or if you can fill a full MSS."

That is:

  • Batch writes.
  • Wait for ACKs before sending small payloads.
  • Great for bandwidth efficiency. Terrible for latency.
  • This is Nagle's algorithm – the default on all TCP sockets.

The Problem with Nagle

Nagle assumes this pattern:

write(sockfd, buffer, big_size);  // infrequent big writes

But modern apps often do this:

write(sockfd, "MSG_TYPE:", 9);
write(sockfd, &header, sizeof(header));
write(sockfd, &payload, len);

You send 3 small chunks – each under the MSS (~1460 bytes) – and Nagle will buffer them until:

  • The previous data is ACKed.
  • Or the accumulated data fills the MSS.

Result? Unexpected delays.

That's where TCP_NODELAY comes in.

TCP_NODELAY: Opting Out of Nagle

Defined in <netinet/tcp.h>:

#define TCP_NODELAY 1

This socket option disables Nagle, allowing immediate sends.

int flag = 1;
setsockopt(sockfd, IPPROTO_TCP, TCP_NODELAY, &flag, sizeof(flag));

Now every write() hits the wire as soon as the syscall returns.

You're bypassing kernel buffering logic and saying:

"I know what I'm doing. Let it fly."

Code Example: Without and With TCP_NODELAY

Without TCP_NODELAY

write(sockfd, "A", 1);
write(sockfd, "B", 1);
write(sockfd, "C", 1);

Nagle delays A, B, and C, buffering until ACK comes or MSS is filled.

With TCP_NODELAY

int flag = 1;
setsockopt(sockfd, IPPROTO_TCP, TCP_NODELAY, &flag, sizeof(flag));
write(sockfd, "A", 1);
write(sockfd, "B", 1);
write(sockfd, "C", 1);

Each character is sent as its own packet, immediately.

Observe with tcpdump:

sudo tcpdump -i lo -nn -v 'tcp port 9000'

Count the segments. See the magic (or the madness).

Game Use Case: Low Latency Beats Bandwidth

In a real-time game:

write(sockfd, &player_input, sizeof(input));

Delaying this 20ms to coalesce is unacceptable.

Solution:

setsockopt(sockfd, IPPROTO_TCP, TCP_NODELAY, &on, sizeof(on));

Boom. Player position updates hit the wire instantly.

Kernel Internals: tcp_nagle_check()

The kernel decides whether to send immediately or buffer:

static inline bool tcp_nagle_check(...) {
    if (tcp->nonagle & TCP_NAGLE_OFF) return true;
    ...
}

Enabling TCP_NODELAY sets tcp->nonagle = TCP_NAGLE_OFF.

Result: TCP stack bypasses nagling entirely.

Hard Mode: Delayed ACK + Nagle = Latency Hell

Here's the devil's combo:

  • You write tiny data.
  • Nagle buffers your send.
  • The peer delays its ACK.
  • You're stuck in TCP stalemate.

App waits for ACK → ACK waits for more data → Nagle waits for ACK → repeat.

This is the infamous Nagle + Delayed ACK deadlock.

Fix?

setsockopt(sockfd, IPPROTO_TCP, TCP_NODELAY, &on, sizeof(on));

Also optionally:

setsockopt(sockfd, IPPROTO_TCP, TCP_QUICKACK, &on, sizeof(on));  // Force peer ACKs fast

Now you're breaking the feedback loop.

TCP_NODELAY vs TCP_CORK vs TCP_QUICKACK

Let's line 'em up:

Option        What It Does                      Direction  Behavior
------        ------------                      ---------  --------
TCP_NODELAY   Sends data immediately            Outgoing   Disables Nagle
TCP_CORK      Delays send until uncork or full  Outgoing   Manual batching
TCP_QUICKACK  Sends ACKs immediately            Incoming   One-shot ACK flush

Real power: Combine them.

setsockopt(sockfd, IPPROTO_TCP, TCP_NODELAY, &on, sizeof(on));   // For low latency
setsockopt(sockfd, IPPROTO_TCP, TCP_QUICKACK, &on, sizeof(on));  // For fast ACKs

When NOT to Use TCP_NODELAY

  • High-throughput bulk transfers.
  • Large file downloads (e.g., via sendfile()).
  • Anything over slow, congested networks.

Why?

  • Tiny packets = more syscalls, headers, interrupts.
  • Overhead skyrockets.
  • Wastes bandwidth, power, and time.

Use TCP_CORK instead if you know you want batching.

Hyper-Advanced Trick: Adaptive Nagle Disabling

Create a user-space heuristic that toggles TCP_NODELAY dynamically:

if (write_size < 100 && time_since_last_write < 10ms) {
    setsockopt(sockfd, IPPROTO_TCP, TCP_NODELAY, &on, sizeof(on));
} else {
    setsockopt(sockfd, IPPROTO_TCP, TCP_NODELAY, &off, sizeof(off));
}

You're now implementing your own smart congestion signaling logic on top of TCP.

Security Angle: Fingerprinting TCP_NODELAY

Some applications can be fingerprinted by how their TCP stack behaves:

If every write() emits a segment → probably using TCP_NODELAY.

If packets arrive in coalesced batches → likely using Nagle.

Intrusion detection systems (IDS) can even infer application types from packet timing.

Enabling TCP_NODELAY leaks implementation detail into packet flows. Watch yourself.

Final Thoughts

TCP_NODELAY is more than just a latency knob – it's a statement that says:

"I understand my application's traffic patterns, and I take full responsibility for transmission behavior."

It's an escape hatch from TCP's automatic pacing, but with that freedom comes danger.

Used correctly, it makes your system feel telepathic in responsiveness.

Used carelessly, it burns bandwidth, CPU cycles, and makes the kernel cry.

Further Reading

  • RFC 896: Congestion Control in IP/TCP
  • Linux kernel source: tcp_output.c, tcp_timer.c
  • Wireshark filter: tcp.len == 1
  • See also: TCP_NOTSENT_LOWAT, SO_SNDLOWAT, TCP_CORK, TCP_QUICKACK

Closing

Want to get even more ridiculous?

Next time: we build a sendfile() proxy that uses TCP_CORK for coalescing, TCP_NODELAY for low-latency metadata, and TCP_QUICKACK to pulse ACKs like a custom transport protocol. Socket kung fu.

Just say the word.