← Tolerance · exhibit 06 of 06
06

Drop It on Purpose

TCP congestion control · bufferbloat · CoDel (Nichols & Jacobson, 2012) · fq_codel (RFC 8290)

The internet delivers your bytes reliably over links that lose packets, and TCP depends on those losses. TCP has no idea how fast the path is, so it speeds up until a packet gets dropped, reads the drop as "too fast," backs off, and repeats. Van Jacobson added this in 1988 after the early internet went into congestion collapse.

Then memory got cheap. Routers and cable modems got huge buffers, so they could avoid dropping anything until a full second of traffic was queued. TCP never gets its signal, so it fills the buffer, and every packet behind it waits in line, including your video call. A router that refuses to drop packets is hiding the one signal TCP relies on.

ping now
—
video call
—
link used
—
packets dropped
—

Try this

  1. Leave it on plain FIFO with one upload. Ping starts near 30 ms, then climbs past a full second as TCP fills the buffer, and it stays there. Your link is 100% used and useless for anything interactive.
  2. Shrink the router buffer. Latency drops and throughput barely changes. The extra buffer wasn't adding speed, only delay.
  3. Switch to CoDel. It watches how long packets have been waiting, and once that stays above 5 ms for 100 ms it starts dropping, gently at first and then more often. TCP gets its signal early and the queue stays short.
  4. Switch to fq_codel and add uploads. Each flow gets its own queue, and sparse flows like your ping skip past the bulk. The "Smart Queues" checkbox on some home routers is this algorithm.

The deep cut

In October 1986 the link between Lawrence Berkeley Lab and UC Berkeley, 400 yards and a few router hops apart, dropped from 32 kbit/s to 40 bit/s. Every sender kept retransmitting into a network that was already drowning. Jacobson's fix, which treats loss as a congestion signal and backs off multiplicatively, is why the internet grew past that point.

Bufferbloat is that fix breaking from the opposite direction. Jim Gettys named it in 2010 while trying to work out why his home connection became unusable whenever he uploaded a file. The cure wasn't more bandwidth. It was to drop packets deliberately, before anyone has to wait.