Latency, Throughput, and the Numbers Every Quant Dev Knows
3 min read
Trading systems are judged in microseconds, and systems interviews at trading firms test whether your intuition for time is calibrated the way your Big-O intuition is calibrated for operations. This lesson is the number sense plus the three concepts that organize it.
The latency ladder
Approximate, and worth knowing to the order of magnitude:
| Operation | Time |
|---|---|
| L1 cache reference | ~1 ns |
| Main memory (RAM) reference | ~100 ns |
| Send 1KB over 10Gbps network | ~1 μs |
| SSD random read | ~100 μs |
| Round trip within a datacenter | ~500 μs |
| Spinning-disk seek | ~10 ms |
| Round trip NY ↔ Chicago (fiber) | ~13 ms |
| Round trip NY ↔ London | ~70 ms |
The ratios are the lesson: RAM is ~100× cache, network round trips are ~1,000× memory references, and geography is physics — light in fiber covers ~200 km/ms, which is why microwave towers (straighter, faster-than-fiber paths) were worth hundreds of millions to HFT firms, and why the Chicago–NY arbitrage has a ~4 ms speed-of-light floor no software can beat. Quoting that floor when asked "how fast could this possibly be?" is exactly the calibration being tested.
Two derived habits: batch across the slow boundaries (one network call fetching 100 items beats 100 calls by ~100×; same logic as vectorization in the Python lesson), and know where your data lives — an algorithm that fits L2 cache can beat an asymptotically better one that thrashes RAM, the standard caveat to pure Big-O reasoning.
Throughput vs latency, and the queue between them
Latency is one item's travel time; throughput is items per second — and they're coupled through queueing. The core facts:
- Little's law: items in system = arrival rate × time in system (). Model-free and constantly useful: a matching engine holding 500 in-flight orders at 100k orders/sec implies 5 ms of queueing — no further detail needed.
- Utilization cliffs: as a server approaches 100% utilization, queue length (and thus latency) explodes nonlinearly — waiting time scales like . Systems run at 60–80% for a reason, and "we'll just run it hotter" is the wrong answer to a capacity question.
- Backpressure: when a consumer falls behind, something must give — buffer (latency grows), drop (data loss), or push back on the producer. Market-data systems in fast markets face exactly this; naming the three options and choosing per use case (drop stale quotes, never drop orders) is a complete answer.
Concurrency, minimally
The interview-relevant core: a race condition is two threads touching shared state where outcome depends on timing; locks fix it and introduce contention and deadlock risk (avoid via consistent lock ordering). The trading-flavored twist: many low-latency systems sidestep locking entirely — one thread per instrument, single-writer queues between stages — because at microsecond scale, not sharing beats synchronizing. Knowing that design instinct exists matters more than pthread trivia.
The interview version
"Roughly how long for an order to go from your strategy to the exchange and back, same city?" — Serialization ~μs, a couple of hops, exchange processing tens of μs to ms: order of hundreds of microseconds to ~1 ms; cross-country, add the ~13 ms physics. "Your feed handler falls behind at market open — diagnose." — Burst arrival rate vs throughput, queue growth via Little's law, then the backpressure menu. Numbers first, structure second, and no answer that violates the speed of light.