Latency numbers every programmer should know

Most of these latencies were measured and written up for a bunch of systems by Carl Staelin and I back in the 1990's. There is a usenix paper that describes how it was done and the benchmarks are open source, you can apt-get them. http://www.bitmover.com/lmbench/lmbench-usenix.pdf If you look at the memory latency results carefully, you can easily read off L1, L2, L3, main memory, memory + TLB miss latencies. If you look at them harder, you can read off cache…

The % of programmers that actually need to know any of these numbers is tiny. I've done plenty of optimizations for performance-critical systems and have never known any of these numbers. Certainly, I know general principles like "disk is a lot slower than memory" and "don't block the UI on a network call". But knowing whether an L1 cache read is 0.5ns or 5ns has never been necessary. You can optimize so much "highly optimized" code out there without those…

Relevant: Grace Hopper explains Nanoseconds [1] https://www.youtube.com/watch?v=JEpsKnWZrJ8

Now includes multi-second llm latencies