Myths Programmers Believe about CPU Caches (2018)

If cache coherence is relevant to you, I strongly recommend the book “A Primer on Memory Consistency and Cache Coherence”. It’s much easier to understand the details of coherency from a broader perspective, than an incremental read-a-bunch-of-blogs perspective. I found that book very readable, and it cleared up most misconceptions I had. It also teaches a universal vocabulary for discussing coherency/consistency, which is useful for conveying the nuances of the topic. Cache coherence is not super relevant to most…

This article gives the impression that everything is the compiler's fault when you end up with conflicting reads in different cores, but that's not right. From the point of view of someone outside the CPU, yes you can say that simultaneous reads might never give different answers. But everything happening at that level barely resembles the original software. Dozens of instructions are happening at any moment, overlapping each other, starting and finishing in very different orders from how they're stored…

Here's my favorite practically applicable cache-related fact: even on x86 on recent server CPUs, cache-coherency protocols may be operating at a different granularity than the cache line size. A typical case with new Intel server CPUs is operating at the granularity of 2 consecutive cache lines. Some thread-pool implementations like CrossBeam in Rust and my ForkUnion in Rust and C++, explicitly document that and align objects to 128 bytes [1]: /** * @brief Defines variable alignment to avoid false sharing…

A lot of these 'misconceptions' are absolutely true though, for certain architectures. Every architecture has a specific set of guarantees about memory ordering and the coherence protocol used is an implementation detail relevant only to performance. The article is about x86 but i.e. ARM has much weaker guarantees about memory ordering. Relying on the behavior of a specific architecture here is not a good idea. This Linux document about memory barriers[1] repeatedly calls out problems specific to the Alpha…