Everyone Should Know SIMD

Summary

SIMD has a reputation for being complex. I've met many very good software engineers who dismiss it as something too complex to learn or a niche optimization meant for only the highest-performance software, not useful in everyday programming. I think that's wrong. SIMD can be simple to understand1, and common "process N values at a time" SIMD code to speed up a naive for loop almost always follows the same general shape. Once you learn the basics, writing SIMD is just about as easy as a for loop. And when it's not, it's usually a good sign to skip it for now. Every developer should know at least that much SIMD. This post uses Zig for examples but is a general piece that applies to any programming language. Support for SIMD instructions varies by programming language and I hope that more programming languages expose these generic concepts in the future! I hate that I have to do this for every post now, but I also want to note this was completely hand-written with no AI assistance. Table of Contents Background: What Is SIMD? If you already know what SIMD is, skip this section. SIMD allows a CPU to operate on multiple values in parallel. For example, instead of comparing one byte at a time, a CPU can compare 4, 8, or even more bytes with a single instruction. If you ever see loops like this in your code: for (byte in bytes) { /* ... */ } for (character in string) { /* ... */ } for (value in array) { /* ... */ } There is an opportunity to use SIMD. SIMD turns those into this: for (8 byte chunk in bytes) { /* ... */ } This results in a localized speedup that directly maps to the parallelism: you process data 4x, 8x, or even faster. The only real requirement for this to pay off is that you need to be regularly processing a large enough number of bytes. If you're doing these for loops across data that is only ever a handful or dozens of bytes, it's not worth it. But if this is iterating over hundreds, thousands, millions of bytes, the payoff will be huge. That's the basics....