Hyperfine: A command-line benchmarking tool

Perhaps interesting (for some) to note that hyperfine is from the same author as at least a few other "ne{w,xt} generation" command line tools (that could maybe be seen as part of "rewrite it in Rust", but I don't want to paint the author with a brush they disagree with!!): fd (find alternative; https://github.com/sharkdp/fd), bat ("supercharged version of the cat command"; https://github.com/sharkdp/bat), and hexyl (hex viewer; https://github.com/sharkdp/hexyl). (And certainly others I've missed!) Pointing this…

Nifty! Step up over just using time. Unfortunately, timing things in general isn't going to be a very effective benchmark. Without understanding what a program is doing, you don't understand what is impacting your results, and have no real knowledge on how things are going to differ when you go to use them in the "real world". Is one process faster when single threaded vs. a low core count, but another is massively parallel, and loses out until scaled higher?…

Most -- nearly all -- benchmarking tools like this work from a normality assumption, i.e. assume that results follow the normal distribution, or is close to it. Some do this on blind faith, others argue from the CLT that "with infinite samples, the mean is normally distributed, so surely it must be also with finite number of samples, at least a little?" In fact, performance numbers (latencies) often follow a heavy-tailed distribution. For these, you need a literal shitload of…

hyperfine is such a great tool that it's one of the first I reach for when doing any sort of benchmarking. I encourage anyone who's tried hyperfine and enjoyed it to also look at sharkdp's other utilities, they're all amazing in their own right with fd[1] being the one that perhaps get the most daily use for me and has totally replaced my use of find(1). [1]: https://github.com/sharkdp/fd