Cold Showers

> Benchmarking cutting-edge graph-processing algorithms running on 128-core clusters against a single-threaded 2014 Macbook Pro. The laptop consistently wins, sometimes by an order of magnitude. LOL, this hits close to home. My company had a modeling specific VM set up to run our predictive modeling pipelines. Typical pipeline is about 50,000 to 5 million rows of training data. At best, using an expensive VM, we managed to get 2x training speed from lightgbm on the VM vs my personal work…

I love that sqlite article. It seems like "everyone" is certain that sqlite can only be used for up to a single query per second, anything more and you need to spin up a triple sharded postgres or Hadoop cluster because it 'needs to scale'. I love being able to show that study, if you properly architect your sqlite system and am willing to purchase hardware, you can go a long long way, much further than almost all companies go…

I am very skeptical of research about software development. Whether it's about tooling (static typing, etc.) or process (pair programming, etc.). It's really hard (I would go so far as to say impossible) to set up useful metrics and it's really hard to create comparable scenarios. And even if you had those, very few things apply generally to all fields of software, let alone the different types of personalities that developers have, even if the stereotypes have a bit of…

(from the last "shower"): > I'm guessing their servers are 100k each? No basis for that guess though. Seems strange to make the guess without basis then says so.. except that it's suspiciously in the right OOM. Perhaps it was unconsciously based on that "shower" article providing a cost estimate for AWS, as well as an estimate of the premium over hardware. https://blog.expensify.com/2018/01/08/scaling-sqlite-to-4m-q... https://news.ycombinator.com/item?id=16118776