Whom the Gods Would Destroy, They First Give Real-Time Analytics (2013)

> The turnaround time also imposes a welcome pressure on experimental design. People are more likely to think carefully about how their controls work and how they set up their measurements when there's no promise of immediate feedback. This seems like a cranky rationalization of the lack of a fairly ordinary system. Sure, you shouldn't draw conclusions for potentially small effects on < 24 hours of data. But of course if you've done any real world AB testing, much less…

Full-time web analyst here. Total agreement. Information is as useful as your ability to act on it--no more, no less. Real-time analytics is something that sounds sexy and gets a lot of headlines (and probably sales), but it's not particularly useful, especially compared to the cost to implement. Most organizations aren't capable of executing quick decisions of any significance. In fact, quite a few business models wouldn't have much to gain even if they were capable of it. My experience…

Author here. The main thing that inspired this happened a few years before I wrote it down. Etsy had gotten a new CEO, and they spent one of their first few weeks in long hours at my desk, iterating on the homepage design in what could only be described as a radically fast iteration loop. We'd ship a tweak, look at statsd for ten minutes, then change something else. This would have been a bad idea for all of the…