Why software projects take longer than you think – a statistical model

An important aspect of being a professional software engineer is having the backbone to sometimes say things like: - “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.” - “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chinese and…

> A reasonable model for the “blowup factor” (actual time divided by estimated time) would be something like a log-normal distribution. Interestingly, we did extensive time tracking on a multi-year in-house software project and collected data comparing the estimated completion time of tickets with their actual time. The software department was under a lot of pressure to improve their forecasting, and were somewhat despairing that their estimates were off by a factor of about 1.6 on average, and sometimes a…

All this smokes and mirrors, when the reality is: if you gave accurate estimates, you'd get reprimanded, not get the project approved, etc. Same reason that infrastructure project go over budget. The way anything actually gets done is by initially being overly optimistic, ignoring potential future problems, getting the project approved, then lock it in via sunk costs, now as problems turn up, you can claim it could never had been foreseen, it's an impossibly hard problem bla bla. But…

"Adding up estimates rarely work when you end up with more than a few tasks. Instead, figure out which tasks have the highest uncertainty – those tasks are basically going to dominate the mean time to completion." -- Erik Bernhardsson explains why you should worry about risks and tackle them first, as they can have a drastic impact on your time estimations. A good lesson to take from it: teach software engineers how to do proper risk management, delay work…