Why Your Software Estimates Suck

Software estimates suck. Developers hate giving them, product managers hate being held accountable for them, business development teams hate explaining them, and, well, designers, a lot of time are unfortunately just along for the ride.

Estimates suck because they can’t be trusted and they suck because, guess what, there is always someone lurking in the background claiming that they can do it faster and cheaper.

Even worse, there are a number of absurd “app” calculators out there on the web that misinform clients by claiming they can accurately assess the cost of a project.

Why Your Estimates are Wrong

There is a lot written on why traditional software projects fail. And it seems like everyday there is something about how costs on some government project swelled to millions and sometimes billions of dollars.

But software estimates are inaccurate for a variety of reasons, and these issues are not unique to the government space.

More importantly, as we have moved away from traditional desktop/web software with long release cycles and toward more nimble mobile applications, the water has become murkier. Traditional businesses must adapt archaic organizational practices to a world in which annual or quarterly release cycles will not be the norm.

When I see software projects fail, nine times out of ten it’s usually because they were doomed from the start. From the start. Client expectations aren’t set properly, risk mitigation is ignored, the list goes on and on. But I classify the inaccurate estimates into three major categories:

Human Issues
Variability Issues
Universal Truth Issues

For the purposes of this post, I am primarily talking about engineering estimates out of a sales process. Design estimates and more operationally focused “point planning poker” are valuable topics, but there is a lot written about them already.

This post also assumes you are Agile (not Waterfall) and assumes that you aren’t producing 150-page app documentation. If you are, chances are its inaccurate and you just wasted a lot of time upfront thinking you mitigated risk when you really didn’t (Trust me, I know. We used to do this and stopped for many reasons!).

Human Issues

Humans are unreliable and unpredictable, and when it comes to humans, we have to protect ourselves against, well ourselves. So here are some of our major drawbacks:

Time: Time is never our friend. Having the proper amount of time to do the estimate and lightly iterate with the client is crucial. There are a lot of key conversations that materialize during the sales process that can affect a software estimate. Something as simple as “social integration” can mean many different things to different stakeholders. Identifying these things up front is key.
Optimism: It’s human nature to be optimistic. Think about a time when someone has asked you how long the laundry will take? Or how long it will take you to prepare dinner? How about if someone asks you how long it will take you to run a mile, without telling you if there are hills or it’s flat road. Now multiply that inaccuracy by ten and you get a software developer’s estimate. Developers often think they can do something faster than they really can.
Estimating for someone else: I can’t tell you the amount of times I have seen an engineering manager estimate for someone else. Heck, I have been guilty of it too. But it’s a big no-no. Engineering managers are looked to as an authority and the person who “knows how long something is supposed to take,” but they have no framework to evaluate how their teams perform holistically and what they are capable of.
The bootcamp developer: Don’t get me wrong, I love the General Assembly and Code Academy’s of the world. With the growing need for mobile developers and engineers skyrocketing an interesting dynamic has formed. The difference between the traditionally educated software engineer vs. the career-changing engineer is an interesting one, and one that would require its own blog post to explore fully. All I am pointing out is the skillset of your staff, particularly engineers, can vary greatly, so prepare yourself with a hiring process that works for you.

Variability Issues

Change and variability is an estimate’s version of the “Grim Reaper”. Development of a mobile Beta or MVP can take anywhere from three to twelvemonths from inception through execution. That’s a big window for change across the board.

Market Changes: The market can change. Competitors will enter and exit the market. A differentiating feature today is a household name tomorrow.
Teams Change: People don’t stay in jobs for 30 years like they used to. People will leave and turnover will happen.
Clients Change: There are a lot of influences on your client. ^Third party stakeholders, investors, and more, each of whom may have input that could change product direction on the stop of a dime. Anything from aesthetics (UI and UX) to pieces of major functionality are fair game.
Technology Changes: This is key. Technologies are changing at an exponential pace. This isn’t like the Windows of yesteryear when OS’s were pushed every few years. Major versions of mobile operating systems are updated annually (not to mention the slew of minor releases) and can include a number of SDK changes as well as complete Visual and UX overhauls (see iOS 7 and Android’s Lollipop adoption of Material Design). On top of that, there is a good chance that the SDK is not the only thing that your application relies on. It probably integrates with one or more cloud services, and cloud services come, go and change like the wind. Don’t bet on your APIs being stable and consistent. Have a risk mitigation plan.

Universal Truth Issues

This category is more or less everything that is not encompassed in the above. There are issues that always spring up that no one actually thinks about or considers, but I guarantee they will happen.

Not Data Driven: Estimates are rarely backed by real world data.How have similar past projects performed? Data around your teams and their speed and accuracy are usually absent and/or not considered.
Absent Risk Analysis: Proper risk analysis is never performed. What are some of the personality traits of your clients that may pose a risk? What are the third party libs that you are going to be integrating the product into? Having an external and internal 360-degree view of risk is important.
QA Who?: The underappreciated black sheep of the process. QA time varies greatly depending on the type of app you are building. Streaming audio/video apps take a different amount of time compared to location-based apps or social apps. What portions of the code and functionality warrant automated tests? Unit Tests? Integration tests? Do you need more than one person testing two different user experiences that interact at the same time? Plan for QA.
“You don’t know what you don’t know”: The fact of the matter is what you are building, most likely, is new and portions of it are uncharted territory (as described in the famous Quora post regarding estimates). There’s no API or SDK repository to dip into, third party libs don’t exist and your colleague probably hasn’t seen it before either. Chances are you aren’t cranking out templated software that runs on an OS that stands the test of time.

You Can Do Better

So those are the main reasons why most software estimates suck. But I have good news: they don’t have to.

Your team can do better. Or if you’re a client looking to hire a developer who can deliver an accurate estimate, you can find someone who will do it right. Next time, I’ll offer point-by-point tips on how we’ve used data and better development processes to address these issues and create more accurate estimates that will keep clients and your team happy.

Filed under: Product Engineering | Topics: app developer, apps, software development