Lies, Damn Lies, and Estimates

If you’ve ever been involved with a custom software development initiative, big or small, you’ve probably stumbled upon one of the central tenets of software engineering: estimates based on time are wrong. Always. Anyone who tells you that they can accurately estimate how long a
project will take with a high degree of precision is either naive, arrogant, and probably both.

How could a field seemingly based on logic and mathematics be so terrible at estimating its efforts?

Though this truth is counterintuitive, it shouldn’t cause fear. Software design and engineering has little in common with the fields to which it is often compared. Unlike construction, farming, or manufacturing, it’s not innately repeatable. No two projects are identical, or even very similar, regardless of the level of uniformity
in their requirements. Software relies on people, communication, knowledge, creativity and evolving process. It is a messy endeavor.

Despite the uncertainty, we have financials to prepare, executives to brief, and investors to reassure. So when we are considering a software project, we need to know how much it will cost in time and money. But, rather than jumping to estimation, let’s consider the power of the humble budget. The concepts of budgeting and estimation seem related, but they are actually opposite ends of the same process. Let me explain.

What exactly do you mean by budget?

A “budget” simply means the amount of money available and needed for a purpose. We’ll get to the needed part later on, for now I want to focus on available. Available implies that there is a fixed amount that can be used, once it’s gone, it’s gone. In software terms we’re talking about time and materials versus fixed bid engagements. This means that we can separate scope from cost, allowing scope to be fluid and cost to be set by the budget. I’ll address the evils of fixed bids in a later post.

Three Reasons That Budgets Work Better Than Estimates

3 Estimates Pile Up, Budgets Count Down

When a project is estimated up front, progress looks a lot like putting items from the grocery store shelves into a cart. The only thing you can really see is how much is in there, and perhaps that the cost of potatoes has really gone up. This viewpoint is entirely rearward facing, it only has clarity in hindsight.

Instead of big and complicated estimates, when we develop a reasonable budget instead our view is forward looking. Budgets are concerned with the intrinsic potential of the remaining resources and less so with what’s already been spent.

2 Budgets Constrain, Estimates Decay

In any creative endeavor, constraints are incredibly valuable. Without constraints, the lines of effort wander and waste seeps into the process. Estimates don’t give you constraints – they merely quantify the options, and when we use them, we stop asking, “is this actually the best way to spend our resources?”

Further, estimates atrophy, much like documentation gets out of date. Original estimates become less and less valuable as the realities of a project veer away from the initial understanding.

Establishing a budget, however, shines a bright light on prioritization. If funding is limited, then we always need to be focusing on the next most important thing. Even if the budget is artificial, having that constraint provides a valuable lens through which to evaluate scope and possible tradeoffs.

1 Estimates Highlight Failure, Budgets Highlight Success

Nobody is ever happy about an estimate being wrong. Whether it was too high or too low, an incorrect estimate throws off all our predictions. And, if estimates aren’t good predictors of effort, why bother?

When tracking against a budget, however, a team has a much higher chance of reaching some success. Here success is defined as anything at (or better than) the allocated amount. This gives us a wide target to hit, and the flexibility to change things as we go, without our estimates going stale.

So how do we arrive at a budget?

At this point, I’m sure you’re asking yourself how do we set a reasonable budget if we don’t do estimation? The answer is, “you don’t.” Estimation still has to take place, but it’s when you estimate, and what you do with the estimates, that changes. Our projects typically employ two different types of estimation: sizing and story points.

Sizing

We always start a project (or a milestone within a project) with a sizing exercise. There’s nothing scientific or formulaic about sizing. Sizing is more of an art, which relies on the experience of senior folks from the various disciplines on a software team.

Sizing involves developing enough understanding of the high-level requirements to allows us to break down the project into feature units. These aren’t as granular as user stories, but are smaller than what you’d put on a bullet list of features in a marketing campaign. Each unit is then roughly “estimated” at a specified number of days or weeks.

It’s important to keep in mind that we’re not just throwing numbers out. Sizing requires thought and experience. It’s key to have several different people weigh in and ask questions, a healthy debate should develop during these discussions.

Once all of the rough estimates for modules are tallied up, then we adjust for staffing. How many developers do we think we’ll need? How many designers? What about project management? After factoring in staff and rates, we have our budget. Pretty simple.

One last point to make here. Once we have our budget, it’s important to discard the details of the sizing. If we don’t it’s tempting to just view the sizing document as an estimate breakdown, which invalidates the whole budgeting approach.

Story Points & Iteration Planning

The second type of estimation is done in real-time and is focused on helping us make smart decisions with how to spend the remaining budget. Working in one week iterations, we hold a planning meeting at the beginning of each week that determines what user stories will be worked on during the cycle. As part of this meeting, we add point estimates to each story. These are not focused on time, but focused on complexity. A simple story might get a single point, a more complex story may get 3 to 5 points.

After a few weeks, the project team begins to develop a velocity, the average number of points likely to be completed in a given iteration. Velocity helps us project where we’ll be within our budget at any given time. Rather than always looking backwards to assess whether we are keeping pace with our initial estimate, we can look forward and have rational and motivated conversations on how to spend the remaining budget.

A Final Note

Ultimately, the way you and your teams choose to create budgets, or estimates, should reflect what works best for you. Whatever process you implement, it needs to leverage the experience and intuition of your team, rather than being based in methodologies or formulas. Software, despite having roots in mathematics and engineering, is a collaborative art. As such, it cannot accurately be predicted by a calculation. What we can do however, is set realistic goals in regards to time and money, and provide our teams the necessary tools to measure and track a project’s progress.

JC is the founder and CEO of DevMynd and leads the company’s human-centered strategy practice.