If We Must Estimate

This post has been in my mind for a couple of years since my stint at Appsmith as an engineering manager. My team went from chaos to killing it at estimation, and the journey was both fun and inspiring, thanks to the great engineers that I had the honour of leading.

First, an important caveat: even in an imperfect world, No Estimates would be the way to go. That however, requires the entire organization to drink the same cocktail. In the absence of that rare scenario, there are other ways to get there.

• The central question for leadership is "how long will it take to build feature X?"

• Engineers have to answer that question at a time that they know the least about the problem and solution.

• There's also the pervasive tendency of everyone but engineers to greatly underestimate the cost (usually means time) to build something

At Appsmith, these very common problems existed, but we also had extremely progressive and critical minded leaders who allowed us the freedom to test, experiment and learn together. That gave me the opportunity to think through the fundamentals, and clarify the problems at the root:

• Executives complained the team was slow

• PMs had no idea what anything cost to build

• Engineers felt they were just cogs in a machine with no say in their own timelines

We all wanted to fix this. One of the superpowers of a good engineering manager is knowing what 1 or 2 interventions will be most effective from the 40,000 best practices in the market. So, simply adopting DORA or something similar was out of question. The main thing to solve in the situation was the accuracy of our estimates. And my secret desire was to just be done with estimates some day. We smashed the first one, and got pretty close to solving the second.

The need for economic analysis

Engineering is probably the most important function in the company, but in most companies they operate without any understanding of unit of work. As Sidu puts it, a feature is not the unit of work, it is the unit of distribution. Variably called effort, cost, etc., it is the thing the team invests to realise downstream value. And as units go, it needs to be a standard, interchangeable measure. The tongue-in-cheek-ness and the audacity of applying this concept to building software is intentional.

Building the habit

We already were using a kanban board, and the straighforward approach I introduced to the team was t-shirt sizes. During our fortnightly iteration planning, the team estimated every planned item together. We also maintained a doc to describe with examples the types of tasks for each tshirt size. This was the first half of the story. The first habit.

What I needed out of all this was the cycle time. By standard definition, this should be the time it takes to build something from idea to live. I took the liberty of tweaking that meaning in two ways:

• so that it pertains to the unit of work instead of the unit of distribution. (reasons explained in the previous section)

• start the clock when a task goes to "in progress" and stop it when it reaches "done". (we didn't want to be fussed with all the chaos upstream and downstream at this point)

Finally, I built an Appsmith app to do these and several other calculations. I wanted a graph that shows, broken down by tshirt size, the median of times it took our team to take a task from start to end. Of course, as we went along I had to make a couple of minor tweaks e.g., to exclude weekends.

A world emerges

Here's the graph. You can see this live on the app too.

In the beginning, it looks like Eminem's spaghetti. That was expected. But you can see rationality setting in over time. This is the outcome of the team learning the meta information about their work, learning to trust each other, and learning the value of slowing down and speeding up as needed. The graph ends at the dream state we all aspire to:

• XS and S tasks take ~1 day

• M tasks take ~2 days

• L tasks take ~3 days

• XL tasks take ~6 days

The discipline of decomposition

This is one part of the story. Remember, the secret plan for world domination was to ultimately get rid of even having to do these estimates. For that, the team needs to also get insanely good at decomposing tasks into the simplest possible increments. So, a second graph. A simple reporting of the monthly number of taks broken down by the estimate size. We want the larger ones to vanish. We were well on our way to it until the end of reporting (my leaving the team).

By breaking complexity into smaller, discrete units, we reduced the risk profile of our releases. We stopped trying to ship monoliths and started shipping predictable increments. This skill is perhaps the most valuable technical lesson the team carried forward to their next roles.

Professional predictability

By making our output measurable, we solved the original frictions:

• Executives were relieved of the "slow" narrative because we could finally project capacity with confidence.

• Engineers regained agency, as accurate estimates protected them from the chaos of over-commitment.

• PMs could finally make trade-offs based on the actual cost of building.

In the end, engineering leadership is about creating a system where engineers can work with clarity. We didn't just ship more code; we built a culture of predictability that served the business and the engineers equally, while learning to take more pleasure doing it.