There is no such thing as “absolute estimation”

Posted on August 25th, 2015 in Agile by siddharta || 3 Comments


One of the discussion topics I’ve seen raised again and again in agile circles is about absolute vs relative estimation. The theory goes that human beings find it difficult to estimate in absolute numbers like estimating the height of a building in metres, but find it easy to do relative estimations like estimating that a coconut is four times as big as a guava. And this is the reason why estimating a story in story points (relative estimate) is superior to estimating a story in hours (supposedly an absolute estimate). This theory finds it’s way into every agile discussion on estimation, as well as many trainings.

And yet, this theory, is completely wrong! (At this point I’ll admit that I’ve myself said this before)

There is no such thing as “absolute estimation”

Let us first get one thing out of the way: There is no such thing as absolute estimation.

ALL common measurements-like measuring the height of a building for example-is a relative measurement. After all, a metre is just a standard baseline defined by the International Bureau of Weights and Measures. If you call something as 5 metres long, it just means that it is 5 times as big as this standard baseline. Someone else might say that it is 16 feet long, and they get a different number because they use a different baseline. So measuring distance in feet or metres is just a relative estimate of multiples to a reference value.

It is no different from setting a reference story as 1 story point and measuring multiples against that reference.

The same goes for other types of measurements, like time for instance. Someone just decided to divide a day into 24 pieces and defined a reference value of 1 hour. When we say something will take 5 hours, we just mean it is 5 times as long as this reference. It is a relative estimate, just like story points.

So it is completely wrong to say that story pointing is better because humans are better at relative estimating, because estimating in hours is ALSO a form of relative estimating.

Comparing vs Estimating

One aspect of the first statement is true: it IS easier to say that a coconut is four times as big as a guava, than it is to estimate a building height in metres. The reason is not anything to do with relative or absolute estimating (remember that BOTH are relative estimates), but with how well humans can compare things. We are good at comparing objects that are similar in measure, but bad at comparing things that are of much bigger or smaller scale. As an example, it is easy to use a guava as a reference and say that the coconut is about four times bigger. But if I were to ask how big the Earth is relative to a guava, then most would be wildly off the mark (by wildly I mean more than a trillion times off the mark). However, if I were to ask how big the Earth is relative to Mars, folks would be reasonably close to the answer.

Putting this to use

Alright, that’s a lot of theory, but is it of any use? Fortunately it is!

Here is the anti-pattern: I’ve seen teams define a tiny 1 point story as a reference baseline and then try to size larger features or epics relative to this tiny story. It just doesn’t work because we are no good at sizing a big thing using a tiny thing as a reference. And then the team wonders why the superior “relative estimation” is not working.

If you’ve read this far, the fix should be obvious: Create a reference story at every scale. You should be having a reference 1 point story for tiny fixes. A reference 10 point story (13 points if your team rigidly sticks to fibonacci) for new features. A reference 100 point story for larger epics, and so on. Then compare your stories against the appropriate reference story.

So, to get back to the hours vs story point debate, does this mean that if we could create reference stories in hours, like saying that this is a 1 hour story, that one is a 10 hour story and so on, and then comparing stories against the reference, then an hourly scale would work just as well as story points? I’ll leave that question open to address in the next blog post.

Being agile with a fixed release date commitment

Posted on December 1st, 2014 in Agile by siddharta || 3 Comments

Recently someone asked me what to do in the following situation:

At the start of the project, there was some high level estimation done. It was estimated to be a six moth project based on which the executives committed a date to a customer. This was necessary because the customer needed some lead time to setup up some infrastructure and hire people to run it and they needed to know the go-live date by when they had to get everything prepared. It is now a month to go to release. Various uncertainties have resolved and we see that it will actually  take another three months. The team is pressurised to meet the date because the client would stand to make losses as they hired all these people, but the software was not ready.

The above scenario is fairly typical in the business world. I want to state up-front that I see nothing wrong in commitments. I have heard many people talk about how agile teams never make commitments, but that is not always the right path for me. Software is not developed in isolation, and in many cases there are other downstream dependencies after the software is built. There may be a need to start marketing, perhaps we need to start training the sales team, and so on.

In the example above, we see that the client needs to start hiring a couple of months before the go-live date because hiring is a process that takes a few weeks. But if they hired people and the release date was pushed out by two months, then it is not a good situation for them. So they need to know in advance when the software would be ready.

What is the answer to this situation?

Luckily, agile is perfectly suited to handle this. Here are some techniques.

Adjust scope

The first thing to realise is that everything in agile is about fixed date. That is what a timebox is — a fixed start date and end date. Everything in agile is based on timeboxes, or in other words, it is all about dealing with fixed end date. We handle this by varying the scope. Usually not everything is high priority. And if you have been prioritising well, then most of the high priority features are completed early in the release. So, it should be possible to cut scope and release by the end date.

Agile is actually a much better way to handle fixed dates than a waterfall process. This is because in waterfall it is very hard to cut scope near the end of the release — most of the development is already complete and the whole application is in integration or testing. So the only options are to extend the date or to cut quality; both bad choices compared to cutting scope.

Make commitments as late as possible

Adjusting scope is a fairly standard response to dealing with a fixed date. On the other hand, making commitments as late as possible is a technique that very few organisations practice. It is a technique that has become popularised in mainstream agile with the Real Options work of Chris Matts and Olav Maassen. I highly recommend their book on this topic; appropriately enough it is called Commitment.

Going back to the example at the top of this post, we ask ourselves — what is the last responsible moment to fix the release date? Since the client needs 2 months to hire everyone, the answer is that we need to fix the date when we think we have 2 months of work remaining. There is no value is fixing the date earlier — the client will still need to start only with two months to go. Yet, in project after project, the date gets fixed right at the start of the project. So, a date is fixed many months in advance, when our knowledge is the minimum, and uncertainty is highest. And then that uncertainty comes to bite the team near the release date.

A better approach is to leave the date open at the start. It isn’t time to commit yet. Then, when we see that there are 2 months of work remaining, we work with the client to fix the date. This gives the client the lead time to get their hiring done, and it is MUCH more accurate since we are about 2/3 of the way through the project, most of the uncertainties are resolved and we all have a pretty good idea of the release date by now.

Learning from movies

It is also instructive to see how movie release dates are committed. Like software, movies also have many downstream dependencies. Marketing starts much in advance, but if marketing starts too early, then it is less effective. Distributors have to be selected, contracts negotiated, and advances paid. The film has to be transported to movie theaters, and bookings open a couple of weeks before release. Copies have to be sent to critics and reviewers so that their reviews are in the papers before release. All these activities require sticking to a very precise release date.

However, if we see how that release date is committed, we see that they follow the “commit late” principle outlined above. At first, there is nothing mentioned about a release date. After some of the movie has settled in, we may see an initial trailer with a vague date like “Coming  in 2015″. Once shooting is done and post-processing is ongoing, we may see something more specific, like “December 2015″. It is only a couple of months to release that the actual date is finalised, “In theaters on 23rd December”. By this time, 90% of the movie is complete and it is a pretty accurate date.

Imagine if movies took the software approach and fixed the release date of 23rd December 2015 before they even started !! It is simply not a recipe for success. Maybe we should learn from that.

What constitutes “minimally viable agile”?

Posted on June 25th, 2014 in Agile by siddharta || No Comment

There is an interesting discussion on LinkedIn on “minimally viable agile“. What is the bare minimum that a team should do in order to be considered agile? Should a team do standups to be agile? What if they don’t do standups, does that mean they are not agile, or is it possible to be agile without it? What does it even mean “to be agile”?

This is a topic that comes up every so often without fail. Here are some approaches that are inevitably put forth. I have ordered it in terms of my personal preference, from my least preferred approach to my favourite.

It’s agile if it includes these practices

This school of thought professes a set of practices, like standups or retrospectives, which need to be followed in order to be agile. So, if you aren’t doing retrospectives, then you aren’t agile. The problem with this approach is twofold. First, there are many different agile processes–Scrum, XP, Kanban, FDD, Crystal–and each has its own set of practices, with only very small overlap. So it is hard to nail down a set of practices which MUST be followed. Secondly, there are so many teams that follow ALL the practices, but someone who observes them would not call them agile. Maybe they are co-located, but they dont talk much. Maybe they are developing software incrementally, but rarely release to production and get user feedback.

It’s agile if it follows the manifesto

Another school of though is that it’s not agile if it doesn’t follow the manifesto. Individuals over Process, etc. But there are problems with this approach because it is just too vague. How do you really tell if a team is following the manifesto? The manifesto is a very high level vision statement rather than something actionable where you can go and say, “Yes, this team is following the manifesto”. It can also lead to strange situations, for example a team doing cowboy coding, because the team members just feel like it, would be better aligned to the letter of the manifesto. After all, it is the individuals who have decided to dump the process. But it is hard to think of such a team as being setup for success.

The 7 properties of successful projects

If a set of practices is too constricting, and the manifesto is too vague, then what is the solution?

My guide has been Alistair Cockburn’s excellent 7 properties of  successful projects. Crystal Clear was one of the very first books I read and it made a very strong impression on me, and I’ve been carrying this list around ever since. Unlike other thought leaders who are mostly working from anecdotal experience, Alistair’s list comes from specific research into successful projects. Based on this research he identified 4 core properties that most successful projects possess, and 3 additional properties that improve the chance of success, but are not totally critical (ie, it is harder, but possible, to succeed without them).

Here are the 7 properties (top four are the core properties):

  • Frequent Delivery: Do you deliver software to production, at least once a quarter?
  • Reflective Improvement: Do you take time to look back at how you are doing, and where you can improve?
  • Osmotic Communication: Does it take less than an hour to get the answer you are looking for?
  • Personal Safety: Can you give people bad news?
  • Focus: Does the team know what their top priorities are?
  • Easy access to expert users: Does it take less than three days to get an answer from an expert who understands the customer?
  • Strong technical environment: Does the team have automated tests, frequent integration, etc

There are two things that I love about this list.

First, they are outcome based. Take Osmotic Communication for example. Maybe one team might want to make it happen via colocation. Another team might be distributed, but ask all the team members to be online on team chat. Both practices can deliver on the goal: Does it take less than an hour to get the answer you are looking for? Conversely, there may be a team where everyone is sitting together, but they work in isolation without talking. If a team member has a problem, he tries to solve it himself for days without asking anyone. Such a team might be colocated, but they don’t have osmotic communication.

The second thing I love is that it is easy to evaluate. It is not a pie in the sky vision statement. It is specific, and you can easily go to a team and say whether the delivery frequency is good or not, whether communication channels are good or not, etc. It is a great way to figure out where a team stands on the path to agility.

Best of all, it is practice agnostic. This list is applicable whether the team does Kanban, or Scrum, or some custom hybrid.

Being a change agent

Posted on May 25th, 2014 in Agile by siddharta || 2 Comments

Yesterday, I spoke at Absolute Agile @ Hyderabad. The topic of my talk was 1/3 Agile, or being a change agent. The slides are above, but they don’t make much sense on their own, so here is some explanation for the slides:

What does it take to be agile?

There are three areas to focus on to improve agility:

  • The process itself: stories, feedback, incremental and iterative development…
  • Building strong teams: Self organisation, collaboration, continuous improvement…
  • Using effective technical practices: TDD, continuous integration, clean code…

Out of these three, most organisations focus a lot on the process, but ignore teams and technical practices. This leads to what I call 1/3 Agile, where we are limited to only a subset of the benefits.

What stop us from building better teams and using better technical practices?

The audience raised a number of obstacles that stop them from having strong teams and tech practices. These obstacles generally fall into three buckets:

  • Culture: Some say that their organisation culture is still command and control, and these practices wont work there
  • Authority: Some say that they don’t have the authority to make the change, and it must come from senior management
  • Skillset: Some say that they would like to do these but don’t know how

What now?

We have two options when faced with these obstacles.

  • We can give up and stay with the status quo
  • Or we can be a change agent and shape the future

Do we want to be remembered as someone who wanted to do something great, but wasn’t allowed to by the organisation, and in the end you did nothing of note? I hope not :) and in that case, the other option is to be an active change agent.

Continue reading ‘Being a change agent’ »

Four ways to make agile ceremonies more productive

Posted on May 6th, 2014 in Agile by siddharta || No Comment

​One of the common complaints about agile is that there are too many ceremonies. The issue isn’t with meetings per-se. The problem is that too many of the meetings are not as productive as they could be. Here are 4 ways to make the meeting more productive:

Keep the ceremony on track

A common cause for an ineffective meeting is that the conversation goes into tangential topics and the meeting outcome is never met. This leads to another meeting to be scheduled, or delays in the work. Here are some examples:

  1. ​In a grooming meeting, the conversation goes away from preparing upcoming stories. The PO and team instead start discussing the status of current stories. By the end of the meeting, none of the stories are groomed. When the next planning meeting came around many stories will not be in a position to be picked up.
  2. In a daily standup, the discussion goes towards solving one particular blocker. The whole team is standing and getting totally bored while two team members have a long discussion on the blocker.

In all these cases, keeping the ceremony on track would have saved everyone some valuable time. The next time you notice a discussion going on a tangent, put the discussion on the parking lot, or schedule another meeting specifically for that discussion. Then get back on track for the reason you have set up the meeting.

Reduce distractions

Be sure to run meetings without laptops and phones on silent. The biggest meeting killer is when someone is talking and meantime everyone else is checking email or doing something else. When folks are distracted, the meeting loses its purpose, and further discussions have to take place during the to go over the same ground that was covered in the meeting. When everyone is focused, the meeting can be finished earlier and everyone can get back to work.

Set aside certain slots for scheduling meetings

Software development requires a block of time to concentrate. Having a meeting in the middle can disrupt that time. It is better to have a 60 minute meeting followed by 4 hours of coding time, rather than 2 hours of coding time, then the meeting, followed by another 2 hours of time. Agile ceremonies happen on a regular schedule, so you can easily schedule them in the most convenient time. For example, some teams schedule all their ceremonies in the morning and keep the afternoons free of meetings. This is more productive than randomly having meetings throughout the day.

Ensure that the ceremony pre-requisites are met

A few days before the ceremony the Scrummaster or team should make a quick check that everything is ready. For instance, the stories should be well formed, and ready for discussion. If they are missing acceptance criteria or there are outstanding decisions to be taken, then the story should be de-prioritised from the discussion. Otherwise you will waste time in the ceremony going off track. Similarly, if some architecture review was identified in the grooming, then ensure it happens before the meeting.

« Previous Entries