Several people are tweeting and writing blogs criticising or even pouring scorn over #NoEstimates

Let’s look at some of them:

Ryan Ripley

Categorizes the arguments for #NoEstimates into:

1. Estimation is difficult
2. Estimates and velocity are misused
3. Fixed Scope of work = Commitments
4. Estimates are waste

He argues that the NoEstimates group is chasing the wrong problem
Agile Answerman

Gene Hughson says Noestimates lacks credibility

Most of the criticism is saying its a bad name, because its advocating forecasting or projection which is essentially just another form of estimation.

It doesn’t really bother me that there is confusion around the name. I see it as marketing in the same way as a headline is written with the goal of grabbing your attention, rather than being verbose and precise.

The #noestimates hashtag has been tremendously successful in getting people interested in the topic. The topic we are discussing is better ways to do estimation, and times when we can avoid it altogether.

Daniel Marlow pointed out the repeating problem with debating #NoEstimates, which is
Laynes Law.

Layne’s Law of Debate:

Every debate is over the definition of a word. Or
    Every debate eventually degenerates into debating the definition of a word. Or
    Once a debate degenerates into debating the definition of a word, the debate is debatably over.

This has happened over and over with #NoEstimates with both sides eventually reaching the conclusion that forecasting is essentially just another form of estimation. Oh!

I would like to move past this and talk about the real topic, which is how can we do estimates better, and are there situations were we are doing estimates when we don’t actually need to do them?

I only have good things to say about my employer but I don’t like to talk about my employer in my blog. This blog is about my personal thoughts and opinions and like to keep arms length because they do not necessarily represent any of my employers views. However in this case I need to share some details in order for you to understand where I am coming from.

My company is in a steady state. There are no significant risks of us going bankrupt any time soon. We are growing, but not so fast that it’s dizzying and we don’t know our own colleagues anymore. There are still fewer than 100 employees in the company across the UK, and the branch I work in has 8 employees.

There is absolutely no official policy that we don’t do estimates. We don’t follow the form of estimation covered in the No Estimates book. We don’t have anyone with the title of “Project Manager”. There are 5 full time developers, 1 tester and 2 managers in my branch and the managers get involved in some of the programming. We are recruiting!

We aim to release to live regularly. In practice we release a new version of our main application every 2-4 weeks. Inevitably some people would like us deliver more things in less time. In fact we would all like to be able to deliver more things in less time.

I believe that the most important concept in project management is fast, cheap, good pick two. Or a little less informally Schedule, Cost and Product.

This is typically represented as a triangle. What pick two means is the customer can pick two of those things, but the development team needs to have some degree of control over one of those things to prevent everything going wrong.

If a customer has control of all three things and decides to demand too much product for almost no budget and almost immediate delivery, it is easy to see that there is absolutely no way that the development team will be able to satisfy that and that both the customer and the development team will end up either disappointed or frustrated or both.

A good developer or manager will always do their best to work with the customer to explain the different options available:

– Can speed up the schedule without reducing the product by increasing the cost (usually to hire more staff)
– Can speed up the schedule without increasing costs by cutting or adjusting features (many ways to do this)
– Can reduce the cost without cutting features by increasing the schedule (reduce the number of developers working on the project)

It’s not always “you don’t get nowt for nowt” – much of the time developers can actually provide more value more quickly by gaining a better understanding of the requirements and identifying more valuable alternatives that are easier to develop as well. However once all of these opportunities are exhausted everyone is at the point where a compromise needs to be made.

Estimates aren’t Schedules

I should point out that Estimates and Schedules are of course different things. In my opinion the main problem with estimates is they tend to be closely correlated with schedules.

It is common for estimates and schedules to have a 1:1 mapping. My many years of IT experience before moving to Scrum taught me that in medium to large companies the project manager tends to be the last person to know what’s been going on.

I have had a number of good experiences as well, but I found that in general they:

Work on a portfolio of different projects
Work on a separate floor in the office (communication issue)
Are the last to hear about problems (communication issue)
Spend a lot of time on the initial Microsoft Project Gantt charts, and like to stick to delivery dates based on Microsoft Project says so.

The initial Gantt chart is printed out and stuck to a wall for everyone. Within a couple of weeks everyone can see it doesn’t fit with reality and is clearly out of date, however it remains on the wall throughout the project.

Precision is not accuracy

The term precise is often taken to mean accurate, but it refers to how narrow the range measurement is. You can gain accuracy by decreasing precision. For example a delivery date of Q3 2016 is much more likely to hold true than a delivery date of 1st September 2016 15:13:45 UTC.

So whether we provide one date or two, we actually always provide the customer with a range, even if that range is only one second.

Although customers will not always know it, they tend to place a much higher value on whether the delivery is within time range, than whether the range is precise enough.

When I had to give man days estimates I used to give 3 numbers instead of one (optimistic, nominal, pessimistic). I was the only person who did this as far as I’m aware. I did it to try to be more helpful but it never caught on.

Story points are fairly effective at combating some of the above problems. I did find it a little ironic however that the Scrum team spent far more time estimating than was ever spent doing estimates under a PRINCE2 model.

Henrik Ebbeskog points out that much of the arguments in favour of #NoEstimates are a form of Sturgeon’s law, that it 90% of estimates are “crap” but 90% of everything is, including books, films, paintings and software. Lack of quality might be a reason to practice until you can do something better rather than giving up.

However, in the field of Software Development, estimates are not our product. The software is our product and our goal is to create better software. Time spent estimating does not create any software, and there is a poor correlation between the amount of time spent estimating and the accuracy of the estimate. So I agree that estimates can be waste.

However the greater risk to software quality is if an estimate leads to an overly aggressive schedule.
Also an overly pessimistic schedule is also damaging. No software is perfect, but once software is “good enough” for the customer we should release it. A comprehensive definition of good enough would be very lengthy so I won’t try here. An overly long schedule is another form of waste.

I find that often when someone asks for an estimate, what they really want is the schedule. So I believe that it is sometimes better to provide facts and let the other party infer the schedule based on that.

Facts instead of estimates

Not too long ago I had a support issue where the customer was demanding an estimate for when the problem would be fixed. At this point is didn’t know what the solution was so would not have been able to provide precision better than between 1 hour and 1 week. If I said that it would have likely only made the customer more angry.

The customer did not really want an estimate. He just wanted to know that it was going to be fixed soon. About an hour later I had written a fix. So I reported back that I had written a potential fix that needed further testing and that we would release as soon as safe to do so. That was enough to satisfy the user without making promises that might not be possible to keep. Knowing that we are doing our very best is the most important thing in my opinion
Fortunately testing was successful and we released the fix a few hours later.

Realistic estimates can be useful, but realistic schedules much more so

I’ve had good and bad experiences of estimates and schedules. The following examples are from a successful financial services company that I previously worked for. Although there’s no reasons to name names, I wish to preface this by saying that this company employs very good developers and good quality staff in general.

1. New product launch

There’s a new idea for a product. IT are told it will be very similar to the existing main product but with just a few differences, and ask for an estimate. Estimate comes back saying 500 man days of work. Armed with this, the CEO announces the exact date of launch to the public. The project spends a few months on the fuzzy front end, and then a detailed requirements document is delivered. This involves far more work than was ever mentioned before. Estimates are done again: 2,000 man days. There are a few months left before the new product has to launch…

After the product launched, and many emergency fixes were made, the company decided not to follow the normal process of having an in-house post-mortem, and instead asked for an independent auditing company to review the project. I think this decision showed a lot of maturity. The auditors made a number of recommendations, I didn’t see them but I believe moving to an Agile model was one of them.

2. Lump in ambitious architectural changes with essential requirements and hard deadlines

Fast forward one year. Myself, 3 other developers and 2 testers I have been invited to sign up for a Scrum pilot team. We are feeling highly motivated. Outside of our team, there is much derision of Agile, and some fundamental misunderstandings of what it is about. The company is aiming to deliver more than 50 projects this year, and two of them are regulatory requirements that must be delivered by an exact date. One of project these seems to be going okay, and the other is showing signs of trouble.

The architecture team decided many months ago that replacing a major subsystem in the legacy monolith application with a new CRM based solution should be part of the project work. A CRM upgrade project was done. The lead developer gave estimates, but they were see to be too high. In order for the main project to have enough time to succeed, the upgrade project would need to delivered quickly.

The lead developers estimates were argued down to 1/3 of what they were so that they only accounted for known work, i.e. the basic steps to upgrade the system were known but the amount of work fixing problems afterwards was completely unknown.

On the testing side as well, the estimates were seen as a problem. Regression testing a large CRM system involves a lot of work and takes a lot of time. To cut the schedule, a decision was made that the only testing that would be done was “hotspot” testing: a few areas of the system identified to be at the highest risk of having problems.

The initial build given to the testers had numerous bugs. These were tracked and given back. The amount of time to address these took the developers up to their original estimates. But now there was a build were no problems could be found by the hotspot testing. The new system was put live…

This is the point were the scrum team is born. Initially the senior CRM developer is working completely off the board on CRM defects and is under a lot of pressure. I recommend that all of this work is added to the board.

3. Don’t worry we’ve done your estimates for you!

A few months later another regulatory project is given to our team. We are meant to be a small change team (mostly maintenance work) but this was a major project. There is a short paper spec with a vague description and a total man days estimates of 45 man days best case, 60 man days nominal and 75 man days worst case. Looking at the description I cannot understand how anybody has been able to give anything other than a wild guess. We re-estimated in story points, approximately 100 story points equating to about 300 man days. We were still pretty optimistic. The project would end up involving around 1,000 man days of effort and disbanding the small change team.

4. Tactical solutions and their consequences

While these regulatory projects were going on, a business manager asked for a new system to launch a new product. There were no developers available until the following year. The business manager argued that this wasn’t acceptable. He needed this to be done by Christmas (about 6 weeks away). Our prospective partner company might not sign the deal if we’re not ready to launch. Surely there must be some way that this could be done? The IT manager offered a tactical solution: he would develop a simple application in Excel that communicates with a SQL Server database. This would need to be replaced with a professional solution later on.

I became involved in this project about 8 months later, working on the project on a 50% time basis for the next 3 months. The other company had reviewed several contracts at its leisure and requested amendments in its favour during this time, whilst the business continued to ask for more functionality in the application. It was now thousands of lines of VBA and a lot of very complex stored procedures.

After the initial application went live, the team moved to a Scrum approach. Then the business manager asked for a new project to launch a similar financial product. Because it was similar to the product we just launched, it would be quicker to do it in Excel again, although not quick. The business manager requested that the project be completed in 6 weeks. I advised the team to avoid treating any dates as deadlines, and to stick to the normal Agile estimation process.

Some of the other developers asked for the team to be split up so that they would not have to work on this project. There was some logic to it because use of spreadsheets meant only two or at maximum three developers could work on the project at the same time without overwriting each others changes.

The team split did not happen, and once we got away from discussing impossible schedules, motivation improved. We delivered the project about 4 months later. This work included about 3 times the original backlog of stories and, with today’s perspective, this was a great achievement for the team.

Estimates are most useful is when they prevent the approval of a project that would be a huge waste of money. My best memory of how estimates can be helpful is a set of estimates that I wasn’t even involved with.

The legacy Excel application just described was pushed too far for the technology and taking up a lot of the teams time with support issues. The business decided they wanted to do yet another project for a large set of enhancements to the application. Another member of my team spent half a day giving story point estimates for each requirement. This informed the business that it was going to be expensive, and thankfully the project plan went no further.

What I think is often a problem is the mindset that everything needs to be a project, or part of a project. Planning at a large project level is where the bigger risks are involved. A small piece of work usually has value on its own as a story.

I hope this post hasn’t been too much of a ramble. Estimation is a big topic, there’s plenty more to say on it so I will keep updating this with gradual improvements.

Further Reading
Improving the accuracy of project estimation
Steve Fenton: No Estimates in Practice
The yestimates hashtag
Peter Kretzman The Case Against No Estimates
Humpty Dumpty and #NoEstimates