Several people are tweeting and writing blogs criticising or even pouring scorn over #NoEstimates
Let’s look at some of them:
Categorizes the arguments for #NoEstimates into:
1. Estimation is difficult
2. Estimates and velocity are misused
3. Fixed Scope of work = Commitments
4. Estimates are waste
He argues that the NoEstimates group is chasing the wrong problem
Gene Hughson says Noestimates lacks credibility
Most of the criticism is saying its a bad name, because its advocating forecasting or projection which is essentially just another form of estimation.
It doesn’t really bother me that there is confusion around the name. I see it as marketing in the same way as a headline is written with the goal of grabbing your attention, rather than being verbose and precise.
The #noestimates hashtag has been tremendously successful in getting people interested in the topic. The topic we are discussing is better ways to do estimation, and times when we can avoid it altogether.
Daniel Marlow pointed out the repeating problem with debating #NoEstimates, which is
Layne’s Law of Debate:
Every debate is over the definition of a word. Or
Every debate eventually degenerates into debating the definition of a word. Or
Once a debate degenerates into debating the definition of a word, the debate is debatably over.
This has happened over and over with #NoEstimates with both sides eventually reaching the conclusion that forecasting is essentially just another form of estimation. Oh!
I would like to move past this and talk about the real topic, which is how can we do estimates better, and are there situations were we are doing estimates when we don’t actually need to do them?
I only have good things to say about my employer but I don’t like to talk about my employer in my blog. This blog is about my personal thoughts and opinions and like to keep arms length because they do not necessarily represent any of my employers views. However in this case I need to share some details in order for you to understand where I am coming from.
My company is in a steady state. There are no significant risks of us going bankrupt any time soon. We are growing, but not so fast that it’s dizzying and we don’t know our own colleagues anymore. There are still fewer than 100 employees in the company across the UK, and the branch I work in has 8 employees.
There is absolutely no official policy that we don’t do estimates. We don’t follow the form of estimation covered in the No Estimates book. We don’t have anyone with the title of “Project Manager”. There are 5 full time developers, 1 tester and 2 managers in my branch and the managers get involved in some of the programming. We are recruiting!
We aim to release to live regularly. In practice we release a new version of our main application every 2-4 weeks. Inevitably some people would like us deliver more things in less time. In fact we would all like to be able to deliver more things in less time.
I believe that the most important concept in project management is fast, cheap, good pick two. Or a little less informally Schedule, Cost and Product.
This is typically represented as a triangle. What pick two means is the customer can pick two of those things, but the development team needs to have some degree of control over one of those things to prevent everything going wrong.
If a customer has control of all three things and decides to demand too much product for almost no budget and almost immediate delivery, it is easy to see that there is absolutely no way that the development team will be able to satisfy that and that both the customer and the development team will end up either disappointed or frustrated or both.
A good developer or manager will always do their best to work with the customer to explain the different options available:
– Can speed up the schedule without reducing the product by increasing the cost (usually to hire more staff)
– Can speed up the schedule without increasing costs by cutting or adjusting features (many ways to do this)
– Can reduce the cost without cutting features by increasing the schedule (reduce the number of developers working on the project)
It’s not always “you don’t get nowt for nowt” – much of the time developers can actually provide more value more quickly by gaining a better understanding of the requirements and identifying more valuable alternatives that are easier to develop as well. However once all of these opportunities are exhausted everyone is at the point where a compromise needs to be made.
Estimates aren’t Schedules
I should point out that Estimates and Schedules are of course different things. In my opinion the main problem with estimates is they tend to be closely correlated with schedules.
It is common for estimates and schedules to have a 1:1 mapping. My many years of IT experience before moving to Scrum taught me that in medium to large companies the project manager tends to be the last person to know what’s been going on.
I have had a number of good experiences as well, but I found that in general they:
Work on a portfolio of different projects
Work on a separate floor in the office (communication issue)
Are the last to hear about problems (communication issue)
Spend a lot of time on the initial Microsoft Project Gantt charts, and like to stick to delivery dates based on Microsoft Project says so.
The initial Gantt chart is printed out and stuck to a wall for everyone. Within a couple of weeks everyone can see it doesn’t fit with reality and is clearly out of date, however it remains on the wall throughout the project.
Precision is not accuracy
The term precise is often taken to mean accurate, but it refers to how narrow the range measurement is. You can gain accuracy by decreasing precision. For example a delivery date of Q3 2016 is much more likely to hold true than a delivery date of 1st September 2016 15:13:45 UTC.
So whether we provide one date or two, we actually always provide the customer with a range, even if that range is only one second.
Although customers will not always know it, they tend to place a much higher value on whether the delivery is within time range, than whether the range is precise enough.
When I had to give man days estimates I used to give 3 numbers instead of one (optimistic, nominal, pessimistic). I was the only person who did this as far as I’m aware. I did it to try to be more helpful but it never caught on.
Story points are fairly effective at combating some of the above problems. I did find it a little ironic however that the Scrum team spent far more time estimating than was ever spent doing estimates under a PRINCE2 model.
Henrik Ebbeskog points out that much of the arguments in favour of #NoEstimates are a form of Sturgeon’s law, that it 90% of estimates are “crap” but 90% of everything is, including books, films, paintings and software. Lack of quality might be a reason to practice until you can do something better rather than giving up.
However, in the field of Software Development, estimates are not our product. The software is our product and our goal is to create better software. Time spent estimating does not create any software, and there is a poor correlation between the amount of time spent estimating and the accuracy of the estimate. So I agree that estimates can be waste.
However the greater risk to software quality is if an estimate leads to an overly aggressive schedule.
Also an overly pessimistic schedule is also damaging. No software is perfect, but once software is “good enough” for the customer we should release it. A comprehensive definition of good enough would be very lengthy so I won’t try here. An overly long schedule is another form of waste.
I find that often when someone asks for an estimate, what they really want is the schedule. So I believe that it is sometimes better to provide facts and let the other party infer the schedule based on that.
Facts instead of estimates
Not too long ago I had a support issue where the customer was demanding an estimate for when the problem would be fixed. At this point is didn’t know what the solution was so would not have been able to provide precision better than between 1 hour and 1 week. If I said that it would have likely only made the customer more angry.
The customer did not really want an estimate. He just wanted to know that it was going to be fixed soon. About an hour later I had written a fix. So I reported back that I had written a potential fix that needed further testing and that we would release as soon as safe to do so. That was enough to satisfy the user without making promises that might not be possible to keep. Knowing that we are doing our very best is the most important thing in my opinion
Fortunately testing was successful and we released the fix a few hours later.
Realistic estimates can be useful, but realistic schedules much more so
I’ve had good and bad experiences of estimates and schedules. The following examples are from a successful financial services company that I previously worked for. Although there’s no reasons to name names, I wish to preface this by saying that this company employs very good developers and good quality staff in general.
1. New product launch
There’s a new idea for a product. IT are told it will be very similar to the existing main product but with just a few differences, and ask for an estimate. Estimate comes back saying 500 man days of work. Armed with this, the CEO announces the exact date of launch to the public. The project spends a few months on the fuzzy front end, and then a detailed requirements document is delivered. This involves far more work than was ever mentioned before. Estimates are done again: 2,000 man days. There are a few months left before the new product has to launch…
After the product launched, and many emergency fixes were made, the company decided not to follow the normal process of having an in-house post-mortem, and instead asked for an independent auditing company to review the project. I think this decision showed a lot of maturity. The auditors made a number of recommendations, I didn’t see them but I believe moving to an Agile model was one of them.
2. Lump in ambitious architectural changes with essential requirements and hard deadlines
Fast forward one year. Myself, 3 other developers and 2 testers I have been invited to sign up for a Scrum pilot team. We are feeling highly motivated. Outside of our team, there is much derision of Agile, and some fundamental misunderstandings of what it is about. The company is aiming to deliver more than 50 projects this year, and two of them are regulatory requirements that must be delivered by an exact date. One of project these seems to be going okay, and the other is showing signs of trouble.
The architecture team decided many months ago that replacing a major subsystem in the legacy monolith application with a new CRM based solution should be part of the project work. A CRM upgrade project was done. The lead developer gave estimates, but they were see to be too high. In order for the main project to have enough time to succeed, the upgrade project would need to delivered quickly.
The lead developers estimates were argued down to 1/3 of what they were so that they only accounted for known work, i.e. the basic steps to upgrade the system were known but the amount of work fixing problems afterwards was completely unknown.
On the testing side as well, the estimates were seen as a problem. Regression testing a large CRM system involves a lot of work and takes a lot of time. To cut the schedule, a decision was made that the only testing that would be done was “hotspot” testing: a few areas of the system identified to be at the highest risk of having problems.
The initial build given to the testers had numerous bugs. These were tracked and given back. The amount of time to address these took the developers up to their original estimates. But now there was a build were no problems could be found by the hotspot testing. The new system was put live…
This is the point were the scrum team is born. Initially the senior CRM developer is working completely off the board on CRM defects and is under a lot of pressure. I recommend that all of this work is added to the board.
3. Don’t worry we’ve done your estimates for you!
A few months later another regulatory project is given to our team. We are meant to be a small change team (mostly maintenance work) but this was a major project. There is a short paper spec with a vague description and a total man days estimates of 45 man days best case, 60 man days nominal and 75 man days worst case. Looking at the description I cannot understand how anybody has been able to give anything other than a wild guess. We re-estimated in story points, approximately 100 story points equating to about 300 man days. We were still pretty optimistic. The project would end up involving around 1,000 man days of effort and disbanding the small change team.
4. Tactical solutions and their consequences
While these regulatory projects were going on, a business manager asked for a new system to launch a new product. There were no developers available until the following year. The business manager argued that this wasn’t acceptable. He needed this to be done by Christmas (about 6 weeks away). Our prospective partner company might not sign the deal if we’re not ready to launch. Surely there must be some way that this could be done? The IT manager offered a tactical solution: he would develop a simple application in Excel that communicates with a SQL Server database. This would need to be replaced with a professional solution later on.
I became involved in this project about 8 months later, working on the project on a 50% time basis for the next 3 months. The other company had reviewed several contracts at its leisure and requested amendments in its favour during this time, whilst the business continued to ask for more functionality in the application. It was now thousands of lines of VBA and a lot of very complex stored procedures.
After the initial application went live, the team moved to a Scrum approach. Then the business manager asked for a new project to launch a similar financial product. Because it was similar to the product we just launched, it would be quicker to do it in Excel again, although not quick. The business manager requested that the project be completed in 6 weeks. I advised the team to avoid treating any dates as deadlines, and to stick to the normal Agile estimation process.
Some of the other developers asked for the team to be split up so that they would not have to work on this project. There was some logic to it because use of spreadsheets meant only two or at maximum three developers could work on the project at the same time without overwriting each others changes.
The team split did not happen, and once we got away from discussing impossible schedules, motivation improved. We delivered the project about 4 months later. This work included about 3 times the original backlog of stories and, with today’s perspective, this was a great achievement for the team.
Estimates are most useful is when they prevent the approval of a project that would be a huge waste of money. My best memory of how estimates can be helpful is a set of estimates that I wasn’t even involved with.
The legacy Excel application just described was pushed too far for the technology and taking up a lot of the teams time with support issues. The business decided they wanted to do yet another project for a large set of enhancements to the application. Another member of my team spent half a day giving story point estimates for each requirement. This informed the business that it was going to be expensive, and thankfully the project plan went no further.
What I think is often a problem is the mindset that everything needs to be a project, or part of a project. Planning at a large project level is where the bigger risks are involved. A small piece of work usually has value on its own as a story.
I hope this post hasn’t been too much of a ramble. Estimation is a big topic, there’s plenty more to say on it so I will keep updating this with gradual improvements.
Improving the accuracy of project estimation
Steve Fenton: No Estimates in Practice
The yestimates hashtag
Peter Kretzman The Case Against No Estimates
Humpty Dumpty and #NoEstimates
6 thoughts on “Yes #Noestimates = Estimates, so let’s discuss estimates”
Nice post! Balanced. A little bit of “humpty dumpty” perhaps, but we’ve talked about this already 🙂
There are some things I’d like to point out.
“However, in the field of Software Development, estimates are not our product. The software is our product and our goal is to create better software. Time spent estimating does not create any software…”
I have to disagree. And a very developer-centric view. Let me explain. My job isn’t to create software, that’s just *my* craft to help my customers/stakeholders with what they want.
Compare with a carpenter. His/her product is not the hammer and the saw – it’s to create something that will make someone else happy.
With this said, part of making someone else happy is to be able to give them an idea of how much it, of that thing they what, will cost and when they can expect it. You could compare this with many things in your own life as well.
With this said #2, part of making customers/stakeholders happy is to – with your expertise – help them see that there might be alternative ways to get the thing they want. Perhaps smaller, perhaps larger, perhaps with better quality, perhaps with lower quality but quicker, perhaps cheaper etc. It all depends what the customer/stakeholders wants/expects/accepts. This is key. The software is just our craft.
So. No. Time spent estimating is not waste. Read more here: http://kodkreator.blogspot.se/2015/04/estimates-are-waste.html and here http://everydaylean.info/2014/08/estimates-waste/ (there are more good sources on this). It’s actually waste to *not* estimate, since people can’t even continue without the information they provide.
You also write:
“…are there situations were we are doing estimates when we don’t actually need to do them?”
Yes. Easy one. When it doesn’t affect or confirms a decision. Sometimes expectaions are known (like in a “lean” flow where we’re releasing continously and customers/stakeholders know that what they’ll want usually takes just a couple of weeks and that’s good enough). Then we might not have to deliver estimates explicitly. You could say that we have increased the predictability this way.
Thanks for your feedback Henrik,
I am a developer so I’m not trying to present anything other than my own views.
There is a huge difference between my line of work where there are thousands of end users, and say, a new startup digital design agency with only a couple of customers on its books. In the latter case, the company will need to bend over backwards to do pretty much anything the customer wants because the company could go under if they customer decides to take their business elsewhere. This is not a desirable position for a company to be in. A better position to be in is delivering “shrink wrap” products to a large number of customers.
Take Microsoft for example. If they halted development to give out estimates every time that someone asked for an estimate of when a bug fix or a new feature would be released, then even if by some miracle they were still in business today, we would still be waiting for Windows 95 to be launched.
Glen Alleman has pointed out that Microsoft do indeed do estimates which is course true. The article that made me use Microsoft as an example is EF7 Priorities, Focus and Initial Release.
Entity Framework 7 (EF7) is a big, tough project, almost a ground up rewrite of the whole project. Rowan Miller would ideally like the team to work in a vacuum so that it is not distracted by delivery dates, but concedes that in the real world he needs to make some compromises to get a product out.
He makes absolutely no promises about when it will ship or on whether it will be a complete product when it launches. Since then many developers (oh how the shoe is on the other foot) have contacted him saying we need to know when this is out so we can plan which technology we are using in our next project. In other words he’s been asked for estimates of the variety that would be interpreted as a schedule, many, many times. So my point is even though these requests are coming from people who are customers in a sense and have a legitimate interest, Rowan is doing the correct thing by ignoring that in favour of focusing on the product.
In my view, and broadly speaking, giving out estimates is more likely to result in decreased rather than increased customer satisfaction. The main reason for this is the customer doesn’t know what the relationship is between the estimate and the schedule.
8 story points means nothing to them so it would cause nothing but confusion and a feeling that they are not being respected enough.
Using a non-imaginary unit such man days is even more dangerous because 10 man-days could mean its going live tomorrow or next year. Now I’m sure you are thinking “you need to give the customer not just the estimate but the schedule”. The schedule depends on many different things which are unforeseeable. Just a few of these are:
Are the priorities going to shift towards a different project next week?
Are we going to receive any large support issues that trump the project development work?
Is Julie going to be off sick with the flu for the next week?
Is Tom going to have a family emergency meaning he can’t come in tomorrow?
As we don’t have answers to these things, schedules are built on assumptions that are guaranteed to be wrong at least some of the time. This can lead to all kinds of really nasty compromises such as “should we ship ‘on-time’ with bugs or ‘late’ with no bugs?”. The customers are not going to be very happy either way.
Even if you can hit the target exactly, the product is likely to be not quite as good if the development has spent on significant amount of their time doing estimates instead of delivering better software.
Sorry, I forgot, you don’t think “no estimates” is really about “no estimates”. I have to keep that in mind 🙂 I was more addressing it literally in my response 🙂
Because It’s rather simple really. Whenever you’re going to do work that someone else is paying for, you will have to tell them roughly what that work will cost (what they can expect the invoice to say) 🙂 There’s no way out of that. Or put it in another way; when investing in something it’s very likely you’ll want to have an idea of what that investment will be, and when the investment will start to pay off or at least when to expect being able to have/use it. But if you can get someone to not care about that, that’s awesome (and perhaps stupid? Or more nicely: very brave?) 🙂
About your other points about schedules etc, I think you’ll love this video: http://www.infoq.com/presentations/risk-project-management
And sure, it’s all about context. What works there doesn’t work here. A wonderful read: http://firstround.com/review/the-right-way-to-ship-software/
I’ve just watched the video on Risk Management. Yes I totally agree with this, these are the problems that I have found again and again over the course of my career. I recommend this video to anyone interested in project management or risk management. One of the key points is:
“..they wait until the problem is there, and then they have problem management not risk management…Your name in Project Manager but the real project manager is time. Time is managing your project not you. You’re just reacting, you’re not managing”
Also where he says “these people have no right to estimate less than the season” – this is exactly my point above on accuracy and precision. Typical estimates are given to a ridiculous precision.
And I’ve also regularly seen the approach to risk management “They create a list of risks. And then they file it. That’s not risk management.”
So I would say a common business approach to both project management and risk management is just to create more bureaucracy. This adds more overhead without eliminating the risks.
Instead of recruiting a risk manager, understand that (if you do Scrum) your scrum master, your product owner and your developers are the real risk managers.
Whatever life cycle or methodology you use, your developers are risk managers. Help them to be better at it.
“The right way to ship software” is exactly right. Arguments on which process is “better” are much like arguments on which technology is “better”. The real question is which is the most appropriate for the given context.
This makes discussions on estimation more difficult because we all come from different backgrounds and have different experiences
Steve McConnell has a video clip on #NoEstimates where he makes the same point about doing the right thing for the customer:
The example he uses is more of a design for tools sort of an example. The customers would be looking to buy a set of prebuilt components that the builder assembles. So the builder would actually ask questions to determine budgets and requirements on each component and look at the total materials cost before considering labour costs. This example is far more straightforward than a typical software project where the customer has far less of an idea about what they want in the product.
Thanks for the update. EF7 is one on 100’s of “products” MSFT produces. This example does not follow the products we use and wait for. Updates to platforms, TFS, SP, ProjectServer, AlwayOn DB services, all have “estimated” delivery dates.