One of the first steps in getting a new project underway is figuring out how much time and resources you’ll need to allocate to getting the job done. But estimating these numbers can be challenging even for experienced teams. Here are some of the reasons that our engineering team think estimates are important, and the guidelines we use for making estimating project timelines a little less painful.
The Impact of Project Estimates
We think the point of estimating things is to force you to plan what it is you need to do to complete the task so that you will be less likely to run into unforeseen problems. This is a strategy to reduce surprises and risk, but there is a trade-off involved because it’s not possible to perfectly plan – the best estimate of how long something will take can only come after you’ve completed it and everything that was unknown is known.
Given this, planning is a trade-off in time spent planning verses using that time in “doing”. Generally speaking, the bigger a task the more time you should spend working out what’s involved and how complex each part might be (because getting this wrong will result in bigger problems if we get parts wrong – a 3 month project with a bad assumption could cost 3 months or even more as you might need to work around the work you’ve already done).
The smaller the task the less time you need to plan what you’re doing, because it’s cheaper to make a mistake and because you’re more likely to give a good estimate. Sometimes, just doing the task is the right thing to do, because you’ll learn quicker what needs to be done – this is fine, just make sure that everyone involved knows there is potentially hidden complexity in the task.
Bad Estimates Are Better Than No Estimates
It’s also fine for estimates to be wrong, after all they are our guesses based on imperfect information, and so it’s unsurprising that the unknown will mess up our day.
You should know, we developers are bad at estimating stuff:
Typically, effort estimates are over-optimistic and there is a strong over-confidence in their accuracy. The mean effort overrun seems to be about 30% and not decreasing over time. For a review of effort estimation error surveys, see.[3] However, the measurement of estimation error is problematic, see Assessing the accuracy of estimates. The strong overconfidence in the accuracy of the effort estimates is illustrated by the finding that, on average, if a software professional is 90% confident or “almost sure” to include the actual effort in a minimum-maximum interval, the observed frequency of including the actual effort is only 60-70%.[4]
If you trust everything you read in Wikipedia, this means you should almost always increase your estimate by 30%!
The article 5 Laws of Software Estimates has a lot of things in it that I agree with, whilst you’re taking the time to read this, you should read it as well. For one thing, it has more memes in it.
It’s important to state that when it comes to estimates we’re not looking for people to use mystical formulas or complicated systems (of which there are many in this Wikipedia article) – we’re looking for you to break down tasks and give a back-of-the-envelope estimate of how long they will take. This isn’t rocket science people.
3 General Rules for Better Estimating
1. Write stuff down
Writing forces you to expose all the hidden assumptions you’re making when you’re thinking through things. It’s much easier to spot missing/inconsistencies when they are written down. It means that you can ask other people to check your reasoning whilst you’re doing something else and you can refer back to it when you forget what it is you’re supposed to be doing.
2. It’s ok to be uncertain
Everything is uncertain. This is fine. Say why you’re uncertain and give a range with some explanation of why you’re uncertain and why you’ve chosen the lower and upper bounds of the range.
3. Ask your peers
Unless you’re starting a brand new project, you’re going to be working around someone else’s code, their input on your plan and estimate is likely valuable and well informed, so asking for a second opinion is a good way to confirm what you know, but also find out what you might have overlooked.
How to Come Up with a Project Estimate
Here are some general guidelines for crafting a solid estimate.
1. Break the Project into Tasks
Break the high level task down into the individual tasks and investigate each of those things. Smaller things are easier to estimate, and it’s easier to give concrete estimates for them. For example “add a method to the controller” is simpler and going to have a better estimate than “write an application for checking mobile apps are valid.”
Also, if any single task gets too big, the things you’re guessing about it will probably be too large and abstract and therefore you’ll miss things – so you’ll get a better estimate if you break it down into simpler sub-tasks.
2. Determine the Overall Objective
Is the product objective or outcome of this task clear?
If not, it will take longer because you’ll have to go back and find out the information you need (and wait for the person to work out what they want), or you’ll deliver something only to be told that it’s the wrong thing and you’ll have to do it again. You should try and avoid this…
Is it quick to know whether the task has been completed, or do you need to wait to know the outcome (e.g. is it a daily cron job, do you have to wait for Product to do something, etc)
If you get fast feedback, then as soon as you see it’s working you’re done. Otherwise you have to wait for someone else and then you might have to fix bugs and wait for them and fix more bugs and wait for them and so on and so forth.
3. Identify Outlying Factors Needed for Success
Does the project have tests?
If not, it’ll take you longer because you’ll have to spend more time manually testing or fixing bugs.
Is the CI/CD config up-to-date?
If not, it’ll take you longer because you’ll need to fix the config.
Do you have access to all resources needed to complete the task? (app configuration, external services, etc.)
If not, you’ll have to wait to get the information you need and it might be wrong and it might make it harder to debug problems and it might mean that it takes longer to fix problems (because you have to wait for someone else to change the thing again.)
Does your task have any dependencies or anticipated code conflicts?
If you have to wait for someone else to provide something, they might be late and that could cause your task to be later.
If you’re working on a code path that you know someone else is working on or has had a lot of changes recently (so someone else might be working on it), their changes might break yours, then you’ll have to work together to sort it out and this will probably take longer than expected.
