No. Estimates have a very real purpose, even beyond communicating to managers and customers (whom you should be honest with anyway). Our estimates are used as a team to figure out how much work we can commit to in a sprint. They are also used to determine if you are getting better at things. When we start a project, we don't really have a code base, and many tasks are "new". I expect my estimates to be really wrong at first, and then when I'm implementing things that are very similar to what I've done before, I expect those estimates to get better. I can't really measure that if I'm producing bullshit CYA estimates.
my rule of thumb was if it takes 4 hours to develop, I'll always said 3 days, as then You account for some unknown, testing, building / deploying, maybe some refactor after review, etc esoecially as it was for a product that will run for 10-20 years, with changes allowed like once per year. In 60-70% I'd finish in 1-2 days, but in those 30% it's always good to have extra day available for additional development or debugging or anything else or You can always tell You're finished ahead of scedule and feel really good about Yourself! :D
My go-to reference for this is Evidence-Based scheduling. In a nutshell, the methodology is to NOT change how you estimate—why fight the behavior you're so accustomed to—and instead measure how past estimates differed from actual delivery time. Over time, the model you train can become very accurate at estimating and—importantly—does not require you to do anything different than you normally would. Link [0]
Now here's the other important thing! You must realize that there are multiple types of tasks and they require different kinds of estimates. You can use the Cynefin framework[1] to categorize problems into one of a few domains:
For problems in the Simple domain ("known knowns") you can give an exact number on how long it will take, because the solution is known to you and there's absolutely no uncertainty.
Problems in the Complicated domain ("known unknowns") have some up-front questions that can be answered, and you have the capacity to answer in the requisite time—these problems you give an estimate on, because you don't literally know how long it will take but you have an idea.
Problems in the Complex domain ("unknown unknowns") have a high amount of uncertainty. You must at least start work on them in order to fully understand the problem and the realm of possible solutions. Use Agile here and let the client know that there's multiple steps involved and it's very hard to give a good estimate. Give the client the estimate for the first step and let them know you'll be able to get back to them with more information. If they _require_ an overarching estimate, think really big so when those "unknown unknowns" show up they can't bite you in the ass.
Why You want it as a list? You're doing some sort of research or just out of personal interest?
My $0.02 on this is the more You understand the problem You're solving and details, and more are using the tools and technologies You have experience with, the more correct Your estimates will be. No book or methodology will help You with that!
The more unknowns, the more Your estimate will be off eventually.
Breaking things down into smaller parts will help you estimate better.
Also, sadly, just getting more experience will improve your estimating. Don't expect to be very accurate when you start. Most young developers, myself included, think "oh that will be no problem at all" and lowball everything.
The usual problem with this is that we tend to estimate based on difficulty, not effort.
Building 10 web pages is "easy" because we know how web pages work, no problem. So we allocate half a day per page because it's just HTML, right?
Designing a user interaction path through 10 different application functions isn't hard, but it takes a lot of communication and effort. Half a day per function is a tiny fraction of the effort required.
Great topic. Given the fact that we, developers (as humans), are terrible estimating in time tasks with a high degree of uncertainty (most cases in software development), the most effective way I have found after many years in the field is using velocity (based on effort points) as a (project-dependant) predictor, while applying Scrum, as described in the following article in detail:
As a rule we have a few techniques that have produces our most accurate estimates:
One: since we're a team we get more than one person to estimate independent of each other, then we talk about it. If you're solo, you can still have a trusted friend do the same.
Two, we measure the uncertainty: client says they have a file ready to go once you sign. Client hasn't designed the login page. Client has spec but like most specs it's under specified. Each piece is going to take time to unravel. We try to make sure we look at that uncertainty when estimating.
Three: we always remember that project management takes up tons of time. Even if there is an outside manager you have to manage your workload and communicate to your client. Make sure you factor it in.
If you’ve done the task before, double the estimate. If you’ve never done it before, 10x the estimate. Break tasks down to granularity of 1-8 hours. No more than that. Estimate time != real time because life, meetings, other projects.
>Break tasks down to granularity of 1-8 hours. No more than that.
I do this for sub-tasks that are relevant to myself only (for example, "create function to do X"), but my minimum estimates are 1/2 a day (in extreme cases like a 1-line fix) or 1 day for anything that goes to a manager.
Points are a good way to estimate complexity, but it does not help communicate with the business stakeholders. They don't understand points. There must be a way to convert points into days. Direct estimations in days almost always miss by a lot.
Have been doing this for about 8 years. As others have mentioned here is something you can follow:
1: Break tasks into as small as you can, say a few hours to a day.
2: Add time estimates for writing the main code, writing tests, doing integration test and code reviews separately.
3: Also take into the account the time which might be wasted to unknowns, things which you know needs to be done but do not have much idea on how you would do that. One way to refine this would to be to talk to other developers who might have some experience doing this part of the task.
4: Account for 'unknown unknowns' these are the tasks which will show up but you can't really predict these upfront. Any software developer who has done commercial software can tell you that for every project 10-20% time is used in this which we did not even anticipate. These things always show up.
Now if you break down tasks into these parts, add up the time and then add a 'buffer' say 10-20% on it and give your estimate. I am sure you will face pressure to reduce estimates, but having a break up allows you to provide some support to the numbers you come up with.
This approach seems very time consuming at first, but the first step itself starts to show you the ROI on the invested time.
When I start breaking down a task into smaller pieces, I often come across things I had not thought of when just thinking about the monolith. "Oh, I'm using X here, but where am I getting X from? That's another task."
The problem with estimates is that it's easy to install (and easy to estimate the time involved installing) a new air conditioner (or water heater or other applicance), but it's hard to estimate how long it will take to design/invent a new air conditioner, etc.
In software, the frustration between customers/management and developers comes from this misunderstanding. They think they're asking for a simple install of a thing (which has known risks), but they're often really getting the invention of a brand new thing (with unknown risks).
So my solution is to only give high-confidence estimates for things I've already done once. (e.g. I know it takes me 4 hours to assemble and ship one of my robots. I know because I've done it 100 times. But I have no idea how long it will take to design a new robot using unfamiliar parts from scratch.) But I do try to clearly point out where things are "installations" and where things are "inventions", with the appropriate margin-of-error for each kind of task in the estimate.
Programming is a creative task, it cannot be strictly estimated. Somebody mentioned Evidence-Based Scheduling, and it's the best way to estimate ("doing X is similar to Y, and it took Z hours, so it will be unspecified amount of time, but probably around Z hours"
But maybe significantly longer. Programming is a little special, because if you done it before, then doing it again takes no time, you just reuse the code from last time. So you're always doing something you never done before, or at least in a different context.
Seconded, it's a good book. It doesn't only cover how to estimate but also, to some extent, whether to do so. Finally, it allows you to get very far just reading the first few chapters, a must-have for technical books IMO.
My biggest takeaway from reading this book is that making point estimates is hard and often unnecessary. Saying "umm somewhere between 2 days and 2 weeks" is a much better response in many situations as it conveys to the listener that there's a high degree of uncertainty.
In my experience, the bigger the project, the bigger the multiplication factor. On a big enough scale, multiplication by 2 might be not enough. Most often I find that implementation times can be estimated quite reasonably, but time for testing, documentation, team-coordination and communication are not factored in properly.
It is true that we suck at estimating. Doubling helps you get closer since you always think you'll be less busy and have less distractions. You can also get really good estimates by breaking all tasks down into things that are less than one day. However, this takes time as you are really doing investigation that is part of the task itself to figure out how many sub-day tasks you need for a bigger task.
In the end, estimating is bad for how people use it (imo). It is used to make gantt charts and plan release dates which are useless. It is much better as a personal tool to track how effective you are at turning concepts into working products. If you track time, it should be so you can reflect and improve.
We all really need to move to prioritized iterative development (not scrum) where the team is working on the highest value items in prioritized order. This also means cross training so people don't say things like "Jill doesn't know the image processing code so pick so other (lower priority) task". If you are always completing the most valuable tasks, everything else will come out in the wash.
As others have suggested: Start by breaking down the task into the smallest part possible.
The absolute best way of estimating development task we've seen was suggested by PHK (of FreeBSD fame), or at least I remember it as one of his suggestions. Basically you take your absolute best guess on a task, multiple that by two, and move up one unit of time.
Example: My guess for task X is 2 hours.
2 hours times 2 is 4 hours
Move up a unit, that's 4 days.
My guess for two hours can realistically be completed in 4 days.
This prevents you from taking on to large tasks, guessing that you could bang out a project in 5 weeks would realistically be a 10 month project. A two year project will be done in 40 years (4 decades).
It not a realistic way of estimating time of cause, but it illustrates the dangers of tackling to large tasks.
We work in smaller teams in a scrum like environment. Our tasks are small ranging from 2-7 days mostly. To make it easier for us developers to communicate estimations higher up, we do following :
* Split work as small as possible, and atomic so that aborting/undoing is possible if required. Like all teams we do throw away a lot of stuff.
* If a task contains 5 bullets (best case), we would only give an estimate for the bullets that we know for sure can be done without further dependencies and complications. We refrain from providing an overall estimate in such cases.
* We often proceed with starting a task, and as the task progresses and more clarity emerges, we revise the estimate. Importantly we are encouraged to raise flags as early as possible.
* We have dev servers that we spin out frequently to stage things that are in progress to test. They may break but give us early feedback which is crucial. Sometimes helps to stop a task earlier rather than later.
Still not a perfect system, but works for us without causing too much friction.
Don't overthink it. Do only as much estimation as you need, not more - relative estimation ("project A will take twice as long as project B" ) is often much easier than absolute estimation, and is often all that you need to base decisions on.
Not a list of resources - but my rules (to myself) for estimating are below. My personal bias is that I really dislike under-estimating, and I typically over estimate how long tasks will take (by design) - but usually it's pretty close (i.e. not an order of magnitude off).
1. Usually, the smallest unit is a week. When people say something will take a day, it's usually wishful thinking - they forget about meetings, other commitments, etc.
2. Think about what "done" means for individual tasks when estimating. Often more junior engineers will give estimates where "done" means the code is submitted to source control - but really think about the work that it'll take to get that code into production. Testing, configuration, documentation, monitoring/metrics/alerts, etc. I think a lot of under-estimating comes from this, where the estimates are just for writing the code, and all of the other stuff comes either as a scramble at the end, or a ton of time to actually launch as tons of bugs and missed features are discovered late in the game.
3. Think in latency and throughput. Estimates typically have both baked in - if something takes a week of "at-keyboard' time, it might take a month to get done because of other commitments. Both estimates are important.
4. Think in probability distributions rather than absolutes. I usually set my threshold around 90% confidence - i.e. my estimates are based on where I am 90% confident I can have something done. When you think about how long something takes, if the stars align, a 10 day task might take 5 days, but if things go wrong, that 10 day task could take several months or more. Things can only go a little bit better than expected, or a lot worse than expected. But when people make estimates, I think they are often biased toward the "stars align" case, and forget about this fat tail.
5. With team estimates, add a friction value to longer term estimates to account for vacations, sick days, distractions, etc. Usually around 30% seems to work. So that one month task (20 days) I plan for it taking 20 * 1.3 = 26 days.
6. Work backwards from the end goal. This forces you to factor in things like testing and productionalization. This also helps with coordination too, as you can figure out when to parallelize tasks to get to the end goal.
7. Write estimates down, and go back and reflect. When something that you thought would take a week took a month, sit down and really think about why. This will help you recognize and refine your biases.
I normally look at the task and have a min of 2 hrs -
> 2 hrs - Simple Tasks - text change, update or insert. The estimate includes any admin on top
> 4 hrs - Slightly harder tasks - change method, single new method - inc unit testing, and admin
Then it gets more complicated - I try to break down into the above problem sizes, and attach a confidence level to the estimate of low, medium, high and attach a contingcy of 1.25(high)-2(low)
I try not to spend too long on the estimates, unless its a later iteration as the initial round often requires redesign
72 comments
[ 4.8 ms ] story [ 114 ms ] threadThere are simply too many factors to consider. The team, the client, the project.
I usually follow this rule: take the worst case scenario estimate and then double it :)
https://books.google.com.au/books/about/Software_Estimation....
EDIT: And don't forget the Atwood estimation method. "It'll take between 6-8 weeks". 60% of the time it works every time.
We are much better at looking backwards than we are at projecting forwards
Now here's the other important thing! You must realize that there are multiple types of tasks and they require different kinds of estimates. You can use the Cynefin framework[1] to categorize problems into one of a few domains:
For problems in the Simple domain ("known knowns") you can give an exact number on how long it will take, because the solution is known to you and there's absolutely no uncertainty.
Problems in the Complicated domain ("known unknowns") have some up-front questions that can be answered, and you have the capacity to answer in the requisite time—these problems you give an estimate on, because you don't literally know how long it will take but you have an idea.
Problems in the Complex domain ("unknown unknowns") have a high amount of uncertainty. You must at least start work on them in order to fully understand the problem and the realm of possible solutions. Use Agile here and let the client know that there's multiple steps involved and it's very hard to give a good estimate. Give the client the estimate for the first step and let them know you'll be able to get back to them with more information. If they _require_ an overarching estimate, think really big so when those "unknown unknowns" show up they can't bite you in the ass.
0: https://www.joelonsoftware.com/2007/10/26/evidence-based-sch...
1: https://en.m.wikipedia.org/wiki/Cynefin_framework
My $0.02 on this is the more You understand the problem You're solving and details, and more are using the tools and technologies You have experience with, the more correct Your estimates will be. No book or methodology will help You with that!
The more unknowns, the more Your estimate will be off eventually.
It's simple really! :D
Also, sadly, just getting more experience will improve your estimating. Don't expect to be very accurate when you start. Most young developers, myself included, think "oh that will be no problem at all" and lowball everything.
There is a good question on StackOverflow to check out https://softwareengineering.stackexchange.com/questions/648/...
Building 10 web pages is "easy" because we know how web pages work, no problem. So we allocate half a day per page because it's just HTML, right?
Designing a user interaction path through 10 different application functions isn't hard, but it takes a lot of communication and effort. Half a day per function is a tiny fraction of the effort required.
"Why Asking Developers for Time Estimates in Software Projects Is a Terrible Idea and How to Bypass It With Scrum": http://www.romenrg.com/blog/2015/09/28/why-asking-developers...
As a rule we have a few techniques that have produces our most accurate estimates:
One: since we're a team we get more than one person to estimate independent of each other, then we talk about it. If you're solo, you can still have a trusted friend do the same.
Two, we measure the uncertainty: client says they have a file ready to go once you sign. Client hasn't designed the login page. Client has spec but like most specs it's under specified. Each piece is going to take time to unravel. We try to make sure we look at that uncertainty when estimating.
Three: we always remember that project management takes up tons of time. Even if there is an outside manager you have to manage your workload and communicate to your client. Make sure you factor it in.
I try to avoid working with people/teams where I'm forced to make this kind of estimate. It's a great selector.
I do this for sub-tasks that are relevant to myself only (for example, "create function to do X"), but my minimum estimates are 1/2 a day (in extreme cases like a 1-line fix) or 1 day for anything that goes to a manager.
1: Break tasks into as small as you can, say a few hours to a day.
2: Add time estimates for writing the main code, writing tests, doing integration test and code reviews separately.
3: Also take into the account the time which might be wasted to unknowns, things which you know needs to be done but do not have much idea on how you would do that. One way to refine this would to be to talk to other developers who might have some experience doing this part of the task.
4: Account for 'unknown unknowns' these are the tasks which will show up but you can't really predict these upfront. Any software developer who has done commercial software can tell you that for every project 10-20% time is used in this which we did not even anticipate. These things always show up.
Now if you break down tasks into these parts, add up the time and then add a 'buffer' say 10-20% on it and give your estimate. I am sure you will face pressure to reduce estimates, but having a break up allows you to provide some support to the numbers you come up with.
Hope this helps.
When I start breaking down a task into smaller pieces, I often come across things I had not thought of when just thinking about the monolith. "Oh, I'm using X here, but where am I getting X from? That's another task."
adjust task size accordingly
https://www.youtube.com/watch?v=qtj7gVRjdZI
This is a running gag in the series.
In software, the frustration between customers/management and developers comes from this misunderstanding. They think they're asking for a simple install of a thing (which has known risks), but they're often really getting the invention of a brand new thing (with unknown risks).
So my solution is to only give high-confidence estimates for things I've already done once. (e.g. I know it takes me 4 hours to assemble and ship one of my robots. I know because I've done it 100 times. But I have no idea how long it will take to design a new robot using unfamiliar parts from scratch.) But I do try to clearly point out where things are "installations" and where things are "inventions", with the appropriate margin-of-error for each kind of task in the estimate.
But maybe significantly longer. Programming is a little special, because if you done it before, then doing it again takes no time, you just reuse the code from last time. So you're always doing something you never done before, or at least in a different context.
My biggest takeaway from reading this book is that making point estimates is hard and often unnecessary. Saying "umm somewhere between 2 days and 2 weeks" is a much better response in many situations as it conveys to the listener that there's a high degree of uncertainty.
2 hours = 4 days
3 days = 6 weeks
2 weeks = 4 months
In the end, estimating is bad for how people use it (imo). It is used to make gantt charts and plan release dates which are useless. It is much better as a personal tool to track how effective you are at turning concepts into working products. If you track time, it should be so you can reflect and improve.
We all really need to move to prioritized iterative development (not scrum) where the team is working on the highest value items in prioritized order. This also means cross training so people don't say things like "Jill doesn't know the image processing code so pick so other (lower priority) task". If you are always completing the most valuable tasks, everything else will come out in the wash.
The absolute best way of estimating development task we've seen was suggested by PHK (of FreeBSD fame), or at least I remember it as one of his suggestions. Basically you take your absolute best guess on a task, multiple that by two, and move up one unit of time.
Example: My guess for task X is 2 hours.
2 hours times 2 is 4 hours
Move up a unit, that's 4 days.
My guess for two hours can realistically be completed in 4 days.
This prevents you from taking on to large tasks, guessing that you could bang out a project in 5 weeks would realistically be a 10 month project. A two year project will be done in 40 years (4 decades).
It not a realistic way of estimating time of cause, but it illustrates the dangers of tackling to large tasks.
* Split work as small as possible, and atomic so that aborting/undoing is possible if required. Like all teams we do throw away a lot of stuff. * If a task contains 5 bullets (best case), we would only give an estimate for the bullets that we know for sure can be done without further dependencies and complications. We refrain from providing an overall estimate in such cases. * We often proceed with starting a task, and as the task progresses and more clarity emerges, we revise the estimate. Importantly we are encouraged to raise flags as early as possible. * We have dev servers that we spin out frequently to stage things that are in progress to test. They may break but give us early feedback which is crucial. Sometimes helps to stop a task earlier rather than later.
Still not a perfect system, but works for us without causing too much friction.
Multiply it by 2.
1. Usually, the smallest unit is a week. When people say something will take a day, it's usually wishful thinking - they forget about meetings, other commitments, etc.
2. Think about what "done" means for individual tasks when estimating. Often more junior engineers will give estimates where "done" means the code is submitted to source control - but really think about the work that it'll take to get that code into production. Testing, configuration, documentation, monitoring/metrics/alerts, etc. I think a lot of under-estimating comes from this, where the estimates are just for writing the code, and all of the other stuff comes either as a scramble at the end, or a ton of time to actually launch as tons of bugs and missed features are discovered late in the game.
3. Think in latency and throughput. Estimates typically have both baked in - if something takes a week of "at-keyboard' time, it might take a month to get done because of other commitments. Both estimates are important.
4. Think in probability distributions rather than absolutes. I usually set my threshold around 90% confidence - i.e. my estimates are based on where I am 90% confident I can have something done. When you think about how long something takes, if the stars align, a 10 day task might take 5 days, but if things go wrong, that 10 day task could take several months or more. Things can only go a little bit better than expected, or a lot worse than expected. But when people make estimates, I think they are often biased toward the "stars align" case, and forget about this fat tail.
5. With team estimates, add a friction value to longer term estimates to account for vacations, sick days, distractions, etc. Usually around 30% seems to work. So that one month task (20 days) I plan for it taking 20 * 1.3 = 26 days.
6. Work backwards from the end goal. This forces you to factor in things like testing and productionalization. This also helps with coordination too, as you can figure out when to parallelize tasks to get to the end goal.
7. Write estimates down, and go back and reflect. When something that you thought would take a week took a month, sit down and really think about why. This will help you recognize and refine your biases.
> 2 hrs - Simple Tasks - text change, update or insert. The estimate includes any admin on top > 4 hrs - Slightly harder tasks - change method, single new method - inc unit testing, and admin
Then it gets more complicated - I try to break down into the above problem sizes, and attach a confidence level to the estimate of low, medium, high and attach a contingcy of 1.25(high)-2(low)
I try not to spend too long on the estimates, unless its a later iteration as the initial round often requires redesign