Yes, this is a thesis I've brought up quite some times in relation to sprint planning. The odd misjudgment having a strong influence on the total estimate biases upwards, not downwards.
One take away is for sprint planning - if there is a reasonable amount of uncertainty on a task, take it out, either break it down or do some investigation, and bring it back next time.
You don't want a σ=2 task messing up your sprint!
I believe this is the reason why scrum uses story points instead of time estimate. By putting uncertainty on the same level as effort, you give it more weight. And using a fibonacci sequence rather than a continuous amount with the rule you should round up if unsure tend to correct those defects.
That happens indeed, but the point (!) stands: the fibonacci scale ensures estimates are rounded up, and that small abstraction merges time and complexity into a single number, whereas it feels more difficult to bump a time estimate based on unknowns/complexity.
That's what happens, its all tranlated to hours and cost instantly. I have seen that at several places during poker sessions. The fun starts when management hears this and starts to calculate with it.
there is an uncharitable response to this thread, which is marked dead. I do think there is a kernel of truth in that response, which is nothing more than mocking.
why fibonacci? I think it's reasonable to say time includes an exponential error. an estimated 1 hour task is very different than an estimated 1 week task. i see that 1 hour, 1 day and 1 week estimates are progressively, and likely exponentially, worse.
Is it just the ease of doing the math? (totally reasonable answer, in my humble opinion). or is there something specific about fibonacci that's actually relevant? I think it's the former not the latter. but if you have any evidence to the contrary, i'd love to hear it.
The kernel of truth of that response is that you won't get more certainty about an estimate by adding more uncertainty, no matter in how much pseudo-scientific cargo-culting mumbo-jumbo you cloak it. (Unfortunately, HN readers don't really like sarcastic commentary which requires reader to think it through. Time is precious, and we just want to be coddled, give me your opinion ELI8 straight, because there is no time to think!)
I think you're half right. adding an error band to a far out estimate won't give you a better estimate, but it might help tell you how much you _don't know_ about what you're facing.
Cumulative error is _hard_. see weather prediction.
The reason the Fibonacci sequence is used is because we don't think in an incremental scale but in a logarithm scale. In other words, we think that some simple takes 1 day, a more complex takes 3 and a even more complex takes 5 days. Usually we don't jump from 1 to 2 days when thinking about a task effort.
I kinda want to grab you by the lapels and re-enact that part from Clear and Present Danger. It doesn't matter what's true, it matters what i can prove!
Unfortunately, the phrasing would be more like, It doesn't matter what's true, it matters what i can sell to management.
In the absence of evidence to the contrary, i think i'm ready to take a side close to yours. It's useful to expand error bars exponentially as you move away from the time of the estimate, but there's nothing special about fibonacci.
The reason my team uses Fibonacci is that it cuts down on the “is this a 3, 4, 5, or 6 point ticket?” discussion. Each larger value should have more uncertainty, so there isn’t any point in getting more precise. Not being able to argue over 1 point differences but having to choose between 5, 8, and 13 has a clarifying effect on the process.
It is very easy to answer. Different people have different efficiency, therefore, a task completion does not depend solely on its complexity but also on who is executing it.
You should think about it (even though there are no actual physical units you can use to make this work):
Time to deliver a task = complexity / developer efficiency
This is why it is hard to develop a method to estimate time for development tasks.
If regular time estimation has tricky statistical properties when combining multiple estimates, story point estimation is just hopeless. Especially on a Fibonacci scale. When estimating, nobody thinks that an 8-point story is the same as eight 1-point stories. Even the metric itself is nonlinear!
I push back pretty hard on story points whenever I see them. They’re not useful until several iterations in, once you have a baseline for how many points can be delivered in an iteration, and even then one developer’s 1 point story is another’s 5 point.
When estimating I tend to use a coarse scale of hours, days, weeks, and (in extreme cases) months. All estimates then get turned into a range between the minimum and maximum the estimate could mean - for example hours becomes 1 hour to 8 hours, days becomes 1 day to 5 days, and so on.
Anything estimated as weeks or months probably needs scoping until it can be broken down into tasks that will take between an hour and a week.
I find this method works well because it’s intuitive, comes out with a suitable amount of uncertainty relative to the size of the task, and finally results in estimates with units that are useful to project managers and the wider business in scheduling other work needed to get a feature into production.
An approach that gives very similar results to yours but is more scientific is to multiply the initial estimates by e (2.718). Or, if you're conservative, then by pi.
Extended nitpick: Technically, they only said "then add 20%". They didn't specify of what that 20% is. It's up to interpretation whether they mean 20% of the original value or the doubled value.
Semi-seriously, e is a number defined by recursive growth.
So if you estimate to Level 1, and then break down to Level 2 sub-tasks, then Level 3, your estimate will be out by the same amount each time. and your total error will be a multiplicative series.
If you have an infinite number of levels - which is reasonable for most real-world projects, although this may not be obvious in the planning stages - e will get you nicely in the ballpark.
Eh. I've been successful adding 20% in. I feel like if you have to double first (meaning your end result is 220% of what you originally estimated) then you aren't learning from previous mistakes. Maybe 220 is appropriate for the first time you do work or work with a certain team tho.
220% also works fine if each one of the projects is "practically unique" to you / the team... this happens >50% of the time in the real world, imo, because some fields have very little transferable skills/experience from project to project... a large percent of stuff is genuinely new every time and never repeats even if it's "basically the same thing as that other project"! (Or team composition changes.)
Problem is the fact that you can only justify with a straight-face 45% (100%/220%) of the estimate you give to the client... Then you need to get clever at "bluffing" your way at "inventing" another extra +35% of work that you can't imagine/explain, but you know for sure that will come out of the blue - you know there's this extra 35% work needed for sure, you just have no idea what this work will be. Then finally you can say with a straight face that 20% of estimated time is for unforseen circumstances. There's two ways to see the same math:
The world would be a much better place if clients / managers / project owners etc. would just accept that 55% of time & cost for software dev is unknown-unknowns type of unexplainable/unimaginable-at-project-start-stage-work. That's just how it is! If the problem was really clearly defined and understood, there would likely already be an off-the-shelf software solution for it that could be used instead of paying developers to build custom software!
If we could just be honest with people and show them formula (2) instead of a make-up-ladden version of (1). But people don't, so we get to play the damn elegant-bullshitting game over and over again...
>Then finally you can say with a straight face that 20% of estimated time is for unforseen circumstances.
This is how I always have done it across multiple jobs and I always called that extra contingency. Depending on the project I sometimes had to ask for over 50% of the total as contingency. If anyone queried it I would explain that it is very likely we'll need it based on my experience of many other projects.
Then the client would either accept it as is, or they would want to split the proposal into "what we will definitely pay for" and "what we will pay for only when it is actually needed" which is also fine for both sides if proper progress reporting has been agreed in advance.
Someone could think that no client would agree to such a contract as it would open them to be taken advantage of by an unscrupulous supplier, but at the end of the day what matters in business is trust. If a client doesn't trust the supplier they shouldn't hire them. And in all my previous projects I never met an engineer or developer that would deliberately use up the contingency time when it is not needed as everyone knows the availability of this extra time might come very important towards the end of the project.
That's a smart way to do it. I used to put in ranges (basically contingency) of rate but I found myself consistently hitting a similar % of the higher estimate and just started quoting the higher estimate with the expectation that I'm not a God and there may be unknown factors which could add in more cost. Usually people are happy as you say if there's trust and you show progress.
The doubling-it is for when you build the whole component on top of a library, then discover that the library has a fatal bug you can't work around, and you have to rebuild the whole component on top of a different library, then discover that other library has another fatal bug, so now you need to include both libraries with logic around when to use which one, and then that seems to work but you add plenty of tests and documentation to make sure it works that way in prod too and not just on your dev machine.
I don't see what that has to do with learning from previous mistakes. Pretty much all but the most basic programming turns out to be like that -- dealing with unforeseeable and undocumented problems.
(The 20% is because you weren't planning for sick days, an unforeseen emergency bugfix on another project, nobody remembered about the afternoon retreat next week, etc.)
Going through this now and boy is this true. Nothing like rewriting a 1k+ lib the day before a feature is supposed to be ready and then praying you didn't blow up some other lesser documented part of the code.
Yeah and I've been pretty good at estimating what to expect. I mean you'll always find surprises but personally I'm more inclined to quote a lower estimate and come back explaining the situation than to overcharge. But if you know of any clients who are throwing money around like that I'll gladly accept your 220% estimate for a few referrals ;)
One of my funny-not-funny jokes is "Double it then change up to the next unit of measure" So "I can do that in 3 minutes!" becomes 6 hours. 2 hours become 4 days. 3 days becomes 6 weeks. 2 weeks becomes 4 months...
A professor of mine at university taught us the same thing. It's one of the most valuable things I learned and it is uncannily accurate. Even after 30 years in industry I still fall back on this formula with great results.
Sadly, managers want to believe that you put more effort into estimating deadlines so I'll just whip up a gannt chart retroactively based on the 2/20 rule and they're happy.
> Even after 30 years in industry I still fall back on this formula with great results.
Isn't this Parkinson's law in action, i.e. work expands to fill the time available for its completion? That would mean you overestimate up to 2.4x each time.
> work expands to fill the time available for its completion
Oh, so if I estimate every task as 1 minute I'll finish really quickly?
The 2.4x is to overcome one's innate overoptimism (even if you see yourself as a pessimist), and to add contingency for unplanned complexity, bugs and general overhead.
It's a rule of thumb, not an immutable law, so if one has completed a similar task many times then less padding is needed.
My number is 5. Maybe my estimates tend to be a little optimistic :-). But in general I can look at something for a few minutes, make a quick estimate (maybe ask somebody else), multiply by 5 and be pretty close to the eventual time it will take.
I close to double mine, but it also depends on the tasks. Some tasks I just know I'll complete in 1X my estimate. I do have a rule that nothing is ever estimated at less than 2 hours; no matter how small the task. Invariably, any time I have broken that rule, someone has checked in code that won't build (or something similar) and I have to spend time dealing with that.
I notice some devs like to underestimate to make them look more productive. Most of those tend to spend a lot of time fixing bugs that QA pushes back to them. They sometimes spend ever more time arguing that it's not their bug.
If that 20% isn't for project management, then add another 10-20% for that, based on experience from previous projects for the same stakeholders. My similar rule of thumb is to multiply my estimate by 2.5, before adding the project management percentage. Of course, if it's something we've done before, it's more along the lines of 1.5 to 2.
This is why I like 3 point estimation[1] - if you have optimistic, expected and pessimistic estimates for each task you can pull out which points are high risk. Using a single estimate can't give you that insight.
Sure, in the same way a bad manager will say "No, that's too high, I'm going to reduce your estimate" if you use a single number. Bad managers are a thing. Make sure you get a good one.
I’ve had a lot of success using this method, but it does take having management that understand how software development works. A really good PM will look at an estimate like this as a range, with work complete somewhere between the lower and upper bounds. A bad one will take the optimistic estimate as gospel and build everything else around that.
I've always thought that typically overoptimistic estimates tend to be more based on the mode. I.e. 'this is the sort of task that normally takes 1 day, so I'll estimate 1 day'. The high point on the probability curve is the most noticeable, but it's also way further to the left than either the mean or the median.
I once had a really convoluted metaphor for estimation which involved opening boxes that sometimes contained other boxes which sometimes contained other boxes... I wonder how that models mathematically.
Well, in the most extreme form, you'd wind-up with something like the poisson distribution [1], where your chance of finishing each week would be the same.
This is not entirely implausible but it produces effects even more paradoxical/pathological than log-normal. Here, you could give a correct estimate of the expected time for a project to complete as 50 weeks. You could then reach week fifty and again give a correct estimate of 50 more weeks required. And then you could finish the next week.
US navy has developed something similar with beta statistical distribution. You estimate "Optimistic", "Most likely" and "Pessimistic" time estimates for each task in the project and then use beta distribution on it. Some tasks take way longer than estimated.
I've seen a similar model that I've adopted for my own projections which is 30/60/90 - the 30% chance scenario which is highly successful but less likely, 60% chance scenario that is good with a relatively decent chance of success, and the 90% chance scenario that is the lowest outcome but will almost certainly happen.
I do this but only with best and worst case. Any estimate with a worst case of more than 5 days need to be broken down again unless it's a known quantity. Thanks for the link, I had no idea this was part of 6 sigma
I used it in the past and I think it is a good framework to discover uncertainty and make it more visible, because it makes you talk about optimistic and pessimistic cases.
The estimates are also good enough to come up with a draft schedule.
I find this approach very interesting, but it hinges on the assumption that project completion times follow a beta distribution. What's the basis for that?
It may reflect the observations in OP kind of well -- with one difference: it assumes we're good at estimating the mode, not the median. But other than that, within the range we're talking about (1 < alpha < beta) it has somewhat similar shape to the lognormal distribution.
The three things that still bother me about that idea are:
1. I haven't tried fitting it to the dataset in OP;
2. It's bounded to the right, which seems unrealistic;
3. I haven't come up with intuitive interpretations for the alpha and beta parameters in this context. If the beta distribution means something, then its parameters must have natural interpretations as well.
Something I’ve been trying is estimating for the time in which I’m 80% sure I can finish something. I end up racing ahead of estimates most of the time because they are too high, but then some times I’ll find something that is a bit more complicated than expected and it takes a little longer. Overall this seems to balance out, but also have a lot more predictability, it’s easier to predict at any given time what I might be working on. This has been pretty important on my team where I’ve been doing API work for some iOS developers to use.
Honestly, after doing this whole data science thing for a while now, I'm going to be blunt: I can estimate quite a lot of tasks with quite a lot of accuracy. including software and IT tasks.
What I can't do is make bad management hear what they don't want to hear. Nor can I stop people from accepting an estimate because its closer to what they want to be true, because they've already made a promise that conflicts with the actual estimate, or because a certain process requires or necessitates inaccurate estimates.
I think the whole "software is hard to estimate" myth stems from 2 fundamental causes:
- not controlling for human biases or referencing actual real world data
- processes that don't punish/ reward people who provide inaccurate/accurate estimates respectively.
An important aspect of being a professional software engineer is having the backbone to sometimes say things like:
- “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.”
- “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chinese and Google Translate isn’t good enough. I’d need to insist on professional translation in order to even estimate the second part.”
- “The problem is tracking down a bug rather than building something, so I don’t have a good way of estimating this. However, I can timebox my investigation and if I’ve not found the cause at the end of the timebox, can work on a plan to work around the bug.”
You need to be willing to endure the discomfort of looking someone in the face, saying “I don’t know”, and then standing your ground when they pruessure you to lie to them. They probably don’t want you to lie, but there is a small chance that they pruessure you to. If you don’t resist this pruessure, you can end up continually giving estimates that are 10x off-target, blowing past them as you lose credibility, and your running your brain ragged with sleep-deprivation against a problem you haven’t given it the time to break down and understand.
But when you advocate clearly for your needs as a professional, people are generally reasonable.
Imagine a tradie saying it'd take 1week to finish remodelling your kitchen, and then it turns out they couldn't. As a professional, it's your responsibility to educate the client on the issue - most technical problem could be explained to a layman if the context is set up properly.
The problem occurs when the client doesn't want to hear it.
This is another part of the whole 'communication with clients' thing. You have to be clear whether you're talking coding time or wall-clock time. To be safe, give both.
"Yes we can do that in three weeks. We need the API documentation for _those_ two systems, network access to _those_ resources, and credentials to log in to _those_ services first."
2 months later we're still waiting on critical pieces of those prerequisites, while someone's screaming that the deadline they'd committed to with the customer was 5 weeks ago.
Oh yeah. This is why I'm always very careful about quoting prerequisites for any deliverable. "We can do X in 5 weeks after receipt of Y, Z and W." Then when they say "why don't we have X," I politely inquire after Y, Z and W.
The most important part of this process is getting it in writing so that you can slap them with it later.
That's the difference between a research project and an implementation project. Research, you don't know Y, Z, W, or Q. Implementation, you know the full path from spec to deliverable.
Which is part of why I think the most valuable part of an estimate isn’t the number. Its the task breakdown and identification of risks.
1) You improve your foresight so you can get your manager rolling on getting resources early.
2) You have a concise record of what you think you need. If you notice something else mid-project (which does happen even in otherwise-straightforward implementation projects), you have a concrete point at which to raise a flag to stakeholders.
If you have zero confidence in an estimate from the beginning, then there is no point where its natural to realize “oh hey, this is going to overshoot and I need to communicate about it.”
The other thing you can always refer back to is that the accuracy of an estimate increases the closer it is to finishing it - that is, it's very inaccurate at the start. This is one of the pillars of the agile way of working - you and your team can give a fairly accurate estimation for what can be finished within a week or two. Not so much for a year.
The longer estimations are very much due to changing priorities and a developing environment and understanding of what is being built though, hence the inaccuracy.
Right, thats why steeling yourself for the possibility of having to deal with unreasonableness is important. If you run into it, the first step to dealing with it is to get someone to explain what they expect clearly, in relevant detail, and non-hyperbolically. This short-circuits the possibility of mere misinterpretation. They might have just been expressing their emotions but not yet grappling with their desires in concrete terms—doing so might bring them back down to earth. Then you can resume jointly problem-solving.
If that doesn’t work...I’m actually not sure. But I just started reading the book Crucial Conversations and it claims to have a relevant answer.
I once had a manager/CEO who thought you can negotiate the planning, the same as like a price. "The customer always wants to get it earlier, the engineer later". Me saying that only the developer is able to give a realistic planning, was probably received as "developer wants to push the deadline back in his negotiation".
In the end, his company went bust because he was selling stuff to customers that could never be delivered on time, or even never at all.
There is a type of negotiation that can be done: Negotiation about scope. Work doesn’t get compressed. But stakeholders can identify non-essential business-facing features and cut scope.
Note well: Automated testing is almost certainly essential. If someone says you are “building an MVP”, then make sure it is an actual MVP. Make sure you have a clear market hypothesis to test and check that it couldn’t be built out of duct tape, Google Forms, Zapier, ITTT, manual processes, and excel-wrangling.
Hah! You just described the start of one of our products so accurately it’s scary! And then we built out the parts that were starting to fail as the product scaled. You can get surprisingly far this way!
Testing will happen. The choice you make is who does the testing and when it happens. Internal people testing earlier is generally preferable to external customers testing 'live', but either way, the testing will happen.
“We can’t make those UX improvements to our core open-source product because then we’s have to introduce untested code. The current code is tested because users have used it and reported the bugs and we’ve fixed them.” Is a real thing a project lead said to me at a previous employer.
"Or customers are better at finding the bugs than we are so just give it a quick look over and ship it" - genuine quote from CEO at a company I soon left
Honestly, I'm surprised more companies don't formalize this. There are places that offer customers discounts or feature-request priority in return for being beta testers, which seems like a major improvement in both integrity and efficient feedback.
I suppose it's both slower and more expensive than "just ship bad code at full price and wait", but I'd still expect more companies to choose it as the intermediate option between that and comprehensive in-house testing.
I used to work for an online travel site - before every project kicked off, the business analysts would produce requirements documents that always started with a "flexibility matrix". The flexibility matrix had three rows labeled "schedule", "scope", and "resources" and three columns: "flexible", "moderately flexible" and "inflexible" and they were supposed to check one option for each row. So every single requirements document listed "resources" and "flexible", "scope" as "moderately flexible" and schedule, OF COURSE, "inflexible".
You sound like you're complaining, but your complaint may be incomplete. If those bullet points were honored, that is indeed how pretty much every organization doing development will operate, whether it likes it or not. That is not intrinsically a problem.
I take it these requirements documents were not honored by your non-developers? A document labeling scope as "moderately inflexible" does little if it's treated as written in stone.
One basis for complaint that stands out to me is framing "flexible resources" as a full degree of freedom.
The first question is of course whether that's true, since resources don't tend to flex up unless someone outside the project dictates it. But even if it's accurate, it tends to produce the situation The Mythical Man-Month was written to condemn. If the ship date can't move and the scope won't bend on core features, there's a very real risk of finding that extra resources can't fix the timeline problems.
You're right that most organizations have to operate this way, and it's not inherently bad; I've just found that "flexible resources" is often used as a way to avoid admitting that the real third category is "risk of failure".
Actually it started out as a 2x2 matrix (scope/schedule), and all of them were scope = flexible and schedule = inflexible (of course), and they started adding the "resources = flexible" row so that they could walk back their claims of scope being flexible, because of course it wasn't.
Depends on the number of devs you have working on the project. If you only have 1 then you can flex to 4 pretty easily and still have an efficient team. If you have 7 and flex to 10 then you're just adding inefficiency.
We actually tried asking for more resources when the schedule was in "jeopardy" and were told that there weren't any available. They suggested we try working weekends to make up the shortfall.
Isn't it the same in price negotiation? Only the producer knows the costs. Perhaps the difference is that money can be shifted around more flexibly. Even if I come out with a negative money balance from a negotiation (costs more than the buyer pays), it might be worth it for long term goals. But you cannot borrow time. You cannot do the 1 week project in -1 weeks and then compensate by doing the next 1 week project in 3 weeks and break even.
Price negotiation is not zero sum. Zero sum means one party’s gain is exactly equal to the other party’s loss. Most deals are positive sum - both parties get benefits. Price negotiation is about the seller getting some the upside of the buyer’s benefits but not so much that it sucks for the buyer and not so low a price that the seller will walk.
Some deals are inherently negative sum - if you bought a lemon.
If one thing if you are haggling over a used toaster at a yard sale. It's another when you are negotiating a business relationship where the success of both parties is important. A contract with a supplier doesn't do you any good if the terms are so bad it cripples their ability to execute.
That highlights a beef I have with economics. Failure to understand that business relationships are exactly that. You can't model them using the sorts of time invariant non-network models they use.
First, Not everything is a commodity with perfect substitutes, arguably most things aren’t. Substitutability goes beyond price.
Secondly, benefits are not necessarily equal or a linear function of cost.
Third, whether a game is zero, positive, or negative sum requires a look at the costs/benefits of the whole transaction. Saying that a buyer gains $1 more if the price goes up is obvious, it that doesn’t necessarily make it zero sum when you look at the overall transaction of a $1 price raise. .
Zero sum implies that price, costs and benefits are equal. I spend $20 on a product that cost you $20 to make, and I get $20 of benefits out of it.
Whereas I buy something for $20 , it cost you $5 to make, and I got benefits of $100 out of it, that’s positive sum, as both the buyer and seller got positive gain. If you raise the price, the deal isn’t as great, but it’s not zero sum.
There is such a thing as negotiating too hard on price. The deliverables have lower cost to benefit than if you paid more money. Or worse no deliverables. Worse than that fatal realized liabilities.
I worked with a group that designed telcom IC's. One of our customers developed a new product. Which then failed in the field. The parent company had to issue a recall. Destroyed all remaining inventory and canceled the product. All because someone jacked the microphone supplier down about 5 cents on a part.
In his mind there is one negotiable variable, though - how many hours you work. You want to work 40 hours/week, he figures if he can push you to work 80 hours instead, it'll get done twice as fast. The typical CEO's background as a schoolyard bully gives them plenty of experience in this area.
I'm on a project right now and I am telling people "I don't know how long it will take because nobody here has ever done anything like this. The only thing I can tell you is that nobody is wasting time and we are solving problems as quickly as we can. Look at the JIRA board and see what we have done and what we are planning to do and make your own estimate". So far I haven't been fired. I also refuse to stop work and start long planning meetings like I have seen in previous projects.
I once worked for CEO as a Director of Engineering who was convinced that any product feature could be implemented in 2 days - and this was a non-trivial engineering area.
curious about this person's reaction to things not being implemented in 2 days. did he/she think that people were just dragging their feet? or not as smart? or sabotaging things to make him/her look bad?
He just edited it out from his version of reality! In fact, in all the time I was there (roughly a year including a major new product launch) I don't think anything was ever done in 2 days.
Another interesting characteristic he had was to get upset if he thought projects were running too smoothly...
We have a manager who seems to think that the only way to get good performance is to set impossible deadlines. There are no consequences for missing but it’s pretty unpleasant and stressful to have to deal with this.
That's just a question. Asking questions is OK. If you have such a high need to please other people and such a low self-confidence that you will change your answer simply if someone asks you twice, that's the problem with you, not them. I say it as someone who have been like that and have changed it through therapy.
I was a nervous mess and always thought that people around me don't like me, pressure me and in general was very nervous around any interaction where I and other person have some disagreement. But turns out, firmly saying "No" is quite OK, people react well to it, respect you for it and nobody is going to fire you for giving good estimates.
This is gold. Besides, if someone doesn't react well to your honest estimates, this is probably a red flag and you're better off letting them find someone else. After all, this is supposed to be in everyone's best interests.
It actually is a problem with them because when someone asks "Does it really take that long?", they're implying that it shouldn't. Of course, in reality, they have no idea either and if they're asking you how long it should take, they should accept your answer, not try to weasel out less time.
But are they ever going to admit that they have no idea and accept your estimate on face without playing BS mind games? Probably not. But for some reason that doesn't stop them from trying.
I've never seen someone try and do this with a home contractor.
It can also be that they flubbed at communicating and meant to ask, “what are the things that make it take that long?” Underlying that question would be, “are we on the same page about the scope of this?”, “Are you making business assumptions that I can actually provide updated information about?”, and “Are you lacking resources/clarity that I can get delivered to you?”. Thats a charitable interpretation but it’s occasionally the correct one.
It might also be that they’re surprised at the complexity and want to know more. For this reason, the important part of an estimate isn’t the number. Its the task-breakdown and the identification of risk.
Explicit is better than implicit. Asking is better than guessing. If you think that someone is implying something, ask them.
In most cases, people who ask "Does it really take that long?" are giving you their first emotional reaction and haven't actually thought it through deep enough to decide to manipulate you. They're upset and they want to make sure that they understood you right - which is a thing people often do when they get bad news, and I cannot blame them: if you get upset about something, at least make sure you really should be upset.
So, you should react either in a spock-like "stupid rational", giving no thought to what they might be implying whatsoever with a simple "yes", or, even better, in an emotionally-aware rational way: acknowledge their emotions and offer options. There are always options and trade-offs, they just need to be clearly communicated. Not "OK, I'll do it in three days instead of five", but "I'll can do it in three days with prototype quality so we can test it out, but it will be buggy as hell, not fit for production, and you should document it". I've never had a manager who had a problem with that, and I despite how popular horror stories are, I really doubt that there's a lot of them out there.
Your manager cares about the product, he's emotionally invested in it and wants everything to go as fast as possible - it's not a bad thing! It's only bad if you view his disappointment as your fault - but it's not. It's just how things are. As a developer, you're often a messenger bringing bad news. There's no way around it, and you have to learn to deliver bad news for people. And separate these bad news from you and your professional self-confidence.
Your manager cares about the product, he's emotionally invested in it and wants everything to go as fast as possible - it's not a bad thing!
If he was emotionally invested in the product, wanting everything to go as fast as possible is not necessarily a good thing. I can make things go as fast as possible, but then the questions become no unit tests? no integration tests? spaghetti code?
And this isn't a response from a few go around either. This is often the first thing that comes out of their mouth, right at the beginning of the project.
What happens is as follows:
1) Your manager has software development background and uses it as a superiority contest to show how much better they are than you. His lack of awareness regarding variable skill level or lack of knowledge of an existing system seems to factor little into the question. So one could argue the manager could be more empathetic in this case.
2) Your manager knows nothing about software development and really has no business asking such questions in a condescending manner. So they either pretend to know or they promised someone higher up the chain even though they had no idea, but were scarred to say so, thus the cycle has completed.
3) Your manager has a calm, rational head and accepts your estimate, your explanations, and has a good understanding of the product to negotiate with you in terms of requirements without getting emotional. The manager trusts understands that your estimate is, more than likely, not going to be accurate but accepts the changing winds and doesn't hold you to the fire.
I totally agree its better to be explicit. Which is why it matters to avoid overly emotional reactions and accept what people tell you so hand waving can be avoided.
But this is more of a prevailing problem in software requirements in general. It's rare that developers get projects with clear ideas on what needs to be done because of the ignorance or unknowns in questions that need to be asked in order to deliver a product. Often the developer has to decide, and when they do and its wrong, then they have to fix whatever they decided because no one explained, or bothered to explain, how something was supposed to work in the first place.
> I've never seen someone try and do this with a home contractor.
I'm surprised that you haven't. I've certainly seen the same kind of thing happen, though admittedly it is usually based on cost rather than time. Ultimately, though, the discussion was the client disbelieving a high effort estimate.
Questions have built in assumptions and implications. Most questions don't have enough to be of concern, but some questions definitely imply things beyond the question itself. When asked 'Does it all take that long?', many hear an implied 'Make it not take so long.' command. It is not implied all the time, but it is implied often enough. This does not mean someone has low self-confidence or needs therapy.
If you hear a 'Make it not take so long.' command, thats not an unreasonable interpretation. However, it is also one that should be questioned because it clashes with how software projects work.
If you go from "my boss gave me a command" to "I must execute the command and not question it" without exercising independent judgement, then I would assert that you do have low self-confidence. Whether someone needs therapy is a deeper question that we should be hesitant to answer for another person. But therapy was tremendously helpful for me in realising how my personal and professional life were held back by my reluctance to trust my own judgement or endure the discomfort that can come with honest, respectful, collaborative confrontation.
I should make explicit: Part of trusting ones own judgement is also being willing to say, “I don’t have enough knowledge/experience to make a trustworthy judgement here.”
Can you explain which part, in your opinion, is facepalm worthy?
In your example conversation, the developer starts off by demonstrating that they are not good at thinking through how long things take them, by giving an impossibly optimistic estimate. Even the PM, whose job is not to figure out how to build stuff or how long building stuff takes, knows the estimate is wrong, and pushes back on it, pointing out dependencies the Dev appears to not have considered.
The dev comes back with another estimate, also provided on the spot, without actually thinking through all the tasks, problems, and potential ways to streamline / accelerate. The PM, having just had the dev's lack of thorough thinking with regards to complexity estimation amply demonstrated, asks the Dev to think more carefully, a third time.
Are we facepalming at the dev who can't be bothered to actually consider what it will take to get the thing done, or the PM who for some reason accepts the 3rd estimate, even though the dev appears to just be randomly generating numbers? Both?
It's this part where the estimate becomes a deadline:
> PM: "That is perfect! The customer wanted to put in production on Monday, so we can now confirm. Thanks!"
> pointing out dependencies the Dev appears to not have considered.
I'm not sure if they're dependencies or other priorities in this case. I've had managers before ask for an estimate and and I've given them one (2 days), then they've assumed that I'm dropping everything I'm in the middle of and told clients it would be done in 2 days.
The problem with estimates is that the people asking for them really want promises.
1. The PM just wanted it done by Friday, because the customer wanted it by Monday. This was clear at the end of the conversation, not at the beginning. So it just could have been a "Can you get this done by Friday? Because the customer wants to use it on Monday"
2. The developer clearly had no backbone, and was just throwing numbers to see what the project manager would like. 1 day? Too short! 5 days? Too long! 3? Perfect! Estimates might be a guessing game, but not a guessing game on how long the PM thinks it should take.
So yeah, facepalm to both of them, and the whole situation. But I must say that they were definitely not incompetent people. The PM was one of the best I worked with, and the dev was a smart passionate coder. It was just that situation that was a bit absurd.
Proper way on how to do this:
PM: Can you give me a deadline (not estimate!) on when you can finish X? The customer needs this ASAP, so it has priority over your other tasks.
DEV: Let me look into it and I'll let you know in half an hour. (Works out estimate, possibly asking input, adds a buffer because deadline != estimate, looks into his own schedule of meetings etc).
I'd replace "they are hired for their knowledge" with "professionals are hired for their judgement". Judgement usually depends upon knowledge, but its the exercise of judgement that is more key to emphasise.
On the other hand, you can only argue about the estimate if there is even a channel to argue about it. I have seen several cases of (1) people being told about a deadline through various proxies, so there was no point in time where they could even question the imposed deadline, (2) people not being told about the deadline at all until it was missed, so they could not argue against it, (3) people not being told about the task that has a deadline until after that deadline was missed.
All involved engineers actually had the backbone to say it was impossible, but no chance to say it. Obviously, projects did not go well at that place.
Another nice trick if you're under time pressure is to arrange the project so that you can deliver it in stages that partially meet the requirements, or even deliver something non-functional that people can play with while they wait.
This can keep the sales team rolling or customers happy for a while, buying you some time to write the program properly.
As a bonus, you might get earlier feedback if you're going in the wrong direction, in which case less time wasted.
(Obviously this approach isn't always possible. Many problems won't decompose nicely.)
this is where it becomes crucial to know why you're building the thing you're building. That way, you can often structure the project in phases according to your engineering judgement of the way those phases are built and the business value they enable.
Then you can cut scope by postponing (or not doing) some of the phases. Ideally.
> But when you advocate clearly for your needs as a professional, people are generally reasonable.
This has not been my experience. People want ‘estimates’ at all costs, tell you to not worry about any accuracy, and then a week later tell your manager you committed to x date.
As long as software is a cost center to your company, it will be treated this way. Trying getting a job at a company where software is a core concern. Ideally with a CEO that is not from a marketing/business background.
> In the real world, picking up a new language takes a few weeks of effort and after 6 to 12 months nobody will ever notice you haven’t been doing that one for your entire career.
Spoken as someone who has never taken the time to fully master a programming language and, from the sound of it, has never worked with someone who has either. The difference between someone who has spent 6-12 months with a language compared someone who has spent 6-12 years is night and day. From the general tone of the article, he obviously focuses more on the business value than on the technical side and that is a pretty good approach for making money. But I'll take Peter Norvig's advice (http://norvig.com/21-days.html) over this guys when it comes to mastering a language.
To be fair most of the content of those articles is pretty decent though and it is just a pet peeve of mine when people claim that mastery of your medium doesn't matter.
One thing I've learned working in both kinds of companies is that having software as the core concern of the business isn't automatically a shield from bad project management or bad engineering management.
'Not considering software a cost center' is a necessary but insufficient condition for healthy estimation. A software company often appreciates you more, but is still susceptible to the other myriads of management pitfalls.
I have watched a manager with a master's degree in computer science pull this BS. At an enterprise technology company. He would go as far as repeating a question over and over until he got an answer he liked.
The senior members of his team had a culture of eighty hour work weeks. Draw what conclusions you may.
That doesn’t really matter. In these cases you often have information akin to ‘we want a webshop’, so you estimate something more or less reasonable, then a few weeks later you find out they actually meant ‘we want a duplicate of Amazon’.
The biggest problem is that engineers tend to think in man-hours and managers think in calendar time. If I say a project will take 2 days, it'll probably take 2 days, but it might span 4 calendar days because other things are going on.
The biggest thing that has helped me here is to keep a running list of my tasks, an informal Gantt chart, and show it to the manager whenever they ask me about an estimate. This helps me communicate in a way they understand.
There's a lot of bad to be said about how companies use story points and velocity and all the rest, but I do greatly appreciate the way they delink "amount of work" from "delivery date". It's a nice built-in way of expressing time requirements separately from schedule, which is a lot smoother than constantly reminding people that the difference exists.
I find you have to do this, but it can damage professional relationships.
Recently I've flat out said no, as gently as I could, to doing some work - porting a software stack to run on Windows.
A customer is asking for an estimate to do it, because a project manager who wouldn't be developing or supporting (or even using) the system likes Windows. The sales manager isn't thrilled with me over it.
So he's asked the team as a whole to estimate it because I wouldn't. The rest of the guys are frontend devs. I don't have a clue, they're in a worse position but for some reason they're all up for putting a number to it so I look like a bit of a !
I'm a contractor, so I can and would just turn down the work, but it's difficult politically.
Ok cool but if they have to deliver it, they’ll look like fools in the long run when they can’t, and in the end they’ll see you’re right? So it’s a win for your reputation in the long term I think?
Nobody will revisit the GP's reputation, and after the novice devs fail to do the work people will just scatter into different projects. They may or may not take a reputation hit, while the PM that insists so much on doing the not viable work will take none of it.
My guess is no one will look back and say "Huh, he was right".
The only thing that probably is going to happen is "Huh, why isn't that delivered?" and then they will look everywhere but on the fact that it was, at least partially, just a stupid idea in the first place.
Of course I don't know the exact situation so it's more of a thought on what happens in general in the industry.
At the risk of a totally uncalled-for derail into politics - this is a hilariously apt description of Brexit. It makes me want to reanalyze the entire situation from a project-management perspective. It suggests a possibly illuminating metaphor: party leaders = management + sales, the electorate = the customer, and representatives = developers and engineers.
An alternative to flat out saying no is to point out how expensive and risky what you've been asked would be, in terms of development time, maintenance, and other metrics that the business cares about. Even very rough estimates with a list of risks will usually do.
If somebody else comes out with a much lower estimate, ask them what they think about risk A, B, C... if you've got a point, this will usually work.
That's a bingo! It's never No! It's "Sure, this is what it entails and this is how much it costs". Especially if you're a contractor. If their willing to pay the price, why not take the money and do the work?
I have literally no idea how long it would take though, I couldn't in good faith put a number to it.
I also don't want the work - Windows isn't an environment I have experience in because it's an environment I don't want experience in. They wouldn't allow a rate change to do the work.
If you do this, identify beforehand what you think is non-negotiable. Because it won't be a technical discussion, it will be a negotiation and you're liable to be committed at the end of it.
Of course, once you're holding certain things as non-negotiable... you're saying no.
The thing to understand about politics is you can do everything right and still lose.
Here, the damage was already done by their unrealistic expectations. You were either going to accept the job, fail and make them angry, or reject it and make them angry.
Sometimes your least worst option is to cut your losses.
I agree with you but like others have said, they want a date they can sell to and in the end it will be your fault.
It isn't just software that goes double, look at most custom building projects or government contracts. Most everything takes twice as long and costs twice as much.
I read something once that has stuck with me. "We can never give more than a vague guess because we are literally building something that doesn't exist and has never been built before."
* I do find it annoying that the sales teams generally drive this. I wonder how they would feel about a response of "We'll get you an estimate of when it will be available as soon as you get us an estimate of when this feature will generate the $1M of revenue to cover the costs of building it. Also, please give us a list of customers that are likely to buy it and your expected contract date so we can track it along with development."
> It isn't just software that goes double, look at most custom building projects or government contracts. Most everything takes twice as long and costs twice as much.
There's definitely a correspondence here, but it's a bit indirect.
Government and even private contracts are often knowingly lowballed to secure the deal. Rules like "must choose lowest bid" reward entering an unrealistic bid and then inflating costs after work is going. Or, if overruns come with sufficient penalties, doing work at a loss and then charging a fortune to support a fragile and proprietary result.
In the metaphor to non-contracted software, developers generally don't correspond to the contractors but to the government. Sometimes blown estimates are simply errors, but other times somebody wanted to secure customers, so they pitched an unrealistic scope and arranged to make the inaccuracy land on the programmers. Where the government ends up with late delivery or runaway costs, programmers end up working crunch time or sacrificing quality. And where the government ends up locked into paying for support, programmers end up locked into maintaining rushed, tech-debt-riddled work.
(As for pushing sales for revenue and contract estimates, do it! Some of the most productive teams I've been part of have been ones where sales applied this pressure, but also responded well to the reverse version and kept everyone informed.)
I like to mess with people. "It's 80% done so there's only 80% left to go". "Now' it's 90% done so there's only 90% left to go" etc. Of course my current project is rather higher stakes than usual, so I'm being a little bit more sober than I normally would be. However I am emphasising the need for communication with other stakeholders and identifying and trying to rectify where they are bottlenecks pdq.
I highly recommend the book "Software Estimation: Demystifying the Black Art". I work for an outsourcing company and part of my job is to come up with estimates and it helped me deal with clients and managers.
The book contains a quiz that we used as part of a training exercise with management and the results were hilarious. Here is an online copy: https://scrumandkanban.co.uk/how-accurate-are-your-estimates... please don't look at the answers, I guarantee you will have fun completing it.
For clients, my preferred approach is to show "The Cone of Uncertainty" and ask where they think they are. Since most people have no idea what they want to built I ask if its ok if my estimates are 4x off. That usually gets me a few weeks of peace while a team comes up with a product definition and we start all over again :)
I started reading that book in 2014, but put it down because it really was about project-level estimation on the order of a few weeks to a few months. At the time, I was really looking for guidance on how to estimate tasks—things on the order of a few hours to a couple days.
I’ve gained some experience and confidence at this in the years since, but at the expense of some psychologically painful experiences and a firing. Our industry really calls out for a more basic book aimed at University CS grads (or bootcamp grads or MOOC)-grads on how to estimate tasks.
> Total length of the coastline of the Pacific Ocean
Coastlines are fractal. If your measuring stick is 1 km long, you'll get one number. If it's 1 m long, you'll get a much larger result. And if it's 1 cm long, you'll get an even larger result.
>The actual length of a coastline is infinite when measured with an infinitesimally small measuring stick.
>In actuality, the concept of an infinite fractal is not applicable to a coastline; as progressively more accurate measurement devices are used
The notion of the length of the coastline, and why I think the question is apt as well, is that regardless of the fractal paradox, the practical answer is that there is an answer that is accurate enough to satisfy the business's or the customer's needs.
The analog for software projects, from my point of view, is scope, and it's generally one of the biggest problems with projects that take too long. Product and various stakeholders take outline broad, high-level scope requirements that may or may not actually be informed by the realities of building or delivering that piece of software. They generally kick the can down the road and take a "I'll know it when I see it" type approach that winds up drastically changing things at some arbitrary point later on on the project.
A rough measuring stick for scope always means that the answer will be longer, however, small measuring sticks for scope are absolutely impractical. What's needed is the understanding from the business that refining scope increases development time, and the high-level broad scope that is sufficient at the beginning of the project is not sufficient at the end. Many businesses want to run with the assumption that their broad scope and the estimates that go with them are accurate enough to bring a product to market.
The practical answer is that the question is ill posed, and you need to clarify what is actually being asked. A reasonable way to answer the question as asked would be to model the coastline as a fractal, which would give you an estimated length of infinity. What is needed is not a better way of estimating the answer to the question as asked, because the answer to that question is not useful. Instead, what is needed is a better question (which would require knowledge of why the question is being asked).
Thank you for posting this - I took this quiz a few years ago in a team exercise and all of my answers were off. Since then I wanted to find a copy of this test, but didn't know where to look.
I think out of 15 people nobody got more than 1 or 2 right, except for my manager, who somehow got about 8 right. I'm curious how the HN crowd does on this - care to post your scores?
I got 6 right. For another 2 incorrect estimates an edge of my estimated range was very close to the correct answer. I think the key is not to try giving too narrow estimates and think in logarithmic scale. If it was possible I tried to decompose a problem to smaller pieces and calculate my estimate from information known to me (i.e. radius/circumference of the Earth for geographic questions) and make a range out of it by adding/taking couple of zeroes from my original estimate.
A big +1 for this book. I read it with the team of developers I managed (at a consulting company, where estimates directly made the difference between profit and loss on projects) and we found it very useful. It got us to quit using "gut" estimates quite so much and to always keep an eye out for actual data or at least things to count.
> “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.”
This is my default. Even when I am fairly confident I know what the code does, I want to double check. And it isn't just so I don't short change myself on time. If I tell them X days and deliver on 0.X days several times, they'll start thinking I overestimate everything and cut me off at the knees. It pays to get as accurate as possible.
The lack of a good “intro to task and small project management” MOOC or book is a serious problem for our industry’s ability to onboard junior engineers. I wish I knew how to write one, but TBH I don’t know any principles myself except one.
Lots of people say “just do it and multiply by 3” without appreciating that The important part of estimating tasks is coming up with the task breakdown and identifying the risks.
The best you can do in most scenarios is to research comparable projects and calibrate against them. "They shipped in six months, therefore we can ship in six months."
Have you ever done this and had it turn out successfully? Because my intuition is screaming at me that this would be disastrous.
For one, it skips the step where you break down the tasks. For another, it seems likely to ignore a bunch of variables in the search for something similar enough.
Both the mentionned idea are natural, but it's ultra sad that we can only live by experiment like craftsmen for previous eras. Not that I mind craftmanship but at least school should write it in bold letters so you're not seeking a mathematical model.
once my CEO asked: "can we add more developers and finish the project faster".
I used this famous quote
"9 mothers cannot deliver a baby in one month" and we had a big laugh
My somewhat more nunanced retort to such kind is that it depends how much of this work is linear and how much non-linear and how are they naturally occurring. non-linear work may mean things like understanding a complex codebase or developing an algorithm. these dont speedup by adding more people ( what I dont say is these may speedup by adding more quality/unfragmented hours to the individuals doing it). However stuff like refactoring codebase or minor bugfixes may benefit by adding more engineers.
Unfortunately, the problem then becomes defending whatever timeline you have in mind for non-linear work. also doing too much of this can give bully managers an opportunity to nitpick and micromanage the whole thing.
> You need to be willing to endure the discomfort of looking someone in the face, saying “I don’t know”, and then standing your ground when they pruessure you to lie to them.
This. I have PMs who will ask me over and over until I give a number and I've learned to stand my ground. Because if I don't, I end up being responsible for the estimate I've given (as I should).
Now I make it clear that I will not give a number until I know more. Just a few weeks ago I was asked to give an estimate of how long it would take to do an integration with something I've never used before. I said, "I need 1-3 days to learn the product enough to give an estimate". What I get back is "can't you just give WAG". But this time around I said "You have two choices, 1. you give me time to learn what I need and then I'll give an estimate or, 2. Find someone who already knows the product to give you an estimate"
"SWAG" sets the expectations much better than "educated guess"- you're saying "I got very little idea, but you really want a number, so here's a number". "Educated guess" sounds much more like you've got a good idea, which using the term "SWAG" is trying to avoid.
If your guess is a mathematical formula with assumptions for the inputs, those assumptions can be tested and the formula refined ... It's a SWAG.
You use things like Average/95/99, t shirt sizing, how much work can be done in parallel, 3rd party/client dependencies, ramp up time for a new library ... it's still a wild assed guess but at least there's some basis in reality and you can defend your estimate - show me what I'm wrong about or missing and I'll revise my formula.
Negotiating over the assumptions and inputs is 90% of the battle anyway. Without a model you're standing on sand.
>> I end up being responsible for the estimate I've given (as I should)
You're not responsible for an estimate you give, the person who's asking for it is.
This is the key difference between an estimate and a commitment: If you give someone an estimate, they are responsible for steering the project using that information. If you give someone a commitment, you are responsible for meeting that commitment.
This is why very few people actually want estimates. They want commitments, but they will ask for estimates to get an answer and then interpret them as commitments. Most people asked for estimates know this intuitively, thus "doubling and adding 10" and other communication strategies to turn estimates into safe commitments before they are communicated.
One big insight for me, which you obviously put to use in your story, was learning to give a different number.
"I don't know" is understandably frustrating to people who are trying to plan work and answer requests. They often respond to that by pushing for a delivery date even when it won't be accurate, but a lot of the time what they fundamentally want is a numerical answer. "I'll have it scoped out by Tuesday", "I need at most 3 days to learn enough to estimate", and so on all aid that planning without committing to a project timeline. That rarely ends the conversation, just as it didn't in your example, but it does a lot to show good faith and justify not answering the initial question.
Also clear communication (although you are pretty much describing that too).
I had a friend who had an unusually high frequency of conflicts with various management, project managers (I expect there are always some, but this guy had a lot of bad experiences).
He showed me an instant message conversation:
PM: "Is X done?"
Dude: "Yes, but a, b, c is not done"
A, b, and c were parts of X's requirements.
Now the project manager or such should have been able to read between the lines, but man don't lead with "Yes" and actually describe a situation that is "No".
It's important to realize that the workplace often involves being subjected to what is really a kind of psychological abuse. It is not, I think, a coincidence that we are learning that many "stellar managers" turn out to be abusers, e.g. guilty of sexual harassment. What you're describing is in psychological terms thought of as "maintaining boundaries" against a person who is intent on transgressing those boundaries for their own gain or amusement.
If you validate the invalid assertions of these abusers (e.g. that some unclear task can be estimated with certainty) then you're enabling the abuse.
I think “abuse” is an ill-defined enough term that I’m not sure it adds clarity to this discussion without setting out a definition. Its an important subject and might be a really important lens through which to view this, but as someone who has been searching for a really clear-cut definition of “emotional abuse” since my childhood, and who has seen a lot of acrimonious conversations about it since then...I worry that the framing adds more smoke than light to the discussion. :(
Do you have a good definition? One whereby a scrupulous person can confidently judge “am I acting abusively?” correctly?
I always consider four factors in estimating software cost
- the scope of the task
- the resource/people you have
- the confidence in the resource/people
- and the dependency on both the resource and the task
Estimation goes wrong in many ways. In my experience, I categorize as follow:
- estimated by non-technical PM, unable to gauge technical/business complexity
- estimated too early in development life cycle. Often time someone come up with idea X then ask right away "shouldn't be difficult, how long will it take"
- no view on team member's productivity due to lack of measurement or measuring the wrong thing
- not considering dependency in the development cycle. e.g. The question "whether our only backend engineering available for the task" is often omitted
- not considering testing, documentation, 3rd party integration/procurement, maintenance and deployment cost
The common practise of coming up a number, then double, or maybe triple it up to make the estimation does not address why estimation is off. Mis-estimation is a symptom. The cause and the cure lies in the people
Thesis: Developers are good at estimating median time to finish tasks. But the tasks that take longer, in fact take much, much longer than estimated.
E.g. Dev estimates that time to complete each of A, B, C tasks will be 2 days. In reality, A will take 1 day, B will take 2 days, but C will take 8 days.
Dev was right about the median time to complete each task (2 days) but average was much higher. Article goes into how to statistically model the distribution of actual time to complete tasks.
In the UK, bookmakers offer 'accumulator' bets, where a punter can select many outcomes, getting a big prize if 100% correct.
This takes advantage punters' failure to accurately multiply probability - 10 events with 80% probability have joint probability <11%.
Something similar happens with planning, where people fail to compound many possible delay sources accurately.
Dani Khaneman covered this in Thinking Fast and Slow, also showing that people overestimate their ability, thinking they will outperform a reference class of similar projects because they are more competent.
It doesn't take advantage in the way you say because you get paid along the same lines. If you bet an accumulator with 2 selections at evens, you get paid at 3/1. (4.0 decimal odds) so that is "fair".
It's profitable for bookies because they have a house edge, and that edge is increased the more subsequent bets you make. The house has more edge with an accumulator than a single bet.
People like to do accumulators because it's more meaningful to win large amounts occassionally than more regularly win less meaningful amounts.
So it's a "trick" to simply increase gambling.
If you had to pick out 8 evens shots in sequence and intended to roll over your bet each time it would have the same effect / outcome, but starting with a pound by the last bet you're effectively placing a 128 quid bet on an evens money.
It's not that the player thinks that they have a better chance of winning than 1/256 it's that it effectively forces them to gamble a lot larger amount in the situations where only 7 out of 8 of their picks come in.
And that's before considering the edge. If we consider that these are probably events that happen more like only 45% of the time (at best) then instead of a 255/1 shot we're looking at 600/1 shot.
The interesting thing is that by the central limit theorem, the mean of a mean is normally distributed. This is extremely helpful. Here's what I suggest you do:
Same size your stories to small values. Do 30 stories in a sprint and take the mean. Do 30 sprints and take the mean of the sprint. What you get is the mean amount of time to do a sprint of 30 stories. What's amazing is that this estimate will be normally distributed. You can measure the variance to get error bars.
Of course 900 stories to get good estimates ;-) However, imagine that your stories averaged 2 days each. Imagine as well that you have a team of 10 people. That means that you will finish a "sprint" of 30 stories in 6 days (on average). 30 sprints is 180 days -- the better part of a year, but you probably don't need a 95% confidence interval.
You will find that after a few sprints, you'll be able to predict the sprint length pretty well (or if you set your sprints to be a certain size, then you will predict the number of stories that will fit in it, with error bars).
The other cool thing is that by doing this, you will be able to see when stories are outliers. This is a highly undervalued ability IMHO. Once a story passes the mean plus the variance, you know you've got a problem. Probably time to replan. If you have a group of stories that are exceeding that time, then you may have a systemic estimation problem (often occurs when personnel change or some kind of pressure is being applied to the team). This kind of early warning system allows you to start trying to find potential problems.
This is really the secret behind "velocity" or "load factor" in XP. Now, does it work on a normal team? In my experience, it doesn't because groups of people are crap at calmly using statistics to help them. I've had teams where they were awesome at doing it, but that was the minority, unfortunately.
The central limit theorem is in the limit for the number of variables in the sum approaching infinity. In the finite world, the article explains how it's done. The article is saying, the sum of lognormals is not normal. You are saying: take enough of them and it is normal. The article is still more accurate than your reasoning for 30 stories. From the wikipedia entry for Central limit theorem " As an approximation for a finite number of observations, it provides a reasonable approximation only when close to the peak of the normal distribution; it requires a very large number of observations to stretch into the tails". To prduce a 95% confidence intervals, you have to upper-bound the tails. All methodologies that are based on sum of subtasks estimates are not evidence based. But we knew already sw methodologies are not evidence-based, did we?
You are saying: take enough of them and it is normal.
This doesn't completely undermine your point, but that isn't what they are saying, I think. I read it as saying by CLT that the estimates of the mean of those distributions is normal and centered on [the mean you are actually interested in]. Tails are perhaps somewhat a red herring here, because you don't really care about them unless you are specifically trying to evaluate worst-case-but-really-unlikely.
Yes, that is correct. It's been a very long time since I studied statistics, so I'm not sure if the variance of a mean has the same confidence interval as the mean. I suspect not. So you would indeed need to have a very large number of samples to get good error bars. It's a good point which I hadn't really considered. However it will never really get that far anyway because hopefully you'll intervene before the long tail hits you.
I think those really long tails are more of a problem when you are working with "features" that are much longer. If you have 1 day stories and you've been working on the story for a whole week, you know you have a massive problem. It's time to back up and see if there is a way to break it up, or to do it differently.
If you have a feature that is a month, by the time you get to 5 months, you have so much capital invested in the original plan that it's very hard (politically) to say, "Nope... this isn't working out. Let's try something else". Of course, it is very hard to get your organisation to plan to a 1 day level of granularity.
In the world of small medium projects often the major issue is that software engineers give estimates for writing the software but customers take that to mean time to actual delivery in production and a lot of the time have no idea how big a task deployment and integration are...or don't even have a plan for that.
According to Joel Spolsky¹, programmers are generally bad at estimating, but they are consistently bad, with the exact factor depending on the individual. So by measuring each person’s estimate and comparing it to the actual time takes after the fact, you can determine each person’s estimation factor, and then when they estimate again, you can get a pretty reliable figure.
True, but you should always put up estimated effort and real spent time in the task, whichever bug tracker, or issue tracker you're using. I don't know of any other way to improve the estimation "skills".
I tracked this myself for a few months and found that I was fairly consistently estimating around twice the actual required time. That was for fully scoped out and planned work, mind you (and that I was the one who scoped and planned).
"As estimators gain more experience, their estimating skills improve. So throw away any velocities older than, say, six months."
I don't know how Spolsky is (was) as a boss, but it's is way easier to just adjust your time reports according to the estimates and make your boss happy than to actually try for no benefits to either you or the project. As long as the estimation errors are white noise-ish and in linear relation to actual task length, which is the assumption for his model to work, manipulated time reporting and not manipulated make no difference.
I've noticed that when programmers are introduced to Agile they initially are bad at time estimates until they learn that you actually can report time according to the estimates and no-one will ever notice. Extra points for looking on the burndown chart before reporting time to make it smooth.
"Some developers (like Milton in this picture) may be causing problems because their ship dates are so uncertain: they need to work on learning to estimate better. Other developers (like Jane) have very precise ship dates that are just too late: they need to have some of their work taken off their plate."
Milton is probably doing fair estimates. When planning four months into the future you probably can say where you are within 2 months with 50% probability (or what ever boxplot he uses) of where you think you will be at that time. Janes reporting is obviously manipulated to match the estimates.
I’ve been coding professionally for 20 years and I’m largely no better at estimating completion than day 1, maybe worse since I’m more likely to feign confidence in a figure I’ve essentially pulled out of my ass.
At some point, I came to the realization that my fundamentally poor concept of time was always going to be an insurmountable obstacle to my career advancement.
Well, I think this is a complex topic. Not because of the math, but because the key to an accurate estimate is to understand who has done the estimate and on what basis.
As stated in the summary, the core driver for inaccurate estimates is uncertainty:
> Tasks with the most uncertainty (rather the biggest size) can often dominate the mean time it takes to complete all tasks.
There are different sources of certainty:
- Experience: If someone has done a task 20 times he probably knows how much time he will require. Someone who hasn't done the task yet, probably underestimate the time he requires (e.g. because of the median vs. mean conflict). But don't be fooled just because you have 20 years of work experience in the field but never actually done a specific task doesn't mean you can estimate it better than someone who just started the job but had done that specific task 10 times. However, most of the time, projects are doing something new. So you have to find out which tasks have been done before by a project member and which are completely new ground. If something is completely new, remember to plan time for getting familiar with a problem space plus a handful of complications (together this will be more than the actual task would take someone who is trained for that specific task).
- Detail: The smaller the tasks the larger the overall estimate... or so. Planning on a top-level is rarely going to be accurate. We do it a lot because it doesn't take much time. But if you want an accurate estimate, you have to plan on small, specific tasks.
- Risk management: Every project has risks. Some don't really have an impact and others blow up the whole project. Know your risks and what you are going to do if something should go in the wrong direction. It is not like you wouldn't have time to figure out what to do when the problem occurs but to understand how it would impact your timing and to take preventive actions (e.g. include stakeholders).
If you have people who have done the exact same task a few times, made a detailed plan of every step and know how to handle the most likely or impactful risks you are in a good position to deliver on time. Most of the time you won't have that luxury and have to compensate the resulting uncertainty with a prolonged time to project completion, but that should be just fine when being communicated in the beginning.
With all that said, remember, that some projects don't require an accurate estimate. Sometimes it is enough to deliver just as soon as possible.
On the experience part, this happened to me few times, when the project manager asked me for an estimation, i give it from my perspective and experience, and then he gives the work to someone else with no experience, and obviously takes more than estimated.
Now, i always give two estimates:
An estimation if the work is done by me, and an estimation if the work is done by someone else.
I always think of project tasks as flow charts, where every item either takes 1 day, or 1 week. There's no way of really knowing in advance. Complications happen.
It makes it really hard to calculate the "expected value" of 5-10 tasks.
The more tasks you have to do, the more certain about duration you should be - some of the uncertainty will cancel out and you will get a gaussian distribution. For your example, I expect 10 tasks of between 1 day and 1 week each (with flat probability) to take about 6 weeks in total, with a 95% chance of completion within 7 weeks.
I think the most important issue why projects take longer is not because the time it takes to complete a task is uncertain, but because at the start it will always be unknown which tasks will prove critical.
The further you get, the more tasks will reveal itself that were not part of the original scope, but critical nonetheless.
In my opinion, the reason that software projects take longer than we think and estimate is due to the fact that companies don't know how to handle Underemployment properly...
1) The person(s) doing the estimate aren't qualified to do so.
2) There is a disconnect between wishful thinking and reality.
3) There is some arbitrary future milestone (i.e., "We need to ship by ____ because ____ is happening the following week.") that is independent of software development.
4) Most importantly, when the deadline is missed the estimate / estimators are not questioned, the software team is.
I've been at this a long time - too long? - and the narrative that IT is __always__ at fault is a myth that needs to be buried.
To me, software projects take longer than i think because customers don't know what they actually need until there's a runnable version of what they want.
324 comments
[ 5.9 ms ] story [ 328 ms ] threadIf you want to make the uncertainty clear it's better to allow ranges, like 2h-5d instead of trying to play stupid mind games.
Story points are like setting your watch 5 mins early so you are on time to things. It doesn't work.
why fibonacci? I think it's reasonable to say time includes an exponential error. an estimated 1 hour task is very different than an estimated 1 week task. i see that 1 hour, 1 day and 1 week estimates are progressively, and likely exponentially, worse.
Is it just the ease of doing the math? (totally reasonable answer, in my humble opinion). or is there something specific about fibonacci that's actually relevant? I think it's the former not the latter. but if you have any evidence to the contrary, i'd love to hear it.
Cumulative error is _hard_. see weather prediction.
But no, here we are doing science, that's why we use Fibonacci, hey even plants and rabbits are using it.
Unfortunately, the phrasing would be more like, It doesn't matter what's true, it matters what i can sell to management.
In the absence of evidence to the contrary, i think i'm ready to take a side close to yours. It's useful to expand error bars exponentially as you move away from the time of the estimate, but there's nothing special about fibonacci.
Yes, that's my main beef with it. SCRUM could be based on science, but sadly, it isn't. Here's an interesting book: https://www.amazon.com/Making-Software-Really-Works-Believe/...
You should think about it (even though there are no actual physical units you can use to make this work):
Time to deliver a task = complexity / developer efficiency
This is why it is hard to develop a method to estimate time for development tasks.
When estimating I tend to use a coarse scale of hours, days, weeks, and (in extreme cases) months. All estimates then get turned into a range between the minimum and maximum the estimate could mean - for example hours becomes 1 hour to 8 hours, days becomes 1 day to 5 days, and so on.
Anything estimated as weeks or months probably needs scoping until it can be broken down into tasks that will take between an hour and a week.
I find this method works well because it’s intuitive, comes out with a suitable amount of uncertainty relative to the size of the task, and finally results in estimates with units that are useful to project managers and the wider business in scheduling other work needed to get a feature into production.
e is also just an arbitrary number, but it looks like someone calculated it using sophisticated math.
So if you estimate to Level 1, and then break down to Level 2 sub-tasks, then Level 3, your estimate will be out by the same amount each time. and your total error will be a multiplicative series.
If you have an infinite number of levels - which is reasonable for most real-world projects, although this may not be obvious in the planning stages - e will get you nicely in the ballpark.
Problem is the fact that you can only justify with a straight-face 45% (100%/220%) of the estimate you give to the client... Then you need to get clever at "bluffing" your way at "inventing" another extra +35% of work that you can't imagine/explain, but you know for sure that will come out of the blue - you know there's this extra 35% work needed for sure, you just have no idea what this work will be. Then finally you can say with a straight face that 20% of estimated time is for unforseen circumstances. There's two ways to see the same math:
The world would be a much better place if clients / managers / project owners etc. would just accept that 55% of time & cost for software dev is unknown-unknowns type of unexplainable/unimaginable-at-project-start-stage-work. That's just how it is! If the problem was really clearly defined and understood, there would likely already be an off-the-shelf software solution for it that could be used instead of paying developers to build custom software!If we could just be honest with people and show them formula (2) instead of a make-up-ladden version of (1). But people don't, so we get to play the damn elegant-bullshitting game over and over again...
This is how I always have done it across multiple jobs and I always called that extra contingency. Depending on the project I sometimes had to ask for over 50% of the total as contingency. If anyone queried it I would explain that it is very likely we'll need it based on my experience of many other projects.
Then the client would either accept it as is, or they would want to split the proposal into "what we will definitely pay for" and "what we will pay for only when it is actually needed" which is also fine for both sides if proper progress reporting has been agreed in advance.
Someone could think that no client would agree to such a contract as it would open them to be taken advantage of by an unscrupulous supplier, but at the end of the day what matters in business is trust. If a client doesn't trust the supplier they shouldn't hire them. And in all my previous projects I never met an engineer or developer that would deliberately use up the contingency time when it is not needed as everyone knows the availability of this extra time might come very important towards the end of the project.
???
The doubling-it is for when you build the whole component on top of a library, then discover that the library has a fatal bug you can't work around, and you have to rebuild the whole component on top of a different library, then discover that other library has another fatal bug, so now you need to include both libraries with logic around when to use which one, and then that seems to work but you add plenty of tests and documentation to make sure it works that way in prod too and not just on your dev machine.
I don't see what that has to do with learning from previous mistakes. Pretty much all but the most basic programming turns out to be like that -- dealing with unforeseeable and undocumented problems.
(The 20% is because you weren't planning for sick days, an unforeseen emergency bugfix on another project, nobody remembered about the afternoon retreat next week, etc.)
I used to do 2.5, I read somewhere 2.5 was fairly common
Sadly, managers want to believe that you put more effort into estimating deadlines so I'll just whip up a gannt chart retroactively based on the 2/20 rule and they're happy.
Isn't this Parkinson's law in action, i.e. work expands to fill the time available for its completion? That would mean you overestimate up to 2.4x each time.
Edit: math.
Oh, so if I estimate every task as 1 minute I'll finish really quickly?
The 2.4x is to overcome one's innate overoptimism (even if you see yourself as a pessimist), and to add contingency for unplanned complexity, bugs and general overhead.
It's a rule of thumb, not an immutable law, so if one has completed a similar task many times then less padding is needed.
Humans are optimistic by nature. Even somebody as pessimistic as me.
I notice some devs like to underestimate to make them look more productive. Most of those tend to spend a lot of time fixing bugs that QA pushes back to them. They sometimes spend ever more time arguing that it's not their bug.
[1] https://en.wikipedia.org/wiki/Three-point_estimation
https://en.wikipedia.org/wiki/Hofstadter%27s_law
I have no data to back that up though.
This is not entirely implausible but it produces effects even more paradoxical/pathological than log-normal. Here, you could give a correct estimate of the expected time for a project to complete as 50 weeks. You could then reach week fifty and again give a correct estimate of 50 more weeks required. And then you could finish the next week.
[1] https://en.wikipedia.org/wiki/Poisson_distribution
Here is the link to the time estimation described above with Beta distribution. https://www.isixsigma.com/methodology/project-management/bet...
I used it in the past and I think it is a good framework to discover uncertainty and make it more visible, because it makes you talk about optimistic and pessimistic cases.
The estimates are also good enough to come up with a draft schedule.
The three things that still bother me about that idea are:
1. I haven't tried fitting it to the dataset in OP;
2. It's bounded to the right, which seems unrealistic;
3. I haven't come up with intuitive interpretations for the alpha and beta parameters in this context. If the beta distribution means something, then its parameters must have natural interpretations as well.
What I can't do is make bad management hear what they don't want to hear. Nor can I stop people from accepting an estimate because its closer to what they want to be true, because they've already made a promise that conflicts with the actual estimate, or because a certain process requires or necessitates inaccurate estimates.
I think the whole "software is hard to estimate" myth stems from 2 fundamental causes:
- not controlling for human biases or referencing actual real world data - processes that don't punish/ reward people who provide inaccurate/accurate estimates respectively.
- “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.”
- “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chinese and Google Translate isn’t good enough. I’d need to insist on professional translation in order to even estimate the second part.”
- “The problem is tracking down a bug rather than building something, so I don’t have a good way of estimating this. However, I can timebox my investigation and if I’ve not found the cause at the end of the timebox, can work on a plan to work around the bug.”
You need to be willing to endure the discomfort of looking someone in the face, saying “I don’t know”, and then standing your ground when they pruessure you to lie to them. They probably don’t want you to lie, but there is a small chance that they pruessure you to. If you don’t resist this pruessure, you can end up continually giving estimates that are 10x off-target, blowing past them as you lose credibility, and your running your brain ragged with sleep-deprivation against a problem you haven’t given it the time to break down and understand.
But when you advocate clearly for your needs as a professional, people are generally reasonable.
The problem occurs when the client doesn't want to hear it.
"Sure I can do this in 3 weeks."
Then shows up once every two weeks and finishes around the four month mark.
But was not estimating the actual total time until completion which must include all the lead time, waits and dependencies.
"Yes we can do that in three weeks. We need the API documentation for _those_ two systems, network access to _those_ resources, and credentials to log in to _those_ services first."
2 months later we're still waiting on critical pieces of those prerequisites, while someone's screaming that the deadline they'd committed to with the customer was 5 weeks ago.
The most important part of this process is getting it in writing so that you can slap them with it later.
the problem comes when you didn't know you need Y, Z or W until the work actually starts.
1) You improve your foresight so you can get your manager rolling on getting resources early.
2) You have a concise record of what you think you need. If you notice something else mid-project (which does happen even in otherwise-straightforward implementation projects), you have a concrete point at which to raise a flag to stakeholders.
If you have zero confidence in an estimate from the beginning, then there is no point where its natural to realize “oh hey, this is going to overshoot and I need to communicate about it.”
The longer estimations are very much due to changing priorities and a developing environment and understanding of what is being built though, hence the inaccuracy.
You should do all this but there will still be times when people are unreasonable, it's just human nature.
If that doesn’t work...I’m actually not sure. But I just started reading the book Crucial Conversations and it claims to have a relevant answer.
spot on.
In the end, his company went bust because he was selling stuff to customers that could never be delivered on time, or even never at all.
Note well: Automated testing is almost certainly essential. If someone says you are “building an MVP”, then make sure it is an actual MVP. Make sure you have a clear market hypothesis to test and check that it couldn’t be built out of duct tape, Google Forms, Zapier, ITTT, manual processes, and excel-wrangling.
I suppose it's both slower and more expensive than "just ship bad code at full price and wait", but I'd still expect more companies to choose it as the intermediate option between that and comprehensive in-house testing.
I used to work for an online travel site - before every project kicked off, the business analysts would produce requirements documents that always started with a "flexibility matrix". The flexibility matrix had three rows labeled "schedule", "scope", and "resources" and three columns: "flexible", "moderately flexible" and "inflexible" and they were supposed to check one option for each row. So every single requirements document listed "resources" and "flexible", "scope" as "moderately flexible" and schedule, OF COURSE, "inflexible".
I take it these requirements documents were not honored by your non-developers? A document labeling scope as "moderately inflexible" does little if it's treated as written in stone.
The first question is of course whether that's true, since resources don't tend to flex up unless someone outside the project dictates it. But even if it's accurate, it tends to produce the situation The Mythical Man-Month was written to condemn. If the ship date can't move and the scope won't bend on core features, there's a very real risk of finding that extra resources can't fix the timeline problems.
You're right that most organizations have to operate this way, and it's not inherently bad; I've just found that "flexible resources" is often used as a way to avoid admitting that the real third category is "risk of failure".
However, Giving a developer more time will often improve the quality of the output.
Some deals are inherently negative sum - if you bought a lemon.
That highlights a beef I have with economics. Failure to understand that business relationships are exactly that. You can't model them using the sorts of time invariant non-network models they use.
First, Not everything is a commodity with perfect substitutes, arguably most things aren’t. Substitutability goes beyond price.
Secondly, benefits are not necessarily equal or a linear function of cost.
Third, whether a game is zero, positive, or negative sum requires a look at the costs/benefits of the whole transaction. Saying that a buyer gains $1 more if the price goes up is obvious, it that doesn’t necessarily make it zero sum when you look at the overall transaction of a $1 price raise. .
Zero sum implies that price, costs and benefits are equal. I spend $20 on a product that cost you $20 to make, and I get $20 of benefits out of it.
Whereas I buy something for $20 , it cost you $5 to make, and I got benefits of $100 out of it, that’s positive sum, as both the buyer and seller got positive gain. If you raise the price, the deal isn’t as great, but it’s not zero sum.
I worked with a group that designed telcom IC's. One of our customers developed a new product. Which then failed in the field. The parent company had to issue a recall. Destroyed all remaining inventory and canceled the product. All because someone jacked the microphone supplier down about 5 cents on a part.
That was an entertaining job.
Another interesting characteristic he had was to get upset if he thought projects were running too smoothly...
In my experience, most of the time there was pressure. I remember a funny conversation between my project manager and a colleague:
PM: "We need to have X as soon as possible, how long will it take you?"
Dev: "Oh that's, easy, I can finish it tomorrow."
PM: "Tomorrow??? That's impossible, because it needs a and b, no?"
Dev: "Yes you are right... I can probably finish it by next week."
PM: "Next week??? That is a long time! It's only a, b, c and d that needs to be done. Does it all take that long?"
Dev: "You are right, I can probably finish it sooner. I can get it ready by this Friday."
PM: "That is perfect! The customer wanted to put in production on Monday, so we can now confirm. Thanks!"
I overheard that story (open office), and it was a real facepalm moment.
That's just a question. Asking questions is OK. If you have such a high need to please other people and such a low self-confidence that you will change your answer simply if someone asks you twice, that's the problem with you, not them. I say it as someone who have been like that and have changed it through therapy.
I was a nervous mess and always thought that people around me don't like me, pressure me and in general was very nervous around any interaction where I and other person have some disagreement. But turns out, firmly saying "No" is quite OK, people react well to it, respect you for it and nobody is going to fire you for giving good estimates.
But are they ever going to admit that they have no idea and accept your estimate on face without playing BS mind games? Probably not. But for some reason that doesn't stop them from trying.
I've never seen someone try and do this with a home contractor.
It might also be that they’re surprised at the complexity and want to know more. For this reason, the important part of an estimate isn’t the number. Its the task-breakdown and the identification of risk.
In most cases, people who ask "Does it really take that long?" are giving you their first emotional reaction and haven't actually thought it through deep enough to decide to manipulate you. They're upset and they want to make sure that they understood you right - which is a thing people often do when they get bad news, and I cannot blame them: if you get upset about something, at least make sure you really should be upset.
So, you should react either in a spock-like "stupid rational", giving no thought to what they might be implying whatsoever with a simple "yes", or, even better, in an emotionally-aware rational way: acknowledge their emotions and offer options. There are always options and trade-offs, they just need to be clearly communicated. Not "OK, I'll do it in three days instead of five", but "I'll can do it in three days with prototype quality so we can test it out, but it will be buggy as hell, not fit for production, and you should document it". I've never had a manager who had a problem with that, and I despite how popular horror stories are, I really doubt that there's a lot of them out there.
Your manager cares about the product, he's emotionally invested in it and wants everything to go as fast as possible - it's not a bad thing! It's only bad if you view his disappointment as your fault - but it's not. It's just how things are. As a developer, you're often a messenger bringing bad news. There's no way around it, and you have to learn to deliver bad news for people. And separate these bad news from you and your professional self-confidence.
If he was emotionally invested in the product, wanting everything to go as fast as possible is not necessarily a good thing. I can make things go as fast as possible, but then the questions become no unit tests? no integration tests? spaghetti code?
And this isn't a response from a few go around either. This is often the first thing that comes out of their mouth, right at the beginning of the project.
What happens is as follows:
1) Your manager has software development background and uses it as a superiority contest to show how much better they are than you. His lack of awareness regarding variable skill level or lack of knowledge of an existing system seems to factor little into the question. So one could argue the manager could be more empathetic in this case.
2) Your manager knows nothing about software development and really has no business asking such questions in a condescending manner. So they either pretend to know or they promised someone higher up the chain even though they had no idea, but were scarred to say so, thus the cycle has completed.
3) Your manager has a calm, rational head and accepts your estimate, your explanations, and has a good understanding of the product to negotiate with you in terms of requirements without getting emotional. The manager trusts understands that your estimate is, more than likely, not going to be accurate but accepts the changing winds and doesn't hold you to the fire.
I totally agree its better to be explicit. Which is why it matters to avoid overly emotional reactions and accept what people tell you so hand waving can be avoided.
But this is more of a prevailing problem in software requirements in general. It's rare that developers get projects with clear ideas on what needs to be done because of the ignorance or unknowns in questions that need to be asked in order to deliver a product. Often the developer has to decide, and when they do and its wrong, then they have to fix whatever they decided because no one explained, or bothered to explain, how something was supposed to work in the first place.
I'm surprised that you haven't. I've certainly seen the same kind of thing happen, though admittedly it is usually based on cost rather than time. Ultimately, though, the discussion was the client disbelieving a high effort estimate.
Is it?
Questions have built in assumptions and implications. Most questions don't have enough to be of concern, but some questions definitely imply things beyond the question itself. When asked 'Does it all take that long?', many hear an implied 'Make it not take so long.' command. It is not implied all the time, but it is implied often enough. This does not mean someone has low self-confidence or needs therapy.
If you go from "my boss gave me a command" to "I must execute the command and not question it" without exercising independent judgement, then I would assert that you do have low self-confidence. Whether someone needs therapy is a deeper question that we should be hesitant to answer for another person. But therapy was tremendously helpful for me in realising how my personal and professional life were held back by my reluctance to trust my own judgement or endure the discomfort that can come with honest, respectful, collaborative confrontation.
I should make explicit: Part of trusting ones own judgement is also being willing to say, “I don’t have enough knowledge/experience to make a trustworthy judgement here.”
At the time I made my comment the Parent comment had a negative score and I was trying to be helpful to commandlinefan .
> under the (mistaken) belief that more hours = more software. reply
I'd delete it if I could.
https://www.shrm.org/resourcesandtools/hr-topics/employee-re...
In your example conversation, the developer starts off by demonstrating that they are not good at thinking through how long things take them, by giving an impossibly optimistic estimate. Even the PM, whose job is not to figure out how to build stuff or how long building stuff takes, knows the estimate is wrong, and pushes back on it, pointing out dependencies the Dev appears to not have considered.
The dev comes back with another estimate, also provided on the spot, without actually thinking through all the tasks, problems, and potential ways to streamline / accelerate. The PM, having just had the dev's lack of thorough thinking with regards to complexity estimation amply demonstrated, asks the Dev to think more carefully, a third time.
Are we facepalming at the dev who can't be bothered to actually consider what it will take to get the thing done, or the PM who for some reason accepts the 3rd estimate, even though the dev appears to just be randomly generating numbers? Both?
or who doesn't know that "I'll need some time to get back to you on that" is an acceptable part of their vocabulary.
> PM: "That is perfect! The customer wanted to put in production on Monday, so we can now confirm. Thanks!"
> pointing out dependencies the Dev appears to not have considered.
I'm not sure if they're dependencies or other priorities in this case. I've had managers before ask for an estimate and and I've given them one (2 days), then they've assumed that I'm dropping everything I'm in the middle of and told clients it would be done in 2 days.
The problem with estimates is that the people asking for them really want promises.
1. The PM just wanted it done by Friday, because the customer wanted it by Monday. This was clear at the end of the conversation, not at the beginning. So it just could have been a "Can you get this done by Friday? Because the customer wants to use it on Monday"
2. The developer clearly had no backbone, and was just throwing numbers to see what the project manager would like. 1 day? Too short! 5 days? Too long! 3? Perfect! Estimates might be a guessing game, but not a guessing game on how long the PM thinks it should take.
So yeah, facepalm to both of them, and the whole situation. But I must say that they were definitely not incompetent people. The PM was one of the best I worked with, and the dev was a smart passionate coder. It was just that situation that was a bit absurd.
Proper way on how to do this:
PM: Can you give me a deadline (not estimate!) on when you can finish X? The customer needs this ASAP, so it has priority over your other tasks.
DEV: Let me look into it and I'll let you know in half an hour. (Works out estimate, possibly asking input, adds a buffer because deadline != estimate, looks into his own schedule of meetings etc).
DEV: I can get it ready by Tuesday.
PM: I'll notify the customer.
Here you can see the exact moment when an estimate became a commitment.
https://www.youtube.com/watch?v=p0O1VVqRSK0#t=5m20s
EDIT: The part about estimates is here: https://www.youtube.com/watch?v=p0O1VVqRSK0#t=36m56s
All involved engineers actually had the backbone to say it was impossible, but no chance to say it. Obviously, projects did not go well at that place.
Another nice trick if you're under time pressure is to arrange the project so that you can deliver it in stages that partially meet the requirements, or even deliver something non-functional that people can play with while they wait.
This can keep the sales team rolling or customers happy for a while, buying you some time to write the program properly.
As a bonus, you might get earlier feedback if you're going in the wrong direction, in which case less time wasted.
(Obviously this approach isn't always possible. Many problems won't decompose nicely.)
Then you can cut scope by postponing (or not doing) some of the phases. Ideally.
This has not been my experience. People want ‘estimates’ at all costs, tell you to not worry about any accuracy, and then a week later tell your manager you committed to x date.
- https://www.kalzumeus.com/2011/10/28/dont-call-yourself-a-pr...
- https://www.kalzumeus.com/2014/04/09/what-heartbleed-can-tea...
Spoken as someone who has never taken the time to fully master a programming language and, from the sound of it, has never worked with someone who has either. The difference between someone who has spent 6-12 months with a language compared someone who has spent 6-12 years is night and day. From the general tone of the article, he obviously focuses more on the business value than on the technical side and that is a pretty good approach for making money. But I'll take Peter Norvig's advice (http://norvig.com/21-days.html) over this guys when it comes to mastering a language.
To be fair most of the content of those articles is pretty decent though and it is just a pet peeve of mine when people claim that mastery of your medium doesn't matter.
If you are working with an apprentice, whatever you do, it will seem like you are really experienced.
The senior members of his team had a culture of eighty hour work weeks. Draw what conclusions you may.
I like to handle this problem by providing an estimate RANGE.
The biggest thing that has helped me here is to keep a running list of my tasks, an informal Gantt chart, and show it to the manager whenever they ask me about an estimate. This helps me communicate in a way they understand.
Recently I've flat out said no, as gently as I could, to doing some work - porting a software stack to run on Windows.
A customer is asking for an estimate to do it, because a project manager who wouldn't be developing or supporting (or even using) the system likes Windows. The sales manager isn't thrilled with me over it.
So he's asked the team as a whole to estimate it because I wouldn't. The rest of the guys are frontend devs. I don't have a clue, they're in a worse position but for some reason they're all up for putting a number to it so I look like a bit of a !
I'm a contractor, so I can and would just turn down the work, but it's difficult politically.
Dev: "It's a really bad idea, will take an indeterminate amount of time, and be a maintenance nightmare" PM: "Can I just get an estimate?"
Nobody will revisit the GP's reputation, and after the novice devs fail to do the work people will just scatter into different projects. They may or may not take a reputation hit, while the PM that insists so much on doing the not viable work will take none of it.
The only thing that probably is going to happen is "Huh, why isn't that delivered?" and then they will look everywhere but on the fact that it was, at least partially, just a stupid idea in the first place.
Of course I don't know the exact situation so it's more of a thought on what happens in general in the industry.
If somebody else comes out with a much lower estimate, ask them what they think about risk A, B, C... if you've got a point, this will usually work.
I have literally no idea how long it would take though, I couldn't in good faith put a number to it.
I also don't want the work - Windows isn't an environment I have experience in because it's an environment I don't want experience in. They wouldn't allow a rate change to do the work.
Of course, once you're holding certain things as non-negotiable... you're saying no.
https://medium.com/intercom-inside/the-orange-juice-test-c90...
The thing to understand about politics is you can do everything right and still lose.
Here, the damage was already done by their unrealistic expectations. You were either going to accept the job, fail and make them angry, or reject it and make them angry.
Sometimes your least worst option is to cut your losses.
It isn't just software that goes double, look at most custom building projects or government contracts. Most everything takes twice as long and costs twice as much.
I read something once that has stuck with me. "We can never give more than a vague guess because we are literally building something that doesn't exist and has never been built before."
* I do find it annoying that the sales teams generally drive this. I wonder how they would feel about a response of "We'll get you an estimate of when it will be available as soon as you get us an estimate of when this feature will generate the $1M of revenue to cover the costs of building it. Also, please give us a list of customers that are likely to buy it and your expected contract date so we can track it along with development."
There's definitely a correspondence here, but it's a bit indirect.
Government and even private contracts are often knowingly lowballed to secure the deal. Rules like "must choose lowest bid" reward entering an unrealistic bid and then inflating costs after work is going. Or, if overruns come with sufficient penalties, doing work at a loss and then charging a fortune to support a fragile and proprietary result.
In the metaphor to non-contracted software, developers generally don't correspond to the contractors but to the government. Sometimes blown estimates are simply errors, but other times somebody wanted to secure customers, so they pitched an unrealistic scope and arranged to make the inaccuracy land on the programmers. Where the government ends up with late delivery or runaway costs, programmers end up working crunch time or sacrificing quality. And where the government ends up locked into paying for support, programmers end up locked into maintaining rushed, tech-debt-riddled work.
(As for pushing sales for revenue and contract estimates, do it! Some of the most productive teams I've been part of have been ones where sales applied this pressure, but also responded well to the reverse version and kept everyone informed.)
The book contains a quiz that we used as part of a training exercise with management and the results were hilarious. Here is an online copy: https://scrumandkanban.co.uk/how-accurate-are-your-estimates... please don't look at the answers, I guarantee you will have fun completing it.
For clients, my preferred approach is to show "The Cone of Uncertainty" and ask where they think they are. Since most people have no idea what they want to built I ask if its ok if my estimates are 4x off. That usually gets me a few weeks of peace while a team comes up with a product definition and we start all over again :)
I’ve gained some experience and confidence at this in the years since, but at the expense of some psychologically painful experiences and a firing. Our industry really calls out for a more basic book aimed at University CS grads (or bootcamp grads or MOOC)-grads on how to estimate tasks.
> Total length of the coastline of the Pacific Ocean
Coastlines are fractal. If your measuring stick is 1 km long, you'll get one number. If it's 1 m long, you'll get a much larger result. And if it's 1 cm long, you'll get an even larger result.
https://en.wikipedia.org/wiki/Coastline_paradox
The actual length of a coastline is infinite when measured with an infinitesimally small measuring stick.
Software development effort also increases, unbounded, as discovery proceeds and the precision of requirements increases.
>In actuality, the concept of an infinite fractal is not applicable to a coastline; as progressively more accurate measurement devices are used
The notion of the length of the coastline, and why I think the question is apt as well, is that regardless of the fractal paradox, the practical answer is that there is an answer that is accurate enough to satisfy the business's or the customer's needs.
The analog for software projects, from my point of view, is scope, and it's generally one of the biggest problems with projects that take too long. Product and various stakeholders take outline broad, high-level scope requirements that may or may not actually be informed by the realities of building or delivering that piece of software. They generally kick the can down the road and take a "I'll know it when I see it" type approach that winds up drastically changing things at some arbitrary point later on on the project.
A rough measuring stick for scope always means that the answer will be longer, however, small measuring sticks for scope are absolutely impractical. What's needed is the understanding from the business that refining scope increases development time, and the high-level broad scope that is sufficient at the beginning of the project is not sufficient at the end. Many businesses want to run with the assumption that their broad scope and the estimates that go with them are accurate enough to bring a product to market.
Going to read it. thanks.
I think out of 15 people nobody got more than 1 or 2 right, except for my manager, who somehow got about 8 right. I'm curious how the HN crowd does on this - care to post your scores?
This is my default. Even when I am fairly confident I know what the code does, I want to double check. And it isn't just so I don't short change myself on time. If I tell them X days and deliver on 0.X days several times, they'll start thinking I overestimate everything and cut me off at the knees. It pays to get as accurate as possible.
People talk about project management, gantt diagrams and all that but you have no idea how to estimate even grossly the complexity of a thing.
Lots of people say “just do it and multiply by 3” without appreciating that The important part of estimating tasks is coming up with the task breakdown and identifying the risks.
For one, it skips the step where you break down the tasks. For another, it seems likely to ignore a bunch of variables in the search for something similar enough.
Unfortunately, the problem then becomes defending whatever timeline you have in mind for non-linear work. also doing too much of this can give bully managers an opportunity to nitpick and micromanage the whole thing.
This. I have PMs who will ask me over and over until I give a number and I've learned to stand my ground. Because if I don't, I end up being responsible for the estimate I've given (as I should).
Now I make it clear that I will not give a number until I know more. Just a few weeks ago I was asked to give an estimate of how long it would take to do an integration with something I've never used before. I said, "I need 1-3 days to learn the product enough to give an estimate". What I get back is "can't you just give WAG". But this time around I said "You have two choices, 1. you give me time to learn what I need and then I'll give an estimate or, 2. Find someone who already knows the product to give you an estimate"
Where does the "Scientific" come in? Because I really thought it implied having a good idea, which made SWAG seem like an oxymoron.
You use things like Average/95/99, t shirt sizing, how much work can be done in parallel, 3rd party/client dependencies, ramp up time for a new library ... it's still a wild assed guess but at least there's some basis in reality and you can defend your estimate - show me what I'm wrong about or missing and I'll revise my formula.
Negotiating over the assumptions and inputs is 90% of the battle anyway. Without a model you're standing on sand.
You're not responsible for an estimate you give, the person who's asking for it is.
This is the key difference between an estimate and a commitment: If you give someone an estimate, they are responsible for steering the project using that information. If you give someone a commitment, you are responsible for meeting that commitment.
This is why very few people actually want estimates. They want commitments, but they will ask for estimates to get an answer and then interpret them as commitments. Most people asked for estimates know this intuitively, thus "doubling and adding 10" and other communication strategies to turn estimates into safe commitments before they are communicated.
One big insight for me, which you obviously put to use in your story, was learning to give a different number.
"I don't know" is understandably frustrating to people who are trying to plan work and answer requests. They often respond to that by pushing for a delivery date even when it won't be accurate, but a lot of the time what they fundamentally want is a numerical answer. "I'll have it scoped out by Tuesday", "I need at most 3 days to learn enough to estimate", and so on all aid that planning without committing to a project timeline. That rarely ends the conversation, just as it didn't in your example, but it does a lot to show good faith and justify not answering the initial question.
I had a friend who had an unusually high frequency of conflicts with various management, project managers (I expect there are always some, but this guy had a lot of bad experiences).
He showed me an instant message conversation:
PM: "Is X done?"
Dude: "Yes, but a, b, c is not done"
A, b, and c were parts of X's requirements.
Now the project manager or such should have been able to read between the lines, but man don't lead with "Yes" and actually describe a situation that is "No".
If you validate the invalid assertions of these abusers (e.g. that some unclear task can be estimated with certainty) then you're enabling the abuse.
Do you have a good definition? One whereby a scrupulous person can confidently judge “am I acting abusively?” correctly?
- the scope of the task
- the resource/people you have
- the confidence in the resource/people
- and the dependency on both the resource and the task
Estimation goes wrong in many ways. In my experience, I categorize as follow:
- estimated by non-technical PM, unable to gauge technical/business complexity
- estimated too early in development life cycle. Often time someone come up with idea X then ask right away "shouldn't be difficult, how long will it take"
- no view on team member's productivity due to lack of measurement or measuring the wrong thing
- not considering dependency in the development cycle. e.g. The question "whether our only backend engineering available for the task" is often omitted
- not considering testing, documentation, 3rd party integration/procurement, maintenance and deployment cost
The common practise of coming up a number, then double, or maybe triple it up to make the estimation does not address why estimation is off. Mis-estimation is a symptom. The cause and the cure lies in the people
E.g. Dev estimates that time to complete each of A, B, C tasks will be 2 days. In reality, A will take 1 day, B will take 2 days, but C will take 8 days.
Dev was right about the median time to complete each task (2 days) but average was much higher. Article goes into how to statistically model the distribution of actual time to complete tasks.
This takes advantage punters' failure to accurately multiply probability - 10 events with 80% probability have joint probability <11%.
Something similar happens with planning, where people fail to compound many possible delay sources accurately.
Dani Khaneman covered this in Thinking Fast and Slow, also showing that people overestimate their ability, thinking they will outperform a reference class of similar projects because they are more competent.
https://en.m.wikipedia.org/wiki/Planning_fallacy
It's profitable for bookies because they have a house edge, and that edge is increased the more subsequent bets you make. The house has more edge with an accumulator than a single bet.
People like to do accumulators because it's more meaningful to win large amounts occassionally than more regularly win less meaningful amounts.
So it's a "trick" to simply increase gambling.
If you had to pick out 8 evens shots in sequence and intended to roll over your bet each time it would have the same effect / outcome, but starting with a pound by the last bet you're effectively placing a 128 quid bet on an evens money.
It's not that the player thinks that they have a better chance of winning than 1/256 it's that it effectively forces them to gamble a lot larger amount in the situations where only 7 out of 8 of their picks come in.
And that's before considering the edge. If we consider that these are probably events that happen more like only 45% of the time (at best) then instead of a 255/1 shot we're looking at 600/1 shot.
Same size your stories to small values. Do 30 stories in a sprint and take the mean. Do 30 sprints and take the mean of the sprint. What you get is the mean amount of time to do a sprint of 30 stories. What's amazing is that this estimate will be normally distributed. You can measure the variance to get error bars.
Of course 900 stories to get good estimates ;-) However, imagine that your stories averaged 2 days each. Imagine as well that you have a team of 10 people. That means that you will finish a "sprint" of 30 stories in 6 days (on average). 30 sprints is 180 days -- the better part of a year, but you probably don't need a 95% confidence interval.
You will find that after a few sprints, you'll be able to predict the sprint length pretty well (or if you set your sprints to be a certain size, then you will predict the number of stories that will fit in it, with error bars).
The other cool thing is that by doing this, you will be able to see when stories are outliers. This is a highly undervalued ability IMHO. Once a story passes the mean plus the variance, you know you've got a problem. Probably time to replan. If you have a group of stories that are exceeding that time, then you may have a systemic estimation problem (often occurs when personnel change or some kind of pressure is being applied to the team). This kind of early warning system allows you to start trying to find potential problems.
This is really the secret behind "velocity" or "load factor" in XP. Now, does it work on a normal team? In my experience, it doesn't because groups of people are crap at calmly using statistics to help them. I've had teams where they were awesome at doing it, but that was the minority, unfortunately.
I think those really long tails are more of a problem when you are working with "features" that are much longer. If you have 1 day stories and you've been working on the story for a whole week, you know you have a massive problem. It's time to back up and see if there is a way to break it up, or to do it differently.
If you have a feature that is a month, by the time you get to 5 months, you have so much capital invested in the original plan that it's very hard (politically) to say, "Nope... this isn't working out. Let's try something else". Of course, it is very hard to get your organisation to plan to a 1 day level of granularity.
1. https://www.joelonsoftware.com/2007/10/26/evidence-based-sch...
I don't know how Spolsky is (was) as a boss, but it's is way easier to just adjust your time reports according to the estimates and make your boss happy than to actually try for no benefits to either you or the project. As long as the estimation errors are white noise-ish and in linear relation to actual task length, which is the assumption for his model to work, manipulated time reporting and not manipulated make no difference.
I've noticed that when programmers are introduced to Agile they initially are bad at time estimates until they learn that you actually can report time according to the estimates and no-one will ever notice. Extra points for looking on the burndown chart before reporting time to make it smooth.
"Some developers (like Milton in this picture) may be causing problems because their ship dates are so uncertain: they need to work on learning to estimate better. Other developers (like Jane) have very precise ship dates that are just too late: they need to have some of their work taken off their plate."
Milton is probably doing fair estimates. When planning four months into the future you probably can say where you are within 2 months with 50% probability (or what ever boxplot he uses) of where you think you will be at that time. Janes reporting is obviously manipulated to match the estimates.
At some point, I came to the realization that my fundamentally poor concept of time was always going to be an insurmountable obstacle to my career advancement.
As stated in the summary, the core driver for inaccurate estimates is uncertainty:
> Tasks with the most uncertainty (rather the biggest size) can often dominate the mean time it takes to complete all tasks.
There are different sources of certainty:
- Experience: If someone has done a task 20 times he probably knows how much time he will require. Someone who hasn't done the task yet, probably underestimate the time he requires (e.g. because of the median vs. mean conflict). But don't be fooled just because you have 20 years of work experience in the field but never actually done a specific task doesn't mean you can estimate it better than someone who just started the job but had done that specific task 10 times. However, most of the time, projects are doing something new. So you have to find out which tasks have been done before by a project member and which are completely new ground. If something is completely new, remember to plan time for getting familiar with a problem space plus a handful of complications (together this will be more than the actual task would take someone who is trained for that specific task).
- Detail: The smaller the tasks the larger the overall estimate... or so. Planning on a top-level is rarely going to be accurate. We do it a lot because it doesn't take much time. But if you want an accurate estimate, you have to plan on small, specific tasks.
- Risk management: Every project has risks. Some don't really have an impact and others blow up the whole project. Know your risks and what you are going to do if something should go in the wrong direction. It is not like you wouldn't have time to figure out what to do when the problem occurs but to understand how it would impact your timing and to take preventive actions (e.g. include stakeholders).
If you have people who have done the exact same task a few times, made a detailed plan of every step and know how to handle the most likely or impactful risks you are in a good position to deliver on time. Most of the time you won't have that luxury and have to compensate the resulting uncertainty with a prolonged time to project completion, but that should be just fine when being communicated in the beginning.
With all that said, remember, that some projects don't require an accurate estimate. Sometimes it is enough to deliver just as soon as possible.
Now, i always give two estimates: An estimation if the work is done by me, and an estimation if the work is done by someone else.
It makes it really hard to calculate the "expected value" of 5-10 tasks.
1) The person(s) doing the estimate aren't qualified to do so.
2) There is a disconnect between wishful thinking and reality.
3) There is some arbitrary future milestone (i.e., "We need to ship by ____ because ____ is happening the following week.") that is independent of software development.
4) Most importantly, when the deadline is missed the estimate / estimators are not questioned, the software team is.
I've been at this a long time - too long? - and the narrative that IT is __always__ at fault is a myth that needs to be buried.