Imagine you are going on a hike to the top of a mountain. Countless paths exist between you and your destination. Most of these are dead ends. Every dead end forces you to backtrack, sometimes all the way to the beginning.
Then, on that Hike - Someone decides that the fastest way to the top is to hire 2-3 more guides. But before you can give those guides any meaningful work, you have to take them through all the previous bad paths...
That or the part where when he first calls his friend and says "We will be there in 10 days" and his friend says "10 days! really? I was expecting you here in 5 days"
Because management looks at you funny when you say something simple will take 4 months. Then they lie and say that it couldn't possibly take that long. And the reason they lie is because there's no negative to them tying to you. Their job is to get you to go faster, and they are allowed to lie as part of their job, if it makes you go faster. The reality is that it doesn't make people go faster. It makes people leave faster. It makes people give up.
Never give an estimate to management. Estimating is their job, as they control the inputs to production. If you give an estimate, they will pressure you to meet it, no matter how unreasonable it later turns out to be. That also makes people want to leave.
While I agree with many of the other reasons and believe people in general can't estimate very well, I believe that for businesses this is indeed the fundamental reason. Even in a relatively understanding and engineering-centric environment, when I give what is probably still a bad under-estimate, I will get pressured on how we can reduce that number. I can't even imagine what it would be like in an environment that wasn't relatively understanding and engineering-centric.
If you don't want to hear the truth, you won't. Humans have known that since long before software engineering existed... or a subset of them, at least.
Management giving estimates? No thanks. I'd rather them take my input and do their job of understanding trends. When I say something takes a long time and it takes longer, I'm not bullshitting. When I think something takes a long time and doesn't, its generally because shortcuts are rarely understood up front. Will I find the same shortest path next time? If I'm doing the same exact work maybe.
I do work in a place where we are a software company but very few people, even some of the devs, understand this culture. I don't say something is complex just for the hell of it. I want to give good estimates but at the end of the day that is all it is, a somewhat educated guess before I've gotten in the weeds. I don't need people that absolutely don't understand this concept to mandate a timeline that is unrealistic. It does absolutely no one any favors. That is why I leave. I don't need to promote unrealistic expectations down the line to customers who think everything is extremely simple. Especially when all we do is create custom software where almost 0% is turn key or off the shelf. Its just insulting to continue that nonsense and the quicker I can facilitate a reality check for all parties, the better and more trust is given to my judgment (and theirs if they actually take the time to learn)
I have done it using the XP planning practices. Basically, you make the suit break everything down into relatively small lumps and place them in priority order. Every week, the team completes a few lumps. Before you do them, engineers grade their relative complexity in arbitrary units. (The smallest substantial thing you do is 1 point; something twice as big is 2 points, and so on.) Every week, you count up the points completed. That's your "velocity".
From there, you let managers do all the estimating they want to. If they want the complexity of a unit of work measured, they ask the engineers. If they want to know when X will be done, they look at the team's recent velocity, what's in the queue before X, and do some basic math.
The nice part about this is the mental judo involved. Whenever they want to know when something will be done, it's their problem to trade features against time. Working like this, it's not geeks vs suits; the suits channel their schedule pressure into productive work: grooming the backlog.
I misspoke. What you are describing comes from developer feedback but indirectly in a sense. This, I think, is a nice standard of measurement. Its when people solicit absolutely no feedback either in the form of past projects where hours are measured somewhat or what they just "feel" something should take. I'm all for metrics based estimation because that's how most developers would likely estimate. Its when management seems to pull things out of their ass to get a prospective client I have a problem with. I understand when we need money but I can also trace some of our worst clients to some of the most unrealistic estimates we've ever given. They're almost a 1:1 direct correlation and its like no one sees how much of a drain they can be all around.
I have been there, and feel your pain. It has taken me a long time to learn to say, "Oh, you told somebody that? Well, then you have a problem, don't you?" Of course, there are some companies that are so pathological that this stuff just won't get better: there's a broken feedback loop between the promises and the consequences. Ugh.
If you think about it, though, estimates (time, money, personnel) is really a management function.
You write "I'd rather them take my input and do their job of understanding trends." that's exactly what I'm talking about. They need to take your input, and do their job.
If a worker is told: "Here is what you (personnel) have to do (task), with these tools (environment), and we want you to do it in this amount of time (money)", and the worker fails to complete the work in the amount of time, then it is management's fault. They improperly estimated the amount of time it would take the worker to perform his task.
That management ask the worker how long the task will take means management does not understand the task, and if management does not understand the task, how can they know whether the worker's skill, experience, and knowledge is adequate for the task? Now, they also seem to not understand the worker's skills, experience, and knowledge, so the problem is compounded.
It is a management function to clearly define the tasks that must be performed, attract people whose skills, experience, and knowledge fit the tasks, and fund these people with the appropriate environment, tools, and salaries for them to perform the tasks.
That is the role of management. If they are unable to estimate what tasks must be performed, what people need to be retained, and how much money and time will be spent, they fail at their managerial duties.
That they then blame the workers themselves for improperly estimating the work is doubly wrong, a sign of managerial immaturity.
"Never give an estimate to management. Estimating is their job, as they control the inputs to production."
Unfortunately, it gets even worse if developers have no part in making estimates. If management doesn't know the details of your architecture and code-base (or even the other projects you may be working on, if you go high enough up the management chain), they have no way of coming up with a rational estimate on their own.
Where I work, VPs are constantly making "commitments" to customers to have certain features implemented by a certain date, and only notifying developers after the fact. Then they wonder why the development organization is "so bad at meeting deadlines". (Needless to say, I have no ethical requirement to honor commitments that weren't made by me and are frequently impossible to meet anyway.)
I admit it's funny but if I showed this to a non-programmer friend, he'd be asking why I didn't research my route better or start on an unknown trail with minimal planning.
Because the research will take a significant part of the project time. Once the thorough research is done - third-party libraries are verified (that they do what's needed, and not too buggy), prototypes are written (to confirm libraries and internal dependencies work as expected), proof-of-concepts are run (to actually see that it's possible for a reasonable effort), time is measured (to confirm the way to go), interfaces are clarified (as part of the research, specifically, in the architecture department), results are estimated (basing on all results above), specifications are formalized (and agreed upon) - there is an awful lot less of what to do.
According to DJ Wonk's Law (shamelessly self-named), when discussing software estimation, it is inevitable (only a matter of time) that someone will mention Hofstadter's Law.
You need to calibrate your "optimism factor" as most of the time you'll estimate by the wrong amount. I chalk this up to people being optimists, that is, ignoring a lot of the junk involved in completing a task that they'd rather not do: Meetings, documentation, integration issues, testing and so on.
For most people it's around 2-5 but for some as high as 10 to 20.
You might feel like an idiot when you tell people "Hm, change the color of the logo to a different shade of green? Six hours." This is probably how long it will take, though, for that so-called "five minute fix".
As a developer, I have pressure to keep my estimate down. I can give you a guaranteed date with few problems, but the project is going to end up more time-consuming than what another developer is going to estimate.
The good news is that I will usually significantly beat my estimates under this system. However, I will then be accused of padding my estimates and will be given more aggressive targets, some of which I will miss.
The best way to combat this is to give me a team that has worked together on multiple projects within the domain, and account for changes and ambiguities as part of the process.
So the article tries to describe the hidden complexity of software with a hiking analogy but then it leaves a huge part missing. Let's extend the analogy somewhat... this isn't your first hiking trip. In fact, you've been on dozens and dozens of hiking trips. You're an experienced hiker. Yet, why do you continue to give meaningless as-the-crow-flies estimates of how long it will take you to get to your destination? The reason is that if you miss your estimate, it probably won't matter. Your sherpas will be there at every stop with your food, water, and tent. You have nowhere to be any time soon, so estimating your time just isn't important enough to worry about.
Developer estimates are regularly off because they seldom impact the developer directly. Experienced development managers will pad the hell out of the developers who give them the worst estimates. Most developers will explain all the complexity of what threw their estimate off without acknowledging the huge mistake of not anticipating extra complexity in the first place.
My estimates in my early career were no better than anyone else's, ie way off -- especially for more complex projects. I'd explain what happened to my managers and soldier on. The very next task that came up, I'd give my manager some best-case estimate of how long something would take and the cycle would begin again.
That all changed once I started to do consulting for myself using "not-to-exceed" pricing. The first multi-month project I did killed me. My effective hourly rate went down to sub McDonalds levels and took much longer to deliver than I had expected. After that project, I did a post-mortem on the project to figure out where I went wrong. I came up with several spreadsheet templates and checklists to run through before giving any more estimates.
Mostly I just concerned myself with getting a hell of a lot better at estimating project duration and difficulty. Like most things, when you really pay attention to it and practice it, you get better at it.
"Developer estimates are regularly off because they seldom impact the developer directly."
I think I most disagree with this. Developers, in my experience, are quite often asked to leave the company after such estimates.
It's good to have a template - it helps not to forget things - but it is inherent in the work to have unknowns. Large unknowns. A good consultant would probably refuse to work on a project which would require him to use several major technologies which aren't familiar to him, but developer in a company may not have such a luxury as selecting a kind of a project.
Developers, in my experience, are quite often asked to leave the company after such estimates.
I've seen developers let go because they weren't very good at writing software. I've seen developers let go because they were painfully anti-social to the point that they were negatively impacting the rest of the organization. I have never ever seen a developer let go because their estimates were crappy.
Our experiences obviously differ. Usually it goes like this - the developer gives his best estimate (which is, by the way, hard by itself - a lot of things has to be taken into account), and that estimate is considered too high. So the developer is ordered - in one form or another - to, effectively, "do it faster". Often it's by cutting corners in places deemed least important - but then it also reduces probability of the correct estimate overall. One more thing - specifications are quite rarely are good enough - there are other reasons why that's the case - and the final result causes management to wonder, why it doesn't include this, of why that works this clumsy way. So nobody's happy - and developer pays the price. It may look as the developer isn't good at writing software... because "writing software" is a sort of encompassing figure.
That's not bad estimating though. That's psychopathic management. The developer gives an estimate which is "too long" for the manager, who responds by essentially ignoring the estimate and laying down arbitrary deadlines. Surprise, surprise the arbitrary deadlines are missed, and the psychopathic manager blames the developer.
The developer's ability to estimate accurately is not an significant factor in this scenario.
I'd agree with this. I've seen Project Managers let go because projects overrun, but never a developer because of bad estimation. And this is based on about 20 years experience in Investment Banking IT, not the most cuddly of environments.
well maybe it doesnt happen at a "developer" level, if that's what the management hierarchy calls it. But it definitely does happen on an engineering manager level and is actually fairly common in the VP Engineering role.
Right, I've seen that and I'm okay with it. At the individual developer stratum, you expect the level of naivete that was in the original article we're discussing.
At the VP level, you're expected to understand what the Developers and Development Managers don't know about the SDLC. You're expected to bridge the gap in their lack of understanding by instituting processes and controls while mentoring them to become better at what they do so that the organization can succeed.
Been going at it for a couple years now with consulting and freelancing. I've finally gotten good at hitting my estimates. What worked for me was to keep strict time of everything (shoutout to toggl.com, love it!), so I could learn from my mistakes.
The trouble is now though, my competition seems to be underbidding me, but in reality they're providing those ~33% estimates they will never realistically keep, while I'm at ~100% estimates. Not really sure how to relay that to clients. One of the many reasons people like me need a salesperson in front, I guess.
If you're really good at estimates, try to play that you can keep not only the final delivery, but also the partial ones. Something like "you can be sure I'll get 100% done because you can check when 10% is going to be done". Sure it's not a guaranteed sale - and it's harder to keep both partial estimates and totals, comparing to only totals - but it's still something.
What worked for me was to keep strict time of everything (shoutout to toggl.com, love it!), so I could learn from my mistakes
Right, that was part of my method as well. I learned to keep really good notes of each part of the project. At the end of each project, I compared my "what I thought it would take" with "what it actually took". In subsequent projects, I tried to match up similar complexity items with "what it actually took" notes to remind myself of the pain.
Same! And I'm sure the reason it didn't take that long with getting good at it, was it hurts like hell when you do the 33% thing on a couple large projects, when you're on your own.
You're probably losing more business than you think - customers have no idea who is accurate or who is just faster, etc. They will generally go with the lowest quote regardless of other factors unless they've been burned before multiple times. Most people haven't.
I don't have a solution here though other than noticing that underbidding and then getting skilled at convincing clients to do paid extensions later actually appears to make the most money at the cost of your ethics. I'd avoid that approach, but it does seem to work for a lot of companies.
My personal approach is to quote for very bare projects with only the bare essentials (eg, poor UI design, minimum possible feature for the client to see what they're asking for, etc). This can usually be done a lot cheaper than most people think as 90% of the work is in the last 20% of the features. Then once the client has something, you can give them a quote to touch up the parts they need. Basically you split the project up into many small projects each with their own quote which helps you to estimate tasks as they appear and helps your client to minimize costs by leaving off features that are more expensive than they initially appear.
The other side of that coin is that often the clients I get via referrals have no reasonable alternatives. They often have done all business through referrals and so the alternative to me is the open market which is intimidating and has a significant barrier to entry. The end result is that we build a solid and stable client relationship that generally only gets upended if/when money runs out.
It's a great position to be in, provided you have the throughput to expand to other clients and aren't dependent upon a single client for income.
It's common in the fixed-bid world to deliberately underbid and charge exorbitant rates for the inevitable change orders. There's an opportunity to compete there, but maybe not an easy one.
Good luck. I found myself in a similar situation with pretty good estimates, but unable to bid low. I'd be bidding double or triple the competition. One time, I was bidding at triple, and the previous dev had failed totally. They went with another dev who bid nearly the same price. He failed, too. By that time, I was like, I'm not even going to bother - they don't want to pay what it takes, and buy failure over and over.
Have you shared these spreadsheets and lists online? They sound like something that could turn into an excellent, three-figure or more information project.
Yep, I agree estimates are a lot more important when you work for yourself. I've had at least one client comment on how pleased he was that my estimates were usually accurate. I know another developer was giving him answers like, "That will take 5 minutes!", so perhaps his expectations were low. :-)
On the other hand, giving realistic estimates I often hear clients say, "I asked around, and other folks say this should take half what you're quoting." Usually this happens before the work, but I once had it happen in the middle of a long-term engagement after I delivered a two-week invoice, and they were disappointed I hadn't accomplished more. So if I'm not careful I can lose my clients' trust, which is my highest asset. I've gotten pretty good at explaining why a job isn't 5 minutes (or 1 day, or whatever), but I might save this article as something to share if a client doesn't have much experience doing software projects.
I'm glad you got better at it, but I don't think software estimates can ever be very good. Nor should they be.
That a project is estimatable has to mean that novelty is low and predictability is high. But low novelty is a sign of duplication, which should be factored out into things like libraries, frameworks, and self-service tools. High predictability is either a sign of duplication or it's a sign that people have basically agreed not to learn anything during a project. If you are shipping early and often and study users and metrics as you go, then you will always learn things that affect the plan. Which means that the schedule isn't predictable even in theory.
First of all, don't overly aggrandize what we do. 99% of it is derivative in most ways. Although I've enjoyed doing the work on many of my projects, I'm not under any illusions that what I was doing was ground-breaking.
Besides, most of the things you mention can be accounted for. If part of your development process involves iterating through the design with the customer every couple of weeks then you build that into your estimates.
Likewise, I always build documentation and code handover support into any serious project rather than acting surprised that customers will want such a thing and expect it to be part of what I'm delivering.
99% of what some people do is derivative. But if what they're doing is software, that's expressive duplication, and it's worth trying to DRY it up.
I think we have different intuitions here because it sounds like you're more in a service business than specifically writing software. I agree that a lot of any service business is standardizable, because it's mainly about people and their needs; that has a lot of regularity.
But I don't think the software creation part of a service business is standardizable over the long term. During the first wave of "put smallish businesses on the internet" each web site was custom, hand-rolled software. Early on those schedules were unpredictable, but for a while, it became a known, predictable business.
That business has, in the long term, been basically destroyed. People spotted the regularities and developed common code and tools. What was mainly a problem of software development became a (much smaller) problem of installation and configuration. The competitive advantage for those people now lies not in coding ability, but in customer service and in helping people manage the essential complexity of the domain.
Perhaps you weren't around in the early 2000s when the RAD tools were all the rage.
Perhaps you missed out on the colossal frameworks of the late 2000s when everything was a factory and understanding HTTP was actually a disadvantage as the whole thing would blow up if you actually tried to access the request body.
We went down the road you talk of. It was horrible. Now the pendulum swings to the opposite ends, the light-weight APIs which don't try and abstract away all the details which it turns out mattered a lot as everyone has to do pretty much the same thing, but ever-so-slightly differently. Tiny tools that do one job well.
All you are talking about is writing HTML and doing server config, neither of which are software or programming. When someone in 2000 came to you and said 'I need a website, therefore I need a software programmer to write html and setup a server' they no more needed a programmer back than then they needed one today. Just back then it was developers who knew the markup language and how to configure servers and they weren't about to turn away silly money just because there was not much actual coding involved.
I was in fact around in the early 2000s. And the early 90s. The pendulum swings some, but back in the early days of the web, everything dynamic was hand-rolled, just like I describe.
I do flat-rate projects like this using a library that I have to fill in gaps myself. Even in the context of very high unpredictability, I have to make good predictions else suffer low compensation exactly like the parent poster did.
I like crow-flies vs hiking analogy. I'd like to add upon it for so-called "green field" programming as it relates to getting cost predictions correct and understanding programmer time growth in relation to program size. Some parts are linear. Some parts are exponential.
Before I go on talking about "hello-world" space, note that I do architecture and really like architecture, big-picture solutions etc but I try only to solve what is apparent and then iterate on it whenever I notice two pieces tangling. There are excellent wins to be made here, but a working program itself constitutes part of the information necessary to arrive at the final architectural decisions. This is why even on FOSS code, I try not to sweat the architecture on the first pass -- when I do, dead code results. It calls out to me that I have solved something not relevant to the emergent implementation.
Once I've decided what the API might look like, what data structures have to be accessible to what, and what the minimum program states are, I try to stay in what I call "hello world" space. The idea is that the code I'm writing is never doing anything more than one problem at a time, line by line if I don't know exactly what a line does.
This is incredibly efficient because I deliberately break my problem into something testable and well understandable at every step. However, it's impossible to write production code this way. Not only is test feedback, (printlining and frequently more complex testing) not part of the final behavior, but more importantly, the elimination of concurrence of many states, the overlapping of problem workflows in execution sequence, implicitly says that "hello world" space problems are not production code. They have eliminated some functionality or consideration so as to make "hello world" solutions utterly oblivious to concurrent states or processes.
However, two "hello world" problems that need some coordination themselves creates a third "hello world" problem to implement their coordination. The third problem isn't apparent until the first two are explicitly solved. This implies hiking from the article.
Solving "hello world" problems brings more of the problem domain into "hello world" space. The concurrence problem itself will become apparent as a "hello world" problem when it has manifested itself. If there are two states that have been independently implemented but can overlap in execution, there is at least one set of logical statements for dealing with the maximum concurrence of the two states. Growth for N states is exactly an NxN truth table unless some of the states are sparse. In the worst-case, for N "hello world" solutions, one layer of abstraction where all states depend on all other states will result in N^2 logical blocks. While in practice this concurrence is usually much lower, the growth in abstraction is exponential if one is to keep breaking each problem down into an independently writable, testable piece.
So my conclusion is that growth is pretty much linear in proportion to writing "hello world" problems and exponentially proportional how many layers of integration are necessary. Slamming code down ichi-geki style is linear for problems that don't have a potential for overlap and geometric for ones that do. Code that eliminates duplication of routines through logic creates them in implementation, but the payoff is a net gain in many ways, so of course it still makes sense to do it.
As for making predictions, while running my internal monte-carlo, the concurrence of states has a higher influence on the outcome sometimes than the raw code size I expect to come out in the end. Linear ...
I used to estimate small websites (<70k lines of PHP), and generally got to around 30% of the final time required. My method was to fully describe the product, then break it down into tables, queries and pages. Estimates were based on the previous project's stats and final times required. (My constant used to be 3, but technology changes altered the numbers.)
The big unknown - and the highest risk - was managing integration between external systems, or doing complex integration between internal systems. Anything that might benefit from aspect-oriented programming, like access control lists, or a more elaborate front-controller, or universal exports in json format, or a new paradigm like AJAX, would be difficult to estimate.
That fits your breakdown of projects into linear parts, and exponential parts.
Anyway, my estimates were good, but my business management was not - I was always in learning mode because I never studied business and had no experience. So I went W-2 and went into a different field because an opportunity to learn politics emerged. What's interesting is that successful political campaigns are entirely about estimating and budgeting, in real time more or less - you spend money and see results day by day, or after a couple weeks. It's driven by statistics (and messaging). I just watch from the sidelines, and don't do politics, but I think we could learn something from it. It's more like agile XP than waterfall.
The Obama campaign was XP, by necessity. The Healthcare.gov website thought it was waterfall... again by necessity.
The hiking analogy works pretty well, extended to traveling in general.
I spent a couple of years traveling around to visit friends and family all over the US, and regularly my estimates were off in when I'd arrive somewhere. My software delivery estimates are similarly not perfect, in the later direction like most people.
My conclusion was that my estimation of how long it takes to do anything is off a bit. For me, this is because I'm trying to say I'll do something sooner to meet the perceived needs and desires of others.
What makes the hiking analogy perfect is that people are so accepting of it as an analogy. My first thought: Why the fuck are you hiking to Los Angeles?
Plane, train, or for adventure, Greyhound. A sailboat makes more sense. A bicycle makes more sense. And if you insist on walking, at least follow a fucking road instead of cutting cross country.
If you decide the first step to driving to LA is mining iron ore to smelt for casting an engine block and building a Bessemer furnace for the steel for hand forged pistons, then there's lots of shit that can go wrong, EPA permits and zoning laws, even if you've done done it before.
The reason software production estimates are bad is because they aren't taken seriously. The guild of programmers can say, "we shall get from A to B by cross country hike."
Yes, the analogy is perfect because its absurdity is acceptable.
Part of the problem is good estimates take time. For many things you can't give a good estimate after 10 minutes of analysis much less the 10 seconds some people ask for.
Because of this I offer two estimates: one that might be off by a factor of 100x, and those that are fairly accurate but take more time to make. My boss usually goes for the former.
Edit: also there's two types of estimates. One measures how much of my time it will take. The other measures the date it will be done and/or live. The two are oftentimes different numbers, and as a developer I can't always control the 2nd.
It's more like this: you've been on dozens and dozens of hiking trips. Except these trips generally go through seismically unstable territory and war zones. You just never know when you'll have to dig a tunnel underneath a battlefield to get to the next river you'd have to cross — a river unmarked on your map, or perhaps the result of some aggressive "engineering" by someone since your map was drawn.
Then you get an urgent message from your customer: you weren't supposed to hike from SF to LA after all. You actually need to go from New York to London, on a submarine. Specs require that you do this without surfacing or refueling. So you need to design a nuclear reactor, figure out a way to put it on the leaky old U-Boat your customer just happens to have lying around. All within the same timeline as before, and in the same budget.
You've had all this happen to you a few dozen times, so you factor in the time you know you need to build submarines (all just different enough, of course, that you can't reuse parts, even if the customers let you adapt, say, the propeller shafts to a new customer's design: I hear that in software, this is called "DRY" and "using libraries and frameworks" and "refactoring") — and you give an honest estimate: "it'll take eight months of full-time work, and will cost you about X hundred thousand dollars (a bargain for a nuclear submarine, if you ask me)" — and the customer will then go to someone else, leaving you wondering how you'll pay your bills.
Meh, you're just contriving extreme circumstances for your analogy so that the software developer is both a super hero and has no blame in being late with deliverables.
There are simple methods to combat gross inaccuracies in the client specification. You document the requirements for what you'll be delivering in a carefully worded Statement of Work that the client signs off on, you ask to be paid in chunks as you go with a portion up front, and you implement a system of change control so that it's totally clear if the customer is sending you off in a completely new direction. At that point, you book what you've worked at your hourly rate and you submit new work and time estimates for the changes.
Developer estimates are regularly off because "experienced development managers" are all too eager and willing to allow organizational politics and external influences to add complexity in ways the developers cannot anticipate. Inexperienced developers' estimates are even further off because of their relative lack of experience.
I assure you, the first time a "development manager" tries to scapegoat his subordinates for his own failures to deliver to these external influences his subordinates realize that their estimates impact them eventually, and directly.
>So the article tries to describe the hidden complexity of software with a hiking analogy but then it leaves a huge part missing. Let's extend the analogy somewhat... this isn't your first hiking trip. In fact, you've been on dozens and dozens of hiking trips. You're an experienced hiker. Yet, why do you continue to give meaningless as-the-crow-flies estimates of how long it will take you to get to your destination? The reason is that if you miss your estimate, it probably won't matter.
Nice try, but no.
Developers who are DIRECTLY affected from missing estimates (e.g one man shops paying out of their own pockets, people building an app while having quit work, eating ramen and maxing their credit cards etc, developers in companies who will personally pay for any delays by attrocious, unpaid overtime crunch marches, etc) underestimate the time needed to get something done just the same. Just ask Allan Odgaard (of TextMate 2 infamy).
100% agree on that. I did a lot of bad estimates and was directly affected by them (fixed rate projects that ended up driving my rate to "below mcdonalds worker" and some even with "lateness discounts" because I just wanted to keep the customer happy), and it took me a lot of time to get better at estimating even if I was paying from my own pocket for all the bad estimates.
Now if I am to hire a developer I will never think that "his estimates are bad because he's not directly affected by missing them"!
Okay, I agreed (had no other choice) with management to not have less than 20% variance in my estimates.(read: negotiated-and-reduced estimates) as a part of our performance metrics.
You can think of it this way. A coast line is a fractal line. As you zoom in its length gets bigger and bigger seemingly. From the top level on the map it looks fairly straight.
Related to estimates. Programmers have and operate with idealized mental models. A FIFO queue works kind of like this ("/closes eyes and sees a line of people lining up at the store"), a tree traversal looks like this ("/closes eyes sees a red cursor move down a tree drawn with blue nodes on an white board"). Same with estimates. How long will this take? ("/closes eyes and imagines an idealized sequence of events that will lead from now till project completion -- use this library, develop that API, test a little bit here, done... 2 weeks").
The problem is, zooming in is hard. "Oh this library we are about to use actually doesn't implement these corner cases". So now we are spending 3 days patching it. Oh, while working on this project, a critical support ticket comes from a customer. And then testing reveals a fundamental flaw in our API design. Re-write a huge chunk and repeat.
Some developers and managers just use heuristics. They do the mental estimate then x2 or x3.
^ All the above is true if the developer is everyone is honest and trusts each other. As other posts wrote, that is not always the case.
Then it is a more complicated game. The main question to ask then is "What is the punishment for overestimating vs the punishment for underestimating?".
Have people been fired for underestimating? No, ok underestimate. Have people been fired for missing deadlines? Ok then overestimate.
You can even play this game with yourself. What kind of deadlines do you set for yourself and how to you handle setting a too short or too long of a deadline? Do you get better with time at setting deadlines? If no, why not?
Haven't thought about it before but I realize I too use a heuristic of x2 or x3 when estimating my time. Nothing would stop me from getting a pen and paper out and stepping through the 'trail' in my head in pseudocode beforehand, but for most of my projects I'd benefit from just writing the real code from the start and apply the time I saved to the project.
I loved the article, but I was hoping we'd get a reference to the Coastline Paradox[1]. Basically, since a coast is a fractal, its distance is infinite. The "zoom" that caused the very first delay could keep going, and going, and going.
Most people would agree that practice makes perfect, right? I think a big issue with software estimation is that it is hard to get practice. I'd done full project estimates for 4-6 projects in 3 years of working professionally. Compared to the amount of practice I have writing code, this is nothing. How can I ever hope to get good at something if I only do it once every 3-6 months and it might take 2 years to get feedback on if my final number was close to the actual cost? And unlike other skills, I can't practice on my own or take a course (I've never seen an open source project that did round-trip estimates, maybe that's something to try out...)
Another issue is when you have an "indirect estimate" - basically I will estimate the work, but someone else is the one that ends up doing the project. If you aren't careful to consider who will be doing the work, you might estimate too low (if you are an expert and the work is done by a bunch of new hires).
And none of this even touches on misinterpreting client demands, scope creep, dev team turn-over, or often neglected timesinks like documentation and meetings.
This post was originally an answer on Quora to the question posed in the title [1]. The question has so far received 222 answers - some of them arguably better, though not as popular as this one.
I still don't want to support any site that uses this business model. You want me to use your site? Make it easy for me to come by for the information I need.
I enjoyed reading this the first time around. I purposely illustrates the complexity of software and perhaps inadvertently illustrates the challenge of getting schedules from inexperienced teams. In rebuttal, had they walked from San Francisco to LA along any freeway they would have been done much closer to their predicted date. Which they would have known had they been experienced in walking back and forth to LA from the Bay Area.
The "big" message is that if you have never done something before, you have no idea how long it will take, even if you can describe what you are going to do pretty easily. Returning to the hike metaphor, this is why you need to listen to the cranky old guy saying "walk along the roads, not the beach" instead of the inexperienced folks saying "Hey its just walking, and walking on the beach rocks! Walking along a road is boring and lame!"
The single biggest mistake in the example is giving an estimate for the whole project without knowing team velocity. He tried to plan everything up-front and committed to a completion date, strongly setting delivery expectations.
An Agile approach to this walk would have managed the expected target date much better and improved team morale by not overstretching each day ("sprint").
Lots of people think they know (and implement) Agile. I did, until a course I took about a year ago where I learnt skipping seemingly unimportant bits (measuring velocity, sprint retrospectives, etc.) bring the whole process crashing down.
The single biggest mistake in the example is giving an estimate for the whole project without knowing team velocity.
I'm inclined to agree. Often though a client will want an estimate (to a ridiculous accuracy) before agreeing to the work. Unless you churn out web page templates or something equally repetitive it's hard to have that relevant velocity.
I still come up against client managers that don't "believe" in Agile approaches too.
Oh we still get regular "lively discussions" about exactly what features are going to be in a product with a release date 6 months away. I can completely understand though; it's very difficult to leave a conversation being told "we don't know yet, it's likely to be X & Y (but we're not 100% sure) and possibly Z too".
I love this post; it conveys the feel of the experience so well.
One of my big aha moments about estimation was a bit in McConnell's Rapid Development. He pointed out that most estimates get made with executives pressuring for short numbers. When you iterate a few times with that, you end up with the smallest number that developers can't absolutely prove is impossible.
If you draw out a bell curve of probable completion dates, this is basically the same as picking one far to the left, so you have a 95-99% chance of being late. But somehow, executives are still surprised when their totally biased approach yields the numbers they liked, not the numbers they needed. All the estimation effort was pure waste; our time would have been just as well spent making cotton-candy raincoats.
I now have done a few projects with little to no estimation, and it has gone much better. I have a hard time now seeing why we ever bothered.
I wholly agree with this, and feel that all project plans should be constructed to mitigate the very high risk of initial estimates.
Too many project processes treat estimates as commitments with no backup plan when they (inevitably) go awry. I think a better approach would be to design your system around the idea that all estimates are suspect until proven otherwise, which of course leads to iterative approaches etc.
However, you can't run a business without any idea when anything will ship. Estimation is still, in my view, a very necessary step, so long as there is significant incentive to get it right rather than get it fast, AND the business is receptive to a little ambiguity through the project life cycle. This is a very difficult balance to strike.
In my experience you can have certainty in project scope or time, but not both.
I agree you can't run a business without any idea when anything will ship. But to get a good idea when things will ship, you don't need estimates.
For example, in the Lean world there's something I've heard called "Disneyland scheduling". You break the project down into lumps. You maintain a queue of the lumps. You measure the average time from position X in the queue to being released.
Now you have a reasonable early warning system for when things will ship. Which is generally much better than what people have with most projects, which is a fantasy date.
I once read an argument that went something like this: if you're actually going to go to the trouble to build something, its expected return to the business should be so large that how long it takes (within reason) shouldn't matter. Conversely, if a project is only economic to undertake if it can be done within a certain amount of time, it shouldn't be done at all.
I find that I often don't have a good feel for how long something is going to take until I'm about 1/4 the way through the project. If I absolutely had to produce accurate, high-confidence estimates, I would just do 1/4 of the work first, then estimate the rest.
As to why estimating is so hard, here's how I like to think about it. Tasks form a tree: each project has a number of steps, each of which itself breaks down into sub-steps, and so on. We tend to estimate by thinking about the tasks near the root of the tree; but the actual time required is proportional to the number of leaves -- tasks we can implement without further breakdown. Without planning the entire project out in detail it's hard to know what that number is. But to plan to that level of detail requires making many design decisions; and researching and making those decisions is much of the work of the project.
The bottom line is, good estimates are expensive to produce, and my experience aligns with yours: it's rarely worth the trouble.
Agreed. Certainly, estimates of cost shouldn't be more precise than estimates of value, because ROI is the number that matters. And business is a black enough art that estimates of value are very rarely more precise than a factor of 4x.
What I think people try to get from estimates is a sense of control. But I think there are better ways of giving people not just a sense of control, but actual control.
Forgive my username, there is nothing logical about this.
It's because estimates are really just a test of your imagination and skill. They're a "different game" from delivering something. Here are some examples:
How long does it take you to add 1 + 1 together? The only correct amount of time is "0 - no amount of time." But clearly that's not true, is it?
How long would it take you to multiply 5 by 7? Well, zero. No amount of time - you know it 'by heart'. It doesn't take you any amount of time to multiply 5 by seven. Well, clearly, that's not true is it?
How long would it take you to test whether 31337 is prime or not, by any means? Like, 1 second because Wolfram Alpha can tell me I bet. But clearly it wouldn't take me ONE second, would it?
How long would it take you to write a function to test whether an integer less than 2^32 is prime or not, without having to be super-efficient about it? Well like 10 seconds. But, clearly that's not true, is it?
And so on. How long does it take to put a twitter bootstrap page up? Like, 5 minutes.
How long does it take to...
You should be able to build an app that doesn't crash and does what you're trying to code in like a day.
You should be able to build something that is software you can launch with in like a week, as an MVP.
You should be able to replace someone else's mature product with a reimplementation in like a month.
You should be able to build a scalable business that is ready for a series A in like a year.
These things are not meant to be accurate, so much as show that you have imagination.
This McConnell book[1] is a must read on the subject. Generally developers just fail at estimating at a fine grained enough level and forget about all the ancillary tasks that take up a majority of a project's time.
I just had an estimate/actual conversation that went like this: I think it will take me X time. Ok, but I want it done in .5 X time. Ok, I'll try to get it done in .5 X time. I finish it in X time.
Is it a missed estimate? That depends on who answers.
Personally, I always give my estimates as a range. If you break down your project into small tasks, and estimate each task as an optimistic/pessimistic range, then you can get a pretty good idea of the expected completion date, along with the deviations from that which you might reasonably expect.
However good your estimate, committing to ONE number is madness - it hides the complexity of the situation, making it impossible for you or your manager to see the real picture.
172 comments
[ 3.6 ms ] story [ 204 ms ] threadSoftware is a lot like that.
If you don't want to hear the truth, you won't. Humans have known that since long before software engineering existed... or a subset of them, at least.
I do work in a place where we are a software company but very few people, even some of the devs, understand this culture. I don't say something is complex just for the hell of it. I want to give good estimates but at the end of the day that is all it is, a somewhat educated guess before I've gotten in the weeds. I don't need people that absolutely don't understand this concept to mandate a timeline that is unrealistic. It does absolutely no one any favors. That is why I leave. I don't need to promote unrealistic expectations down the line to customers who think everything is extremely simple. Especially when all we do is create custom software where almost 0% is turn key or off the shelf. Its just insulting to continue that nonsense and the quicker I can facilitate a reality check for all parties, the better and more trust is given to my judgment (and theirs if they actually take the time to learn)
This is actually achievable in a sane way.
I have done it using the XP planning practices. Basically, you make the suit break everything down into relatively small lumps and place them in priority order. Every week, the team completes a few lumps. Before you do them, engineers grade their relative complexity in arbitrary units. (The smallest substantial thing you do is 1 point; something twice as big is 2 points, and so on.) Every week, you count up the points completed. That's your "velocity".
From there, you let managers do all the estimating they want to. If they want the complexity of a unit of work measured, they ask the engineers. If they want to know when X will be done, they look at the team's recent velocity, what's in the queue before X, and do some basic math.
The nice part about this is the mental judo involved. Whenever they want to know when something will be done, it's their problem to trade features against time. Working like this, it's not geeks vs suits; the suits channel their schedule pressure into productive work: grooming the backlog.
You write "I'd rather them take my input and do their job of understanding trends." that's exactly what I'm talking about. They need to take your input, and do their job.
If a worker is told: "Here is what you (personnel) have to do (task), with these tools (environment), and we want you to do it in this amount of time (money)", and the worker fails to complete the work in the amount of time, then it is management's fault. They improperly estimated the amount of time it would take the worker to perform his task.
That management ask the worker how long the task will take means management does not understand the task, and if management does not understand the task, how can they know whether the worker's skill, experience, and knowledge is adequate for the task? Now, they also seem to not understand the worker's skills, experience, and knowledge, so the problem is compounded.
It is a management function to clearly define the tasks that must be performed, attract people whose skills, experience, and knowledge fit the tasks, and fund these people with the appropriate environment, tools, and salaries for them to perform the tasks.
That is the role of management. If they are unable to estimate what tasks must be performed, what people need to be retained, and how much money and time will be spent, they fail at their managerial duties.
That they then blame the workers themselves for improperly estimating the work is doubly wrong, a sign of managerial immaturity.
Unfortunately, it gets even worse if developers have no part in making estimates. If management doesn't know the details of your architecture and code-base (or even the other projects you may be working on, if you go high enough up the management chain), they have no way of coming up with a rational estimate on their own.
Where I work, VPs are constantly making "commitments" to customers to have certain features implemented by a certain date, and only notifying developers after the fact. Then they wonder why the development organization is "so bad at meeting deadlines". (Needless to say, I have no ethical requirement to honor commitments that weren't made by me and are frequently impossible to meet anyway.)
For most people it's around 2-5 but for some as high as 10 to 20.
You might feel like an idiot when you tell people "Hm, change the color of the logo to a different shade of green? Six hours." This is probably how long it will take, though, for that so-called "five minute fix".
As a developer, I have pressure to keep my estimate down. I can give you a guaranteed date with few problems, but the project is going to end up more time-consuming than what another developer is going to estimate.
The good news is that I will usually significantly beat my estimates under this system. However, I will then be accused of padding my estimates and will be given more aggressive targets, some of which I will miss.
The best way to combat this is to give me a team that has worked together on multiple projects within the domain, and account for changes and ambiguities as part of the process.
I occasionally just paste this link with no explaination at the end of emails containing time estimates.
Developer estimates are regularly off because they seldom impact the developer directly. Experienced development managers will pad the hell out of the developers who give them the worst estimates. Most developers will explain all the complexity of what threw their estimate off without acknowledging the huge mistake of not anticipating extra complexity in the first place.
My estimates in my early career were no better than anyone else's, ie way off -- especially for more complex projects. I'd explain what happened to my managers and soldier on. The very next task that came up, I'd give my manager some best-case estimate of how long something would take and the cycle would begin again.
That all changed once I started to do consulting for myself using "not-to-exceed" pricing. The first multi-month project I did killed me. My effective hourly rate went down to sub McDonalds levels and took much longer to deliver than I had expected. After that project, I did a post-mortem on the project to figure out where I went wrong. I came up with several spreadsheet templates and checklists to run through before giving any more estimates.
Mostly I just concerned myself with getting a hell of a lot better at estimating project duration and difficulty. Like most things, when you really pay attention to it and practice it, you get better at it.
I think I most disagree with this. Developers, in my experience, are quite often asked to leave the company after such estimates.
It's good to have a template - it helps not to forget things - but it is inherent in the work to have unknowns. Large unknowns. A good consultant would probably refuse to work on a project which would require him to use several major technologies which aren't familiar to him, but developer in a company may not have such a luxury as selecting a kind of a project.
I've seen developers let go because they weren't very good at writing software. I've seen developers let go because they were painfully anti-social to the point that they were negatively impacting the rest of the organization. I have never ever seen a developer let go because their estimates were crappy.
The developer's ability to estimate accurately is not an significant factor in this scenario.
At the VP level, you're expected to understand what the Developers and Development Managers don't know about the SDLC. You're expected to bridge the gap in their lack of understanding by instituting processes and controls while mentoring them to become better at what they do so that the organization can succeed.
Frankly, I would expect such a company that employs those practices sees two things:
1) A very low retention rate among employees, with a lot of churn.
2) A risk-averse organization, heavily skewed to under-promise and just-enough over delivery.
The trouble is now though, my competition seems to be underbidding me, but in reality they're providing those ~33% estimates they will never realistically keep, while I'm at ~100% estimates. Not really sure how to relay that to clients. One of the many reasons people like me need a salesperson in front, I guess.
Right, that was part of my method as well. I learned to keep really good notes of each part of the project. At the end of each project, I compared my "what I thought it would take" with "what it actually took". In subsequent projects, I tried to match up similar complexity items with "what it actually took" notes to remind myself of the pain.
I don't have a solution here though other than noticing that underbidding and then getting skilled at convincing clients to do paid extensions later actually appears to make the most money at the cost of your ethics. I'd avoid that approach, but it does seem to work for a lot of companies.
My personal approach is to quote for very bare projects with only the bare essentials (eg, poor UI design, minimum possible feature for the client to see what they're asking for, etc). This can usually be done a lot cheaper than most people think as 90% of the work is in the last 20% of the features. Then once the client has something, you can give them a quote to touch up the parts they need. Basically you split the project up into many small projects each with their own quote which helps you to estimate tasks as they appear and helps your client to minimize costs by leaving off features that are more expensive than they initially appear.
I would just as soon not keep customers only interested in the lowest price. They're the ones that typically will be the biggest headache.
It's a great position to be in, provided you have the throughput to expand to other clients and aren't dependent upon a single client for income.
http://www.missionfamilybank.org/software-estimating/
On the other hand, giving realistic estimates I often hear clients say, "I asked around, and other folks say this should take half what you're quoting." Usually this happens before the work, but I once had it happen in the middle of a long-term engagement after I delivered a two-week invoice, and they were disappointed I hadn't accomplished more. So if I'm not careful I can lose my clients' trust, which is my highest asset. I've gotten pretty good at explaining why a job isn't 5 minutes (or 1 day, or whatever), but I might save this article as something to share if a client doesn't have much experience doing software projects.
That a project is estimatable has to mean that novelty is low and predictability is high. But low novelty is a sign of duplication, which should be factored out into things like libraries, frameworks, and self-service tools. High predictability is either a sign of duplication or it's a sign that people have basically agreed not to learn anything during a project. If you are shipping early and often and study users and metrics as you go, then you will always learn things that affect the plan. Which means that the schedule isn't predictable even in theory.
Besides, most of the things you mention can be accounted for. If part of your development process involves iterating through the design with the customer every couple of weeks then you build that into your estimates.
Likewise, I always build documentation and code handover support into any serious project rather than acting surprised that customers will want such a thing and expect it to be part of what I'm delivering.
I think we have different intuitions here because it sounds like you're more in a service business than specifically writing software. I agree that a lot of any service business is standardizable, because it's mainly about people and their needs; that has a lot of regularity.
But I don't think the software creation part of a service business is standardizable over the long term. During the first wave of "put smallish businesses on the internet" each web site was custom, hand-rolled software. Early on those schedules were unpredictable, but for a while, it became a known, predictable business.
That business has, in the long term, been basically destroyed. People spotted the regularities and developed common code and tools. What was mainly a problem of software development became a (much smaller) problem of installation and configuration. The competitive advantage for those people now lies not in coding ability, but in customer service and in helping people manage the essential complexity of the domain.
Perhaps you weren't around in the early 2000s when the RAD tools were all the rage.
Perhaps you missed out on the colossal frameworks of the late 2000s when everything was a factory and understanding HTTP was actually a disadvantage as the whole thing would blow up if you actually tried to access the request body.
We went down the road you talk of. It was horrible. Now the pendulum swings to the opposite ends, the light-weight APIs which don't try and abstract away all the details which it turns out mattered a lot as everyone has to do pretty much the same thing, but ever-so-slightly differently. Tiny tools that do one job well.
All you are talking about is writing HTML and doing server config, neither of which are software or programming. When someone in 2000 came to you and said 'I need a website, therefore I need a software programmer to write html and setup a server' they no more needed a programmer back than then they needed one today. Just back then it was developers who knew the markup language and how to configure servers and they weren't about to turn away silly money just because there was not much actual coding involved.
I like crow-flies vs hiking analogy. I'd like to add upon it for so-called "green field" programming as it relates to getting cost predictions correct and understanding programmer time growth in relation to program size. Some parts are linear. Some parts are exponential.
Before I go on talking about "hello-world" space, note that I do architecture and really like architecture, big-picture solutions etc but I try only to solve what is apparent and then iterate on it whenever I notice two pieces tangling. There are excellent wins to be made here, but a working program itself constitutes part of the information necessary to arrive at the final architectural decisions. This is why even on FOSS code, I try not to sweat the architecture on the first pass -- when I do, dead code results. It calls out to me that I have solved something not relevant to the emergent implementation.
Once I've decided what the API might look like, what data structures have to be accessible to what, and what the minimum program states are, I try to stay in what I call "hello world" space. The idea is that the code I'm writing is never doing anything more than one problem at a time, line by line if I don't know exactly what a line does.
This is incredibly efficient because I deliberately break my problem into something testable and well understandable at every step. However, it's impossible to write production code this way. Not only is test feedback, (printlining and frequently more complex testing) not part of the final behavior, but more importantly, the elimination of concurrence of many states, the overlapping of problem workflows in execution sequence, implicitly says that "hello world" space problems are not production code. They have eliminated some functionality or consideration so as to make "hello world" solutions utterly oblivious to concurrent states or processes.
However, two "hello world" problems that need some coordination themselves creates a third "hello world" problem to implement their coordination. The third problem isn't apparent until the first two are explicitly solved. This implies hiking from the article.
Solving "hello world" problems brings more of the problem domain into "hello world" space. The concurrence problem itself will become apparent as a "hello world" problem when it has manifested itself. If there are two states that have been independently implemented but can overlap in execution, there is at least one set of logical statements for dealing with the maximum concurrence of the two states. Growth for N states is exactly an NxN truth table unless some of the states are sparse. In the worst-case, for N "hello world" solutions, one layer of abstraction where all states depend on all other states will result in N^2 logical blocks. While in practice this concurrence is usually much lower, the growth in abstraction is exponential if one is to keep breaking each problem down into an independently writable, testable piece.
So my conclusion is that growth is pretty much linear in proportion to writing "hello world" problems and exponentially proportional how many layers of integration are necessary. Slamming code down ichi-geki style is linear for problems that don't have a potential for overlap and geometric for ones that do. Code that eliminates duplication of routines through logic creates them in implementation, but the payoff is a net gain in many ways, so of course it still makes sense to do it.
As for making predictions, while running my internal monte-carlo, the concurrence of states has a higher influence on the outcome sometimes than the raw code size I expect to come out in the end. Linear ...
The big unknown - and the highest risk - was managing integration between external systems, or doing complex integration between internal systems. Anything that might benefit from aspect-oriented programming, like access control lists, or a more elaborate front-controller, or universal exports in json format, or a new paradigm like AJAX, would be difficult to estimate.
That fits your breakdown of projects into linear parts, and exponential parts.
Anyway, my estimates were good, but my business management was not - I was always in learning mode because I never studied business and had no experience. So I went W-2 and went into a different field because an opportunity to learn politics emerged. What's interesting is that successful political campaigns are entirely about estimating and budgeting, in real time more or less - you spend money and see results day by day, or after a couple weeks. It's driven by statistics (and messaging). I just watch from the sidelines, and don't do politics, but I think we could learn something from it. It's more like agile XP than waterfall.
The Obama campaign was XP, by necessity. The Healthcare.gov website thought it was waterfall... again by necessity.
I spent a couple of years traveling around to visit friends and family all over the US, and regularly my estimates were off in when I'd arrive somewhere. My software delivery estimates are similarly not perfect, in the later direction like most people. My conclusion was that my estimation of how long it takes to do anything is off a bit. For me, this is because I'm trying to say I'll do something sooner to meet the perceived needs and desires of others.
Plane, train, or for adventure, Greyhound. A sailboat makes more sense. A bicycle makes more sense. And if you insist on walking, at least follow a fucking road instead of cutting cross country.
If you decide the first step to driving to LA is mining iron ore to smelt for casting an engine block and building a Bessemer furnace for the steel for hand forged pistons, then there's lots of shit that can go wrong, EPA permits and zoning laws, even if you've done done it before.
The reason software production estimates are bad is because they aren't taken seriously. The guild of programmers can say, "we shall get from A to B by cross country hike."
Yes, the analogy is perfect because its absurdity is acceptable.
Because of this I offer two estimates: one that might be off by a factor of 100x, and those that are fairly accurate but take more time to make. My boss usually goes for the former.
Edit: also there's two types of estimates. One measures how much of my time it will take. The other measures the date it will be done and/or live. The two are oftentimes different numbers, and as a developer I can't always control the 2nd.
Then you get an urgent message from your customer: you weren't supposed to hike from SF to LA after all. You actually need to go from New York to London, on a submarine. Specs require that you do this without surfacing or refueling. So you need to design a nuclear reactor, figure out a way to put it on the leaky old U-Boat your customer just happens to have lying around. All within the same timeline as before, and in the same budget.
You've had all this happen to you a few dozen times, so you factor in the time you know you need to build submarines (all just different enough, of course, that you can't reuse parts, even if the customers let you adapt, say, the propeller shafts to a new customer's design: I hear that in software, this is called "DRY" and "using libraries and frameworks" and "refactoring") — and you give an honest estimate: "it'll take eight months of full-time work, and will cost you about X hundred thousand dollars (a bargain for a nuclear submarine, if you ask me)" — and the customer will then go to someone else, leaving you wondering how you'll pay your bills.
There are simple methods to combat gross inaccuracies in the client specification. You document the requirements for what you'll be delivering in a carefully worded Statement of Work that the client signs off on, you ask to be paid in chunks as you go with a portion up front, and you implement a system of change control so that it's totally clear if the customer is sending you off in a completely new direction. At that point, you book what you've worked at your hourly rate and you submit new work and time estimates for the changes.
Developer estimates are regularly off because "experienced development managers" are all too eager and willing to allow organizational politics and external influences to add complexity in ways the developers cannot anticipate. Inexperienced developers' estimates are even further off because of their relative lack of experience.
I assure you, the first time a "development manager" tries to scapegoat his subordinates for his own failures to deliver to these external influences his subordinates realize that their estimates impact them eventually, and directly.
Nice try, but no.
Developers who are DIRECTLY affected from missing estimates (e.g one man shops paying out of their own pockets, people building an app while having quit work, eating ramen and maxing their credit cards etc, developers in companies who will personally pay for any delays by attrocious, unpaid overtime crunch marches, etc) underestimate the time needed to get something done just the same. Just ask Allan Odgaard (of TextMate 2 infamy).
Now if I am to hire a developer I will never think that "his estimates are bad because he's not directly affected by missing them"!
You're like the expert sherpa trying to convince people to use your message delivery service when the internet exists.
You can think of it this way. A coast line is a fractal line. As you zoom in its length gets bigger and bigger seemingly. From the top level on the map it looks fairly straight.
Related to estimates. Programmers have and operate with idealized mental models. A FIFO queue works kind of like this ("/closes eyes and sees a line of people lining up at the store"), a tree traversal looks like this ("/closes eyes sees a red cursor move down a tree drawn with blue nodes on an white board"). Same with estimates. How long will this take? ("/closes eyes and imagines an idealized sequence of events that will lead from now till project completion -- use this library, develop that API, test a little bit here, done... 2 weeks").
The problem is, zooming in is hard. "Oh this library we are about to use actually doesn't implement these corner cases". So now we are spending 3 days patching it. Oh, while working on this project, a critical support ticket comes from a customer. And then testing reveals a fundamental flaw in our API design. Re-write a huge chunk and repeat.
Some developers and managers just use heuristics. They do the mental estimate then x2 or x3.
^ All the above is true if the developer is everyone is honest and trusts each other. As other posts wrote, that is not always the case.
Then it is a more complicated game. The main question to ask then is "What is the punishment for overestimating vs the punishment for underestimating?".
Have people been fired for underestimating? No, ok underestimate. Have people been fired for missing deadlines? Ok then overestimate.
You can even play this game with yourself. What kind of deadlines do you set for yourself and how to you handle setting a too short or too long of a deadline? Do you get better with time at setting deadlines? If no, why not?
Haven't thought about it before but I realize I too use a heuristic of x2 or x3 when estimating my time. Nothing would stop me from getting a pen and paper out and stepping through the 'trail' in my head in pseudocode beforehand, but for most of my projects I'd benefit from just writing the real code from the start and apply the time I saved to the project.
Anyway, nice article!
[1] http://en.wikipedia.org/wiki/Coastline_paradox
Another issue is when you have an "indirect estimate" - basically I will estimate the work, but someone else is the one that ends up doing the project. If you aren't careful to consider who will be doing the work, you might estimate too low (if you are an expert and the work is done by a bunch of new hires).
And none of this even touches on misinterpreting client demands, scope creep, dev team turn-over, or often neglected timesinks like documentation and meetings.
[1] http://www.quora.com/Engineering-Management/Why-are-software...
Tell HN: Add ?share=1 to Quora URLs to display content without login
https://news.ycombinator.com/item?id=6300856
The "big" message is that if you have never done something before, you have no idea how long it will take, even if you can describe what you are going to do pretty easily. Returning to the hike metaphor, this is why you need to listen to the cranky old guy saying "walk along the roads, not the beach" instead of the inexperienced folks saying "Hey its just walking, and walking on the beach rocks! Walking along a road is boring and lame!"
An Agile approach to this walk would have managed the expected target date much better and improved team morale by not overstretching each day ("sprint").
Lots of people think they know (and implement) Agile. I did, until a course I took about a year ago where I learnt skipping seemingly unimportant bits (measuring velocity, sprint retrospectives, etc.) bring the whole process crashing down.
I'm inclined to agree. Often though a client will want an estimate (to a ridiculous accuracy) before agreeing to the work. Unless you churn out web page templates or something equally repetitive it's hard to have that relevant velocity.
I still come up against client managers that don't "believe" in Agile approaches too.
One of my big aha moments about estimation was a bit in McConnell's Rapid Development. He pointed out that most estimates get made with executives pressuring for short numbers. When you iterate a few times with that, you end up with the smallest number that developers can't absolutely prove is impossible.
If you draw out a bell curve of probable completion dates, this is basically the same as picking one far to the left, so you have a 95-99% chance of being late. But somehow, executives are still surprised when their totally biased approach yields the numbers they liked, not the numbers they needed. All the estimation effort was pure waste; our time would have been just as well spent making cotton-candy raincoats.
I now have done a few projects with little to no estimation, and it has gone much better. I have a hard time now seeing why we ever bothered.
Too many project processes treat estimates as commitments with no backup plan when they (inevitably) go awry. I think a better approach would be to design your system around the idea that all estimates are suspect until proven otherwise, which of course leads to iterative approaches etc.
However, you can't run a business without any idea when anything will ship. Estimation is still, in my view, a very necessary step, so long as there is significant incentive to get it right rather than get it fast, AND the business is receptive to a little ambiguity through the project life cycle. This is a very difficult balance to strike.
In my experience you can have certainty in project scope or time, but not both.
I agree you can't run a business without any idea when anything will ship. But to get a good idea when things will ship, you don't need estimates.
For example, in the Lean world there's something I've heard called "Disneyland scheduling". You break the project down into lumps. You maintain a queue of the lumps. You measure the average time from position X in the queue to being released.
Now you have a reasonable early warning system for when things will ship. Which is generally much better than what people have with most projects, which is a fantasy date.
I find that I often don't have a good feel for how long something is going to take until I'm about 1/4 the way through the project. If I absolutely had to produce accurate, high-confidence estimates, I would just do 1/4 of the work first, then estimate the rest.
As to why estimating is so hard, here's how I like to think about it. Tasks form a tree: each project has a number of steps, each of which itself breaks down into sub-steps, and so on. We tend to estimate by thinking about the tasks near the root of the tree; but the actual time required is proportional to the number of leaves -- tasks we can implement without further breakdown. Without planning the entire project out in detail it's hard to know what that number is. But to plan to that level of detail requires making many design decisions; and researching and making those decisions is much of the work of the project.
The bottom line is, good estimates are expensive to produce, and my experience aligns with yours: it's rarely worth the trouble.
What I think people try to get from estimates is a sense of control. But I think there are better ways of giving people not just a sense of control, but actual control.
estimate
It's because estimates are really just a test of your imagination and skill. They're a "different game" from delivering something. Here are some examples:
How long does it take you to add 1 + 1 together? The only correct amount of time is "0 - no amount of time." But clearly that's not true, is it?
How long would it take you to multiply 5 by 7? Well, zero. No amount of time - you know it 'by heart'. It doesn't take you any amount of time to multiply 5 by seven. Well, clearly, that's not true is it?
How long would it take you to test whether 31337 is prime or not, by any means? Like, 1 second because Wolfram Alpha can tell me I bet. But clearly it wouldn't take me ONE second, would it?
How long would it take you to write a function to test whether an integer less than 2^32 is prime or not, without having to be super-efficient about it? Well like 10 seconds. But, clearly that's not true, is it?
And so on. How long does it take to put a twitter bootstrap page up? Like, 5 minutes.
How long does it take to...
You should be able to build an app that doesn't crash and does what you're trying to code in like a day.
You should be able to build something that is software you can launch with in like a week, as an MVP.
You should be able to replace someone else's mature product with a reimplementation in like a month.
You should be able to build a scalable business that is ready for a series A in like a year.
These things are not meant to be accurate, so much as show that you have imagination.
[1] http://www.amazon.com/gp/aw/d/0735605351
Is it a missed estimate? That depends on who answers.
Never, ever say this.
Personally, I always give my estimates as a range. If you break down your project into small tasks, and estimate each task as an optimistic/pessimistic range, then you can get a pretty good idea of the expected completion date, along with the deviations from that which you might reasonably expect.
However good your estimate, committing to ONE number is madness - it hides the complexity of the situation, making it impossible for you or your manager to see the real picture.