Evolution. 100,000 years ago, there were humans who could accurately estimate the difficulty of, say, catching a deer. And since they knew it was too much trouble, they just stayed home in the cave.
Meanwhile some crazy optimist went out and to catch a deer. It was many times more difficult than expected, but they returned to the cave with the deer and eventually reproduced more successfully than the accurate estimator.
Or the estimators only hunted deer when they were well positioned to do so (e.g. few predators of humans and many deer around), and the optimists were eaten by bears.
That does not make sense. If eventually it turned out that it was worth it, then their estimate - i.e. 'it is worth it' was accurate - not too optimistic.
Another way to look at this is that the crazy optimists who are the worst estimators never return (themselves eaten by predators, run out of water, hypothermia) or return empty handed, lowering their reproductive success. The optimists who succeed quickly adjust their understanding of the estimating process ("Zog died when he went after deer at night during winter, and I barely survived being attacked by wolves when the deer were most plentiful, so I should go on cool midday hunts when the deer are already slightly down in population").
I find naive and pretentious the suggestion that the average programmer's work is in any way akin to Einstein's search for the truth about our universe. My criticism might seem nitpicky until it's time to measure this article's message against the possible reaction of an engineering or business manager. Launching into a systematic dismantling of the pervasive estimates culture is best done with humble, verifiable assertions and helpful suggestions in tow. Attempting to recruit the long-dead Einstein is unhelpful and hand-wavy.
There must be a better way to package the no-estimates message, and it's not this.
I find naive and pretentious the suggestion that the average programmer's work is in any way akin to Einstein's search for the truth about our universe.
Indeed, most projects are remarkably (and sadly) similar to other projects we've done before.
It doesn't much matter how "similar" they are. Unless you can copy and paste the code you're building new stuff and running the risk of encountering all the issues inherent in building new stuff.
How about: the reliability of software estimates are inversely proportional to creativity requirements, with a unified theory of physics establishing an upper bound on creativity with a corresponding 0 for the estimate reliability.
The thing is that for many companies no estimates isn't possible. You have to be able to tell a client roughly how much they are going to get charged or how long it will take on order for them to even be able to with the options.
Saying no estimates is akin to saying "this is hard, we suck at it so we quit." The problem isn't solved, you still have no way of measuring how complete you are and when you'll be able to turn new things on.
We use a combination of story points and historical reference where we can during our estimating phase. So if we are talking about adding a new feature and wet estimate it out we go look at the actuals from a similar change that we did before to see how is base we might be. We also like story points because they are a ballpark and help us judge if we are estimating reasonably well. If we are continually having things signed low points that take data to finish, we can address that. Fast feedback and lots of iterations. The thing teams fail to do is to measure properly. If you measure too much no one fills out their stuff accurately and then you've just got garbage. Measure the wrong things and you're blind. Measure nothing (no estimates) and you're just giving yo.
We don't try to measure to the minute or hour. Things are basically 'quick' 1hr or less, 2hrs, 1/2 day, 1 day, or a 1/2 week for a task we know is big and hairy that has a lot of sub tasks that we don't really want to dive into the minutiae of it because what's relevant is that X does A and B, not that X requires T, U, and V to be done first.
That's the hard part for us, enough detail that its meaningful without spending all your time planning to plan.
If I had $1 for every time I saw one teammate get roped into spending a week building some goofy little low value feature because another teammate - never a guy with any skin in that particular game game - said, "I could have that done in two hours" during a meeting. . .
You have bad managers if they let that crap fly. The appropriate response to that kind of thing is "then you should do it, because you clearly have the domain expertise to do the work efficiently".
During negotiations with a large corporation for customizations to our SaaS product, a business analyst asked:
"Why is this so expensive? Can't you just copy and paste some of the code from other parts of the system?"
I couldn't even respond right away, I had to censor myself before saying anything.
Background:
They wanted pre-populated values/logic very specific to their workflow. It wasn't just an easy cut and paste because the software would behave differently for them.
1. People need to look into PST from Simon Wardley.
2. Some of my clients do excellent work with estimations delivering on time and on budget and satisfied customer (In the Wardley Six Sigma zone - low uncertainty)
3. Estimation does not work in the Genesis or Product zones (high uncertainty)
We need to grow up beyond the "software estimation not possible".
It depends on the type of product. If the product space is well known, then a mature organization can predict how long it should take.
If the product is in a new or unknown area, or the product uses an existing unreliable/unknown code base, reliability of estimates are much lower.
In general, software is re-usable, and when done right, it is unnecessary to need software engineering for projects that are very similar to previous ones.
The part you're working on (and trying to estimate) in almost all cases should be "new or unknown area".
Both parts of a project need estimates. If you are not estimating for the known parts how can you project how long the project will take? Its the unknown parts that can have a lot more variability to them.
It says "Six Sigma/Outsource" for a reason. The product is "ordered, known, measured, standard, obvious, low margin". If at all possible, this should be an off-the-shelf choice. Previously, that would have meant packaged software; now it mostly means getting something as a library or service.
Six Sigma is fine for manufacturing low-margin items, because you are trying to get high repeatability in the world of atoms. If you have one widget and need to make one million more, you use Six Sigma. But with bits, we already have near-perfect repeatability. If I have one integer and need a million more integers, the computer will do that at a reliability level well beyond six sigmas.
If software is being made in the Six Sigma/Outsource zone, it mostly means that one is reinventing the wheel. Sometimes that's necessary for historical reasons. E.g., if I'm working in a new language but need a library that doesn't exist yet, sometimes I really can just take the old spec and implement it anew. Or if I want developers to use my cloud service, maybe I need to just reimplement chunks of the AWS API, because Amazon won't give me that code.
But the great majority of software development should take place in the higher-novelty zones, where reliable project-scale estimation is somewhere between impossible and nonsensical.
(As an aside, casting people you disagree with as children is insulting.)
If they're duplicating, I agree high certainty can be had.
But once modifications happen, I think you're headed back to the other end of the Wardley spectrum. Do the modifications meet user goals? Do they meet business goals?
In those circumstances, the only way you can return to high certainty is by creating very firm specs and resisting change orders. From the perspective of a contract development shop, I'm sure that's fine. But for the people actually paying for the software, that's a process smell.
The commercial and practical validity of modifications can only be fully evaluated by shipping. But large specs require long release cycles. This creates a long period where learning is deferred. Estimates only become manageable because the project is saving up trouble for later. A period of artificially high predictability will be followed by a period of artificially low predictability.
And even in the pure duplication case, what comes after that? I can't think of many successful pieces of software that stopped at 1.0. Once the duplicate is released, most businesses will need to respond to their customers, doing novel and innovative things specifically for them. So at best, you're back in the fast-cycle end of the spectrum. But at worst, you have executives who are attached to specs and estimates and long timelines, which are the wrong tools for innovation. You have development processes that match. And you probably have a code base that's not set up for flexibility.
Given that, even if I were, say, making that AWS competitor, I'd still use all my tools from the fast-cycle innovation end of the spectrum. Because even a pure duplicate assumes that a) there is nothing left to learn, and that b) the context of sale/use is not changing. Estimation is always a bet that you won't learn anything important between the start and the end. Given how smart my colleagues are, that's a bet I'm rarely willing to make.
The same point as getting a tailor made suit instead of one off the rack: both are non-innovative and have predictable processes for manufacturing, but one is made specifically for your needs.
How long would it take to walk from San Francisco to Los Angeles. Even that almost impossible to estimate and not be off be several multiples, if you're lucky!
The guy in your link did lousy routefinding, and estimated that he could walk 40 miles a day. Anyone with experience backpacking can tell you that's a ridiculous pace (for a group, one mile an hour is a good pace). Plenty of people plan out long hikes with accurate estimations.
But remember you're doing something novel so you've never hiked before, and so you can only estimate how much you can do each day. You may have done other travels but never walked that distance. Even if you half the speed you'll still be way off. And this is just 400 miles, what about a walk that across the whole US? Can you predict that Susie on your team will fall ill for a few week. How many meetings will management need? What about feature changes? How much time do do you have to estimate? So you have months to plan out your walk? Most projects need estimates within minutes to hours. And so on...
You've built a custom system with the exact same business requirements that connect to the exact same different sub systems, api, etc? It may be the same programming language and general framework but that's only a small part of most projects ;)
Can you predict that Susie on your team will fall ill for a few week.
I can tell you, if you have a relatively long project, it's worse than that: Susie (or someone else) will probably quit. Better take that into your estimate calculation.
Remember, estimates are an outer bound; if you finish early, that's ok.
Then you go off and walk for a day and time how long it takes you. Now you can go back and have some clue about how long it will take. Bonus points for expecting to get faster over time.
Another thing to note here that I have experienced is when management has an incompetent mentality that, for a given project, that all tasks for this project are created equal. For example, they expect that if task X took 1 hour that surely task Y will also take 1 hour despite the fact that task X and task Y vary greatly in complexity.
Explaining the difference in complexity helps, but the problem is that often the programmer doesn't really know in detail besides a feeling that comes mostly from many years of experience. And the best thing is if they then get 'their 10x code slinger' who does it indeed very fast, sends the code back to you and you are stuck with something akin to a broken toilet with diarrhea coming out the sides.
True. But it does no good if management is dismissive. But that's another problem outside of the topic of estimating.
> And the best thing is if they then get 'their 10x code slinger' who does it indeed very fast, sends the code back to you and you are stuck with something akin to a broken toilet with diarrhea coming out the sides
A grim picture to think about but I find this to be true a lot of the time.
I always have trouble explaining that it took a week to implement 90 percent of the whole project but then it took another 4 weeks to figure out why some little feature failed sometimes. That's my problem with scrum and velocity numbers. It gives clueless people the illusion that they can estimate everything easily.
Even if you get better at estimating tasks, it's extremely hard to account for bugs that aren't discovered until later.
I usually have a pretty accurate estimate for small tasks but never go back and add the time spent on fixing recessions caused by that task. Or the time spent on extra smoke testing due to missing automation.
That why you use procedures that find bugs early. This has been solve (in various ways) since the 70s yet people are still writing about how it's "not possible".
We do. Unit tests, integration tests, manual smoke tests, code reviews of every commit, design reviews, self hosting...
Sometimes your test matrix is too big. But I agree: sometimes you have to move slower in order to go faster overall and have better estimates. Estimates have to take into account testing and regression testing, which most people don't so it can make you look slow if you start doing that.
it's extremely hard to account for bugs that aren't discovered until later.
Pad your estimates, because you are typically going to have similar bugs (for similar projects), and you can account for those unknowns probablistically. You don't know the exact bugs that will be found, but you've already predicted that some will be found, so include that in your estimate.
> The best you can do is estimate based on historical metrics which ... is not worth very much.
I've had significant success in using historical metrics to accurately estimate software projects.
It takes diligence and professional experience to learn to do this, but it is possible in my experience.
Unfortunately I find this philistine sentiment quite frequently in the software world: "I've had difficulty doing X. I am smart. I am senior. I am a professional. Therefore X must be impossible." Isn't it possible you haven't been stubborn/hard-working enough in your own personal growth?
Could you share more details on how you used historic estimates?
You basically described me. I've never found historic estimates useful in practice, of course I haven't tried to hard, but over the years can't think even single case I'd find useful to use some old estimate.
That's quite simple, really. If you've carefully tracked your past projects, by comparing tasks from the planned project to those you've done of similar complexity you can make more realistic estimations.
It works best on a long-running project. When you plan a new feature, you can gauge really accurately at what speed your current team developed a feature of comparable complexity, so as you go forward you get better and better estimates.
I know, for example, that under a certain annoying development process coughPSPcough, I will produce code at a rate of 30 LOC/hour when measured from initial requirements gathering through handing code off to testers.
I know this because every line of code I've written (using that company's process) for years has been measured and timed. So now the estimation process becomes a matter of figuring out how many LOC (as a proxy for effort) a given feature/project will be and then dividing that number by 30.
It is scarily accurate, but like I said, it's been aggregated over thousands of lines of code for years.
Think of estimation as an outer bound (with a confidence level). You are saying "I am 98% sure the project will be completed by time X." The more unknowns you have, the farther out you should push the deadline.
Give yourself extra time: no one will complain if you finish early.
Meh, i wouldn't say impossible. I worked with amazing people. We were able to hit our goals consistently. Key is to have a great team who knows what they are doing and are honest. Measure by complexity, not time.
Unless you worked on a few products, your going to be bad at estimating larger pieces. A lot of software is very similar.
If I am running a business that is looking at what projects to fund for the next year, and one of those projects is a software project with an unknowable time to completion vs (for example) opening a new store, with a well known time to complete, why would I ever pick the software project? It has a completely unknown ROI! If you cannot even give me probabilities, the lottery is a more guaranteed investment!
Software estimation is required for real business to ever consider software projects. The key is to embrace uncertainty, and instead of estimating that task X will take exactly Y days, estimate a range of possibilities. I like the three point system, estimating a best case, worst case, and most likely case. It explicitly asks you to think about the long tail for when things go wrong. Then use statistical methods to combine estimates and create estimates for multiple confidence levels.
This is all covered in a great book, "Software Estimation: Demystifying the Black Art" (https://read.amazon.com/kp/embed?asin=B00JDMPOVQ&asin=B00JDM...). It's not that long a read, and it goes over the problems covered by this blog post but doesn't throw it's hands up in the air and say estimation is impossible.
>If I am running a business that is looking at what projects to fund for the next year, and one of those projects is a software project with an unknowable time to completion vs (for example) opening a new store, with a well known time to complete, why would I ever pick the software project? It has a completely unknown ROI!
Because the benefits of the software project would far outweigh the benefits of a new store? With the completion of the software project you could open an order of magnitude more stores? Completion of the software project will increase the profitability of existing stores such that they make more than a new store at existing (hypothetical, since the store is not open yet) profitability?
And that would be great, as long as doing the software project doesn't take several orders of magnitude more time. You have to be able to estimate the project with some confidence interval to know if it is worth it.
If opening a store takes 6-9 months, and we have a software project that can make all of my 100 stores ~10% more efficient that needs 2-4 years, then the software project looks a lot more feasible! (ignoring that calendar time is not exactly cost, and that we may need shorter term wins to stay in business)
The problem is if, when asked for an estimate, we say "I don't know and I never will know until I'm done". Now I don't know if stores will be more efficient 2 weeks or 20 years from now.
>> If I am running a business that is looking at what projects to fund for the next year, and one of those projects is a software project with an unknowable time to completion vs (for example) opening a new store, with a well known time to complete, why would I ever pick the software project?
Maybe because you're trying to innovate? Will opening a new store dramatically change your company's market outlook? Will it open up entirely new markets for products you don't yet have? Will it solve new problems for people?
Milton Hershey offered his first chocolate bar in 1900. It took him a further five years to perfect the manufacturing processes that produced the Hershey's Kiss. None of those processes, and none of the machinery to execute them existed when he started. He had to invent them.
Today, 110 or so years later, the company produces around 60 million of them a day which it ships all over the world. Probably he should have just opened another store.
No offense, but people who think the way you do about risk are exactly the reason large companies lose their ability to innovate at all.
> No offense, but people who think the way you do about risk are exactly the reason large companies lose their ability to innovate at all.
Exactly. This is why the startup ecosystem continues to thrive even after the age of the big IPO is over. Startups are akin to R&D for larger companies who don't want to (or can't) take the risk to try something new.
I'm not really making the argument your reacting to. Software projects can be estimated, so it is possible to understand what their ROI is. I'm saying that, because software can be estimated using rigorous technique instead of hunches, that we can continue to justify developing software.
Even if what you are doing is truly research that is completely open ended, there still have to be controls placed on it. Budget/grants/runway will run out at some point.
I'm not a believer in what you're saying, but I dearly want to believe, in no small part because there are significant organizational rewards to be reaped for believing in estimates and acting the part.
What are these rigorous techniques you speak of? I seriously, honestly want to know.
The book I linked up above is one of the best I've read on software estimation. It's short, it covers exactly the problems we run into everyday (managers take estimates to mean commitments, optimism on most estimates, constant pressure to reduce estimates because they cannot be defended, etc). And it covers several different estimation methods that you can choose from based on circumstances, the pros and cons of each, etc.
It's "Software Estimation: Demystifying the Black Art" by Steve McConnell (https://read.amazon.com/kp/embed?asin=B00JDMPOVQ&asin=B00JDM...). Same author as "Code Complete" and "Rapid Development", which are also great books but much longer and showing their age a little more.
Second, in estimation, previous performance is often indicative of future performance. If something took you X days before, it will likely take you roughly X days next year. It's not likely to take 10X the time. The real problem is that very few teams actually track their time on task accurately so no one on the team knows what X was. Without that data, trying to get accurate estimates is a waste of time.
A very large part of the problem is that most developers are educated in Computer Science or other areas and simply haven't been trained in estimation techniques. Software Engineering is about the process side of development, and one of the benefits of process is predictability.
Managers, for the most part, keep saying that. Usually its because they have, or they've read about someone who has estimated accurately one or more times. How do I know they were not simply lucky? We cannot accurately estimate the time it will take to construct a building, and we've been constructing buildings for thousands of years.
There are other kind of businesses where estimation is hard.
Producing movies, for example. Even if you cast established actors and limit yourself to low-risk sequels of successful franchises, chances of box office bomb are substantial. Book publishing would be even better example; thankfully, writing a book is dirt cheap comparing to developing software or making movies. Pharmaceutic research is absolutely unpredictable — but big pharma companies are usually quite profitable, like software companies (their research/production patterns are similar).
New software development is _always_ innovation (because, if your problem is already solved by existing software, it is way cheaper to buy this software instead — as cost of production is close to zero compared to the cost of development). Innovation is _always_ unpredictable. Building the business on innovation is therefore a hard problem.
It's fairly easy to estimate the time needed for a novel or a textbook. The only possible reasons for a holdup in (say) a tech book is if the product is in beta and big changes happen after writing but before publication.
Producing a movie isn't rocket science either. But if anything it's more complex than most software projects, because you have to arrange the finance, hire hundreds of people spread across tens of teams, move people and stuff around, design and build things, knock things down, add a lot of stuff in post - which may need custom software - and so on.
And that's just the footage. You also have to market and distribute the movie worldwide, work out deals for residuals on various media, and so on.
And everyone has to be paid on time, near as, or work stops.
Estimating profit is a completely different job, unrelated to keeping the machine running from start to finish. It's barely a footnote. Studios are mostly not bad at it, but they do sometimes get it very wrong.
Designing a CRUD website that scales is nowhere close in terms of complexity.
But here's an interesting thing: movies only get finished because there's a hierarchy of talent and delegation. The director/producer can hire an expert or a ready-made team to handle part of the project, and delegate that entire element to them. From then on the producer/director needs to do some oversight, but the details (should) "just work."
Startups don't work like this. Startups hire everyone on a permanent contract basis, often as individuals, try to build them into teams internally, often on the basis that anyone sub-senior is pretty much interchangeable with everyone else at the same level, and generally delegation doesn't happen in the same way.
There's really no reason why teams of programmers couldn't get together, set up service companies, and hire themselves out as consultant solution providers. After a few successes a team should have it made.
This doesn't happen. I think that cultural difference is very interesting.
> ...but doesn't throw it's hands up in the air and say estimation is impossible.
It is impossible to know for sure. It's a bet. It's also impossible to say precisely what the DOW will be at next Thursday. But you can make a guess with confidence intervals, long or short the appropriate index, and make some money.
The problem is that "real business" people want impossibly precise estimates for software projects in particular. If software estimates were more like medical prognoses or financial reports, they would include confidence intervals and risk assessments, and we would be having another conversation.
That's precisely why so few companies who can produce good software, are also proficient in some other area. Successful software development mostly happens when it's the main focus of the organisation.
Apple used to be the prominent exception, but their software's quality seems to have decreased in recent times.
It's not as simple as you make it to be. How long do you think it would take to walk from San Francisco to Los Angeles? I guarantee you that your max will not be high enough. Try it and then read the following article and let's see how close you came http://www.michaelrwolfe.com/2013/10/19/50/
Just because you can't always have an accurate estimate doesn't mean it's now worth it.
Also remember that estimates are based on resources. We were able to go to the moon in a decade but a lot of work was done before and the budget to get there was much higher than what most software development efforts ever get relatively speaking. It might not be going to the moon but that doesn't mean some projects aren't very complex either.
Actually another great example is a new pharmaceutical drug. Homw can you accurately predict that effort. If something is complex and or newer sometimes it's very hard if not impossible to predict.
That article is more like asking someone who has never programmed before how long it will take to do a project. If you are estimating something that you have truly never done and have no experience in, then it's going to be more difficult. But if you at least acknowledge that, you can take steps against it like lowering your own confidence level in your estimates (this feature will take 4-8 weeks, and I'm only 25% confident the actual timeline is between those). The issue is more when you are over confident, which it sounds like no one here is!
I'm not an avid hiker, but I've done enough weekend hikes to know that 10 miles per day can be pushing it for me on some terrain. I wouldn't ask someone who has never hiked ever before to estimate how long a hike would take. I wouldn't ask a programmer to estimate how long it takes to construct a bridge.
Have you ever tried to sell something to management with a 25% confidence level with an estimate that is at least 2x (4-8 weeks). You've just basically said your estimate is anywhere from a week to half a year, which basically means you have no estimate ;)
In terms of never having done it before, this is true of most software projects. If you're building the same house from the same plans then this is the same as say installing a Wordpress blog, easy to estimate. But if you were to ask how long it would take to build Wordpress from scratch, even having a Wordpress sample app, it would be very hard. Not only that but there are a lot of assumptions. Is it for a handful of visitors or are we talking a million visitor blog?
Ignoring that, let's go back to the pharma example. How long do you estimate developing a new drug will take? It's not the first drug your company has developed. Anything beyond a week is hard to estimate.
Back to your estimate of 25% within 4-8 weeks, this would never be accepted because it could mean anything.
The worse part, I an tell you right now, even if new functionality or changes are added, you will be elf to 4-8 weeks, the 25% will be ignored. And in most cases management will hold you to 4 weeks because this is what was sold to their bosses.
Actually, it means a lot. It tells us how much we understand the project's timeline. Many projects get estimated way to early in the process, and the true level of uncertainty at that point is massive. If the low confidence level means the estimate has an upper end that isn't feasible, there are things we can do to increase confidence. Proof of Concepts/Spikes are explicitly for the purpose of taken an unknown feature and getting a better understanding of their complexity, scope, and timeline. (http://www.construx.com/Thought_Leadership/Books/The_Cone_of...)
So then the hard part is communicating to the business the current estimates and confidence level and that we can do some up front work to tighten down our estimate and the schedule. This is actually part of ACM's "Software Engineer Code of Ethics": "3.09. Ensure realistic quantitative estimates of cost, scheduling, personnel, quality and outcomes on any project on which they work or propose to work and provide an uncertainty assessment of these estimates."
As for pharmaceuticals, I have no experience there. I doubt that they really just let scientists go off and do whatever they want with unlimited budgets, and just live with the results.
So let me ask you a simple question, how much time do I have to give the initial estimate and how long do I get to increase confidence? Is it in the order or days, weeks, or months? I ask because a 25% confidence level on 4-8 weeks would most likely require some time to increase the confidence, more than you would be permitted.
I agree with what you're saying and that's all great and awesome but in the real world it unfortunately doesn't work that way. In most cases you have a day to a week to come up with an estimate for a year long project. If you by great luck get a cognate to do a spike, you get maybe an extra day. Ok most cases actually you would've maybe gotten an hour to a day at best and there would be no spike, and if you did get a spike it would be maybe an hour. Again I agree with what you're saying but it just doesn't work this way for the vast majority of companies.
Also is this real time or actual time? In other words does it include time for meetings, holidays, sick days, change in staff, etc.
Not to argue with you Steph, but that indicates a management problem. If I ask one of my guys for an estimate, I expect that it will take them time. If I need a SWAG, I'll say that. If I need an accurate estimate, I'll tell them that and ask how long it will take. Most (competent)management understands that everything has a cost. The problem that I've seen repeatedly is that devs toss off a quick guess without thinking about it very much and then act surprised when they're held to it.
That said, I believe that a week is plenty of time to estimate a year-long project. However, you need experience estimating.
Look, I'm no genius at this but when asked to estimate a task, this is what my manager would normally get:
Time to Research fuzzy task C
Find libraries for X, Y, and Z
Assume cost of $A for libraries. Make sure we have this in the budget to avoid delays
Repository setup time
Tool configuration time (if different)
Feature A
Feature B
...
Integration Test time for x, y, z
I'm on vacation for a week
Bug fix
I need to interact with Susie in Manufacturing around this date and she says she will be having a baby and out for 6 months. You need to find me a replacement contact.
Incorporate feedback from Manufacturing. Historically this adds about 2 weeks to any project.
Add up all the times and provide an estimate along with any confidence intervals around each line item.
This is how I have done it in the past before we implemented more rigorous processes and never gotten complaints. It gives management plenty to data to work with and talking points for them to ask questions about.
I absolutely agree with you, what I'm saying is that this rarely happens. Most people say it's just bad management, but that's the norm. I use to consult until I started my own company and over the many companies I worked with maybe only a couple did it correctly.
Just to give you an idea, do you think Steve jobs ever really allowed an estimate like this or would've accepted it? It's not just him think Amazon, oracle, EA games, etc. they all have reputations for being aggressive and you either provide and estimate they like or they will provide it for you. And not just the big companies but most of the companies are like this too. Working long hours to try and meet estimates is more than common in our industry.
Again I agree with you, and I've seen a couple times, but that's it. It's very very rare is all I'm saying. It's part of the reason I started my own company. I understand that estimates are just estimates, and in most cases they aren't going to work without spending a good amount of time on them.
But even then an estimate is an estimate, the same as you get an estimate when doing some major renovations on your house. It's only when you open up the walls and do the actual work that you'll know the true time and costs. You can't predict the plumbing is bad and leaking in your estimation. And sometimes you don't even know who the staff will be. The only way to do a proper estimate is to start the actual work.
The only thing I disagree with is that what your describing is the exception rather than the norm. Well that and a week is not enough to do a year long estimate. Of course that also depends on the size of your team, it's easier to estimate for one person than a 20 person team. But 1 week is too short. but even with the appropriate amount of time you should still be ready to accept that once the walls are opened you never truly know what you will find...
Just a quick note, the reason many developers give quick estimates is that they know management won't give them the time to perform a proper estimate or they won't accept anything they don't want to hear, or after all is said and done will just override whatever you say and tell you how long it takes. This is wrong on all accounts but I would say this happens at 80-90% of companies. So after a while developers learn there's no point in even trying to give a proper estimate, or at least not spend a lot of time trying to be too accurate. That and many times there would be change requests, changes in features and functionality, but even less common was to see a revision of the estimate. I never saw a single revision, the original estimate was used, regardless how much chbage had happened since. I'm all for changing the features as you learn more but I've never seen an estimate be adjusted. If it's a year it stays a year.
What's even more interesting is that most of the discussions around anything tend to be bike shedding discussions, discussing the color of the paint, rather than actual hard stuff.
There are exceptions but that's much more the norm. You estimate based on what you know right now and it will never be adjusted if things get added. And you will only be given the minimal amount of time to make an estimate. And of course you hope they listen and agree, many many times your estimate gets overridden.
But that is exactly how I normally respond to estimate requests. "I have 70% confidence it will take between X and Y hours." That's a huge difference from saying "I have no fucking clue." If I say I think something can be done in 6 weeks and I'm 70% sure, Management now can weigh the risk that I'm wrong against the benefit.
Most software projects in a given company really don't vary all that much, so it often is like installing a WP blog. Since most companies do only minor variations of the same thing, you have pretty good history of how long the last variation took.
I can think of my last job for example. We built extremely complex medical devices, but yet we could say: software for this product will take us 4 years from concept to getting it on the market and we'd be accurate within a couple of months. It takes work and planning and sitting down and literally thinking of every major task along the way and trying to estimate how long it will take. Estimation of that sort is expensive and you have to have a good reason to take the time to do it. You also have to keep re-estimating as time goes on. Schedules aren't fixed in stone if they are to mean anything. Have to make a market window and you're running late? Well if your estimates mean anything, then features have to be removed or you ain't gonna make it!
I do agree with the management comment. You need good management that understands what they're doing. Management that understands if they change features, there is a cost. But it's up to developers to continue to make this clear to them.
> Estimation of that sort is expensive and you have to have a good reason to take the time to do it.
Just wanted to emphasize this sentence.
Good estimates take time and cost money -- time and money that aren't necessarily worth spending. In many cases you're better off living with order-of-magnitude estimates and spending most of your time doing the actual work -- depending on the nature of the business, of course.
It is honestly all over the board. I've worked with clients that had estimated the return, in many cases because it was very easy to do. Migrating from an expensive proprietary product (IBM MQ) to a much cheaper open source product (Red Hat AMQ) when you have 1000s of brokers and yearly subscription costs makes it very easy to calculate the savings. Another client was working with us to create a new product line from scratch. As part of that, they had to estimate the return on the product, the budget it would take, then justify that to their owners.
I've also had clients where they are not great at this. They are also invariably the ones with poor scope control and overall immature project management.
How did your store opening estimate get created in the first place? Making a good estimate takes time and it has an unknown ROI. Especially for the first store.
As has long been accepted, can get good estimates of time and cost for software project A if it is quite similar to and done by the same team as successful projects B, C, D, E, F, ....
IMHO, for software project A, break the work down into relatively small pieces and attack those. Make each piece small enough so that if that the work on that piece takes too long or even just fails, then the loss there is not too big.
Some of the most important pieces are about the experience of the team -- have they successfully done just such a thing before?
So, for such experience, start with the skills and, in particular, the documentation. So, start with the skills and experience with the operating system, the text editor, the e-mail program, the file system, the command lines, and the programming languages. Then move on to other related software, e.g., TCP/IP, HTML, CSS, JavaScript, SQL, AJAX, Web site security, collection classes, APIs, etc. So, for each of these, get the documentation, the skills, the experience, etc. E.g., have people write test programs that illustrate and test the tools, and some of the more important limitations on the tools, being learned.
Then start in, say, outline form, the waterfall development process by coming up with a requirements document, that is, what the heck the software is to do.
On writing this document, e.g., what level of detail to include, get some good consulting help.
Then move on to the architecture document. This document is an example of the standard project approach of divide and conquer. So, that division results in pieces, e.g., components of some kind. So, get started on some of the components. And have write some code to test those. Do these pieces one at a time so that a failure is not to expensive.
As long as the project is not struggling at some obstacle, is making good progress, each of the small steps looks okay, then continue on.
Soon begin to get some data on how fast the work is going, at least for each of the stages.
When are well into writing the code for version 1.0, then soon should have some okay estimates of the time to complete version 1.0.
Here see part of the truth: Much of the work is getting relevant skills and experience. Next much of the work, especially the risky part, is getting a clean description of what the heck the system is to do. With those parts done well, we should be talking mostly a nice road downhill from there to a good finish line.
Gee, in the middle ages in Rome there was a big project to move a huge stone obelisk some yards to one side. Big obelisk. Big project. Lots of timbers, ropes, horses, workers. It was successful.
Impressive until ask how the heck the obelisk got there in the first place? Sure, ballpark 1000 years before some of Caligula's slaves went to the upper Nile, cut the obelisk from solid stone, floated it down the Nile, across the Mediterranean, and to Rome, and erected it.
So, Caligula's slaves did some project planning, for a project they were likely mostly doing for the first time. And they got it done.
And now we are struggling with some software project for some Web based order entry and inventory system or some such? Ah, come on, guys!
The difference between a consultant with lots of real world experience and not.
Yes there is slippage, but you learn to plan for it. In the past week I had an employee have a baby and an employee have a parent pass away and another one have a horrible knee injury and be unable to work for a couple days, meanwhile we lost no time on a project by we are up some of our buffer.
If you don't plan to meet your deadlines even when faced with feature and scope creep ( which managing is another discussion entirely ) or things 100% out of your control there is other people that can and so your not going to be picked for projects that require real time mission critical demands.
And yea, if you figure that out, you get to charge more, and sleep less.
The fundamental activity of software development is _learning_.
Although it looks to an outsider like the main thing is typing stuff in (or pasting it in :-), the thing that takes all the time is learning what you should type, or learning why the code you already typed in isn't doing what you expected.
If you've done that exact project before then sure, you can estimate how long it will take to do it again. But that isn't often the case - usually we're trying to do something new, or better, or on a different platform. Always learning.
Not impossible, its akin to having a P vs NP problem.
Properly estimating a software project takes just as long as developing the software itself.
When people want estimates they somehow assume they can be given on the spot, or at most with a few hours of work. Nothing further from the truth.
A good software estimate is a project in its own right. It takes time, a deadline and a budget to break apart a software project, design a solution, identify its subtasks, identify its critical path, research alternatives, evaluate frameworks, tentatively assign tasks to teammates, write down an estimation report, discuss it with the customer, etc.
And then you end billing the customer for 120 hours of work to conclude the project can be done in five days, give or take a week.
This isn't a problem unique to software engineering. All kinds of things in business require estimating the unknowable. Investing in any business enterprise requires forecasting what the return on investment is and how long it'll take before you see those returns. These issues exist whether you're starting a hot dog stand, building an apartment complex, bringing a software product to market, or building an off shore oil rig.
Business people have put a great deal of worthwhile thought into how to manage these risks in ways that make sense for the company. For some fresh, eye opening ideas on how to do project management for software and other businesses, check out the book The Critical Chain. The short section at the end on metrics for investment is worth the price of admission alone.
Heh, I recall when a funny manager asked my team about an estimation. We agreed on a year (exactly something like 12-14 months). Then he went to his managers, sold the project with 6 months deadline, and was quite happy.
The outcome was simple: his estimation was good, the team was lazy.
I must urge any serious developer who has read that article to immediately forget about it!
Let's begin with the first phrase, "Software estimate". An estimate is an approximation. An approximation is an educated guess. So if someone said to you, "Please make an educated guess on when the software you are working on will be completed" To say it would be impossible would be the most absurd thing!
To make an educated guess, You would have to break down the tasks into manageable chunks that you can clearly reason about. Once you have your chunks of task, you will have knows, unknowns, some with risks or not, some that are complex or easy, etc. The problem with estimates is the unknown. If you are constructing software for an entity, then you must not approach it as an R&D, but rather, as a possible Research THEN Development. All your unknowns should be figured out at the research phase, then you estimate the development, else you should fight to remove those unknown tasks from the project.
Think about it, do you get asked for "Software research estimate" or "Software development estimate"? It's the latter, but most developers make the mistake of offering their estimate across both R&D. The really beneficial thing to developers about estimates if they do take it serious, is that it forces you to stop, think, design, and think again before coding.
The biggest concerns for developers is that estimates TURN into deadlines. This is a legitimate concern, but nevertheless this doesn't invalidate your estimate, which is nothing but an approximation. The other concern is that feature/scope creep into the project really puts a wide gap between the estimate and completion date. This again is true, remember your estimate was for the original requirements. So what are you to do as a developer? You must teach your manager if they don't know or whomever you are working for the difference between estimates and deadlines. You must also learn to keep feature creeps at bay or estimate those new features anytime they are added and adjust your estimate accordingly.
If you are a non manager or business. Please do think about this from their point of view. Projects have costs, which we can quickly approximate via cost of people working on it multiplied by the duration of the project. Part of running a business is managing your cash flow, knowing if you should spend or not. Without having an estimate of a duration of the project. How can businesses make decisions on which projects to fund? If you have a startup and $100,000 in cash. No income yet, 5 programmers that you pay $5,000 a month. Would you let them work on a project with no completion date, 1 yr estimate or 2 months estimate?
112 comments
[ 3.2 ms ] story [ 197 ms ] threadMeanwhile some crazy optimist went out and to catch a deer. It was many times more difficult than expected, but they returned to the cave with the deer and eventually reproduced more successfully than the accurate estimator.
Thus our ancestors were the crazy optimists.
More trouble than starving?
There must be a better way to package the no-estimates message, and it's not this.
Indeed, most projects are remarkably (and sadly) similar to other projects we've done before.
Saying no estimates is akin to saying "this is hard, we suck at it so we quit." The problem isn't solved, you still have no way of measuring how complete you are and when you'll be able to turn new things on.
We use a combination of story points and historical reference where we can during our estimating phase. So if we are talking about adding a new feature and wet estimate it out we go look at the actuals from a similar change that we did before to see how is base we might be. We also like story points because they are a ballpark and help us judge if we are estimating reasonably well. If we are continually having things signed low points that take data to finish, we can address that. Fast feedback and lots of iterations. The thing teams fail to do is to measure properly. If you measure too much no one fills out their stuff accurately and then you've just got garbage. Measure the wrong things and you're blind. Measure nothing (no estimates) and you're just giving yo.
We don't try to measure to the minute or hour. Things are basically 'quick' 1hr or less, 2hrs, 1/2 day, 1 day, or a 1/2 week for a task we know is big and hairy that has a lot of sub tasks that we don't really want to dive into the minutiae of it because what's relevant is that X does A and B, not that X requires T, U, and V to be done first.
That's the hard part for us, enough detail that its meaningful without spending all your time planning to plan.
ReRe's Law of Repetition and Redundancy [1]
A programmer can accurately estimate the schedule only for the repeated and the redundant. Yet,
A programmer's job is to automate the repeated and the redundant. Thus,
A programmer delivering to an estimated or predictable schedule is...
Not doing their job (or is redundant).
[1] https://news.ycombinator.com/item?id=11576683
Answer: Well get the previous developer back to do it then. Otherwise STFU, and let me get started.
"Why is this so expensive? Can't you just copy and paste some of the code from other parts of the system?"
I couldn't even respond right away, I had to censor myself before saying anything.
Background: They wanted pre-populated values/logic very specific to their workflow. It wasn't just an easy cut and paste because the software would behave differently for them.
2. Some of my clients do excellent work with estimations delivering on time and on budget and satisfied customer (In the Wardley Six Sigma zone - low uncertainty)
3. Estimation does not work in the Genesis or Product zones (high uncertainty)
We need to grow up beyond the "software estimation not possible".
The part you're working on (and trying to estimate) in almost all cases should be "new or unknown area".
http://blog.gardeviance.org/2015/10/agile-vs-lean-vs-six-sig...
It says "Six Sigma/Outsource" for a reason. The product is "ordered, known, measured, standard, obvious, low margin". If at all possible, this should be an off-the-shelf choice. Previously, that would have meant packaged software; now it mostly means getting something as a library or service.
Six Sigma is fine for manufacturing low-margin items, because you are trying to get high repeatability in the world of atoms. If you have one widget and need to make one million more, you use Six Sigma. But with bits, we already have near-perfect repeatability. If I have one integer and need a million more integers, the computer will do that at a reliability level well beyond six sigmas.
If software is being made in the Six Sigma/Outsource zone, it mostly means that one is reinventing the wheel. Sometimes that's necessary for historical reasons. E.g., if I'm working in a new language but need a library that doesn't exist yet, sometimes I really can just take the old spec and implement it anew. Or if I want developers to use my cloud service, maybe I need to just reimplement chunks of the AWS API, because Amazon won't give me that code.
But the great majority of software development should take place in the higher-novelty zones, where reliable project-scale estimation is somewhere between impossible and nonsensical.
(As an aside, casting people you disagree with as children is insulting.)
Though some of my clients create software for others duplicating and modifying existing products with high certainty.
But once modifications happen, I think you're headed back to the other end of the Wardley spectrum. Do the modifications meet user goals? Do they meet business goals?
In those circumstances, the only way you can return to high certainty is by creating very firm specs and resisting change orders. From the perspective of a contract development shop, I'm sure that's fine. But for the people actually paying for the software, that's a process smell.
The commercial and practical validity of modifications can only be fully evaluated by shipping. But large specs require long release cycles. This creates a long period where learning is deferred. Estimates only become manageable because the project is saving up trouble for later. A period of artificially high predictability will be followed by a period of artificially low predictability.
And even in the pure duplication case, what comes after that? I can't think of many successful pieces of software that stopped at 1.0. Once the duplicate is released, most businesses will need to respond to their customers, doing novel and innovative things specifically for them. So at best, you're back in the fast-cycle end of the spectrum. But at worst, you have executives who are attached to specs and estimates and long timelines, which are the wrong tools for innovation. You have development processes that match. And you probably have a code base that's not set up for flexibility.
Given that, even if I were, say, making that AWS competitor, I'd still use all my tools from the fast-cycle innovation end of the spectrum. Because even a pure duplicate assumes that a) there is nothing left to learn, and that b) the context of sale/use is not changing. Estimation is always a bet that you won't learn anything important between the start and the end. Given how smart my colleagues are, that's a bet I'm rarely willing to make.
http://www.michaelrwolfe.com/2013/10/19/50/
You're not a novice programmer, you've done it before. And if you are a novice hiker, you shouldn't be doing it alone.
Similar enough that I can make good estimates. You can too, with experience.
I can tell you, if you have a relatively long project, it's worse than that: Susie (or someone else) will probably quit. Better take that into your estimate calculation.
Remember, estimates are an outer bound; if you finish early, that's ok.
> And the best thing is if they then get 'their 10x code slinger' who does it indeed very fast, sends the code back to you and you are stuck with something akin to a broken toilet with diarrhea coming out the sides
A grim picture to think about but I find this to be true a lot of the time.
p : number of people
c : number of changes into specifications (= learning during implementation)
f : number of frameworks, libraries, tools.
x : number of new people, frameworks, libraries, tools
e : initial time estimate in weeks
correct estimate: E = e + e(0.1p + 0.2c + 0.3f + 1.1^x)
E=e*5
Almost always works.
I usually have a pretty accurate estimate for small tasks but never go back and add the time spent on fixing recessions caused by that task. Or the time spent on extra smoke testing due to missing automation.
Sometimes your test matrix is too big. But I agree: sometimes you have to move slower in order to go faster overall and have better estimates. Estimates have to take into account testing and regression testing, which most people don't so it can make you look slow if you start doing that.
Pad your estimates, because you are typically going to have similar bugs (for similar projects), and you can account for those unknowns probablistically. You don't know the exact bugs that will be found, but you've already predicted that some will be found, so include that in your estimate.
I've had significant success in using historical metrics to accurately estimate software projects.
It takes diligence and professional experience to learn to do this, but it is possible in my experience.
Unfortunately I find this philistine sentiment quite frequently in the software world: "I've had difficulty doing X. I am smart. I am senior. I am a professional. Therefore X must be impossible." Isn't it possible you haven't been stubborn/hard-working enough in your own personal growth?
It works best on a long-running project. When you plan a new feature, you can gauge really accurately at what speed your current team developed a feature of comparable complexity, so as you go forward you get better and better estimates.
Joel Spolsky explained it there: http://www.joelonsoftware.com/items/2007/10/26.html
I know this because every line of code I've written (using that company's process) for years has been measured and timed. So now the estimation process becomes a matter of figuring out how many LOC (as a proxy for effort) a given feature/project will be and then dividing that number by 30.
It is scarily accurate, but like I said, it's been aggregated over thousands of lines of code for years.
Give yourself extra time: no one will complain if you finish early.
Unless you worked on a few products, your going to be bad at estimating larger pieces. A lot of software is very similar.
Software estimation is required for real business to ever consider software projects. The key is to embrace uncertainty, and instead of estimating that task X will take exactly Y days, estimate a range of possibilities. I like the three point system, estimating a best case, worst case, and most likely case. It explicitly asks you to think about the long tail for when things go wrong. Then use statistical methods to combine estimates and create estimates for multiple confidence levels.
This is all covered in a great book, "Software Estimation: Demystifying the Black Art" (https://read.amazon.com/kp/embed?asin=B00JDMPOVQ&asin=B00JDM...). It's not that long a read, and it goes over the problems covered by this blog post but doesn't throw it's hands up in the air and say estimation is impossible.
Because the benefits of the software project would far outweigh the benefits of a new store? With the completion of the software project you could open an order of magnitude more stores? Completion of the software project will increase the profitability of existing stores such that they make more than a new store at existing (hypothetical, since the store is not open yet) profitability?
If opening a store takes 6-9 months, and we have a software project that can make all of my 100 stores ~10% more efficient that needs 2-4 years, then the software project looks a lot more feasible! (ignoring that calendar time is not exactly cost, and that we may need shorter term wins to stay in business)
The problem is if, when asked for an estimate, we say "I don't know and I never will know until I'm done". Now I don't know if stores will be more efficient 2 weeks or 20 years from now.
How can you estimate something that isn't done? You can throw some words out there, but they are likely meaningless.
If you are operating solely on predictive ROI, then maybe investing in ambitious software projects is not for your business.
>Now I don't know if stores will be more efficient 2 weeks or 20 years from now.
That is your problem, not the software's or of the people creating the software.
Maybe because you're trying to innovate? Will opening a new store dramatically change your company's market outlook? Will it open up entirely new markets for products you don't yet have? Will it solve new problems for people?
Milton Hershey offered his first chocolate bar in 1900. It took him a further five years to perfect the manufacturing processes that produced the Hershey's Kiss. None of those processes, and none of the machinery to execute them existed when he started. He had to invent them.
Today, 110 or so years later, the company produces around 60 million of them a day which it ships all over the world. Probably he should have just opened another store.
No offense, but people who think the way you do about risk are exactly the reason large companies lose their ability to innovate at all.
> No offense, but people who think the way you do about risk are exactly the reason large companies lose their ability to innovate at all.
Exactly. This is why the startup ecosystem continues to thrive even after the age of the big IPO is over. Startups are akin to R&D for larger companies who don't want to (or can't) take the risk to try something new.
Even if what you are doing is truly research that is completely open ended, there still have to be controls placed on it. Budget/grants/runway will run out at some point.
What are these rigorous techniques you speak of? I seriously, honestly want to know.
It's "Software Estimation: Demystifying the Black Art" by Steve McConnell (https://read.amazon.com/kp/embed?asin=B00JDMPOVQ&asin=B00JDM...). Same author as "Code Complete" and "Rapid Development", which are also great books but much longer and showing their age a little more.
Second, in estimation, previous performance is often indicative of future performance. If something took you X days before, it will likely take you roughly X days next year. It's not likely to take 10X the time. The real problem is that very few teams actually track their time on task accurately so no one on the team knows what X was. Without that data, trying to get accurate estimates is a waste of time.
A very large part of the problem is that most developers are educated in Computer Science or other areas and simply haven't been trained in estimation techniques. Software Engineering is about the process side of development, and one of the benefits of process is predictability.
Managers, for the most part, keep saying that. Usually its because they have, or they've read about someone who has estimated accurately one or more times. How do I know they were not simply lucky? We cannot accurately estimate the time it will take to construct a building, and we've been constructing buildings for thousands of years.
Producing movies, for example. Even if you cast established actors and limit yourself to low-risk sequels of successful franchises, chances of box office bomb are substantial. Book publishing would be even better example; thankfully, writing a book is dirt cheap comparing to developing software or making movies. Pharmaceutic research is absolutely unpredictable — but big pharma companies are usually quite profitable, like software companies (their research/production patterns are similar).
New software development is _always_ innovation (because, if your problem is already solved by existing software, it is way cheaper to buy this software instead — as cost of production is close to zero compared to the cost of development). Innovation is _always_ unpredictable. Building the business on innovation is therefore a hard problem.
Producing a movie isn't rocket science either. But if anything it's more complex than most software projects, because you have to arrange the finance, hire hundreds of people spread across tens of teams, move people and stuff around, design and build things, knock things down, add a lot of stuff in post - which may need custom software - and so on.
And that's just the footage. You also have to market and distribute the movie worldwide, work out deals for residuals on various media, and so on.
And everyone has to be paid on time, near as, or work stops.
Estimating profit is a completely different job, unrelated to keeping the machine running from start to finish. It's barely a footnote. Studios are mostly not bad at it, but they do sometimes get it very wrong.
Designing a CRUD website that scales is nowhere close in terms of complexity.
But here's an interesting thing: movies only get finished because there's a hierarchy of talent and delegation. The director/producer can hire an expert or a ready-made team to handle part of the project, and delegate that entire element to them. From then on the producer/director needs to do some oversight, but the details (should) "just work."
Startups don't work like this. Startups hire everyone on a permanent contract basis, often as individuals, try to build them into teams internally, often on the basis that anyone sub-senior is pretty much interchangeable with everyone else at the same level, and generally delegation doesn't happen in the same way.
There's really no reason why teams of programmers couldn't get together, set up service companies, and hire themselves out as consultant solution providers. After a few successes a team should have it made.
This doesn't happen. I think that cultural difference is very interesting.
It is impossible to know for sure. It's a bet. It's also impossible to say precisely what the DOW will be at next Thursday. But you can make a guess with confidence intervals, long or short the appropriate index, and make some money.
The problem is that "real business" people want impossibly precise estimates for software projects in particular. If software estimates were more like medical prognoses or financial reports, they would include confidence intervals and risk assessments, and we would be having another conversation.
Just because you can't always have an accurate estimate doesn't mean it's now worth it.
Also remember that estimates are based on resources. We were able to go to the moon in a decade but a lot of work was done before and the budget to get there was much higher than what most software development efforts ever get relatively speaking. It might not be going to the moon but that doesn't mean some projects aren't very complex either.
I'm not an avid hiker, but I've done enough weekend hikes to know that 10 miles per day can be pushing it for me on some terrain. I wouldn't ask someone who has never hiked ever before to estimate how long a hike would take. I wouldn't ask a programmer to estimate how long it takes to construct a bridge.
In terms of never having done it before, this is true of most software projects. If you're building the same house from the same plans then this is the same as say installing a Wordpress blog, easy to estimate. But if you were to ask how long it would take to build Wordpress from scratch, even having a Wordpress sample app, it would be very hard. Not only that but there are a lot of assumptions. Is it for a handful of visitors or are we talking a million visitor blog?
Ignoring that, let's go back to the pharma example. How long do you estimate developing a new drug will take? It's not the first drug your company has developed. Anything beyond a week is hard to estimate.
Back to your estimate of 25% within 4-8 weeks, this would never be accepted because it could mean anything.
The worse part, I an tell you right now, even if new functionality or changes are added, you will be elf to 4-8 weeks, the 25% will be ignored. And in most cases management will hold you to 4 weeks because this is what was sold to their bosses.
So then the hard part is communicating to the business the current estimates and confidence level and that we can do some up front work to tighten down our estimate and the schedule. This is actually part of ACM's "Software Engineer Code of Ethics": "3.09. Ensure realistic quantitative estimates of cost, scheduling, personnel, quality and outcomes on any project on which they work or propose to work and provide an uncertainty assessment of these estimates."
As for pharmaceuticals, I have no experience there. I doubt that they really just let scientists go off and do whatever they want with unlimited budgets, and just live with the results.
I agree with what you're saying and that's all great and awesome but in the real world it unfortunately doesn't work that way. In most cases you have a day to a week to come up with an estimate for a year long project. If you by great luck get a cognate to do a spike, you get maybe an extra day. Ok most cases actually you would've maybe gotten an hour to a day at best and there would be no spike, and if you did get a spike it would be maybe an hour. Again I agree with what you're saying but it just doesn't work this way for the vast majority of companies.
Also is this real time or actual time? In other words does it include time for meetings, holidays, sick days, change in staff, etc.
That said, I believe that a week is plenty of time to estimate a year-long project. However, you need experience estimating.
Look, I'm no genius at this but when asked to estimate a task, this is what my manager would normally get:
Time to Research fuzzy task C
Find libraries for X, Y, and Z
Assume cost of $A for libraries. Make sure we have this in the budget to avoid delays
Repository setup time
Tool configuration time (if different)
Feature A
Feature B
...
Integration Test time for x, y, z
I'm on vacation for a week
Bug fix
I need to interact with Susie in Manufacturing around this date and she says she will be having a baby and out for 6 months. You need to find me a replacement contact.
Incorporate feedback from Manufacturing. Historically this adds about 2 weeks to any project.
Add up all the times and provide an estimate along with any confidence intervals around each line item.
This is how I have done it in the past before we implemented more rigorous processes and never gotten complaints. It gives management plenty to data to work with and talking points for them to ask questions about.
Just to give you an idea, do you think Steve jobs ever really allowed an estimate like this or would've accepted it? It's not just him think Amazon, oracle, EA games, etc. they all have reputations for being aggressive and you either provide and estimate they like or they will provide it for you. And not just the big companies but most of the companies are like this too. Working long hours to try and meet estimates is more than common in our industry.
Again I agree with you, and I've seen a couple times, but that's it. It's very very rare is all I'm saying. It's part of the reason I started my own company. I understand that estimates are just estimates, and in most cases they aren't going to work without spending a good amount of time on them.
But even then an estimate is an estimate, the same as you get an estimate when doing some major renovations on your house. It's only when you open up the walls and do the actual work that you'll know the true time and costs. You can't predict the plumbing is bad and leaking in your estimation. And sometimes you don't even know who the staff will be. The only way to do a proper estimate is to start the actual work.
The only thing I disagree with is that what your describing is the exception rather than the norm. Well that and a week is not enough to do a year long estimate. Of course that also depends on the size of your team, it's easier to estimate for one person than a 20 person team. But 1 week is too short. but even with the appropriate amount of time you should still be ready to accept that once the walls are opened you never truly know what you will find...
What's even more interesting is that most of the discussions around anything tend to be bike shedding discussions, discussing the color of the paint, rather than actual hard stuff.
There are exceptions but that's much more the norm. You estimate based on what you know right now and it will never be adjusted if things get added. And you will only be given the minimal amount of time to make an estimate. And of course you hope they listen and agree, many many times your estimate gets overridden.
Most software projects in a given company really don't vary all that much, so it often is like installing a WP blog. Since most companies do only minor variations of the same thing, you have pretty good history of how long the last variation took.
I can think of my last job for example. We built extremely complex medical devices, but yet we could say: software for this product will take us 4 years from concept to getting it on the market and we'd be accurate within a couple of months. It takes work and planning and sitting down and literally thinking of every major task along the way and trying to estimate how long it will take. Estimation of that sort is expensive and you have to have a good reason to take the time to do it. You also have to keep re-estimating as time goes on. Schedules aren't fixed in stone if they are to mean anything. Have to make a market window and you're running late? Well if your estimates mean anything, then features have to be removed or you ain't gonna make it!
I do agree with the management comment. You need good management that understands what they're doing. Management that understands if they change features, there is a cost. But it's up to developers to continue to make this clear to them.
Just wanted to emphasize this sentence.
Good estimates take time and cost money -- time and money that aren't necessarily worth spending. In many cases you're better off living with order-of-magnitude estimates and spending most of your time doing the actual work -- depending on the nature of the business, of course.
I've also had clients where they are not great at this. They are also invariably the ones with poor scope control and overall immature project management.
That's pretty much the definition of "estimate".
IMHO, for software project A, break the work down into relatively small pieces and attack those. Make each piece small enough so that if that the work on that piece takes too long or even just fails, then the loss there is not too big.
Some of the most important pieces are about the experience of the team -- have they successfully done just such a thing before?
So, for such experience, start with the skills and, in particular, the documentation. So, start with the skills and experience with the operating system, the text editor, the e-mail program, the file system, the command lines, and the programming languages. Then move on to other related software, e.g., TCP/IP, HTML, CSS, JavaScript, SQL, AJAX, Web site security, collection classes, APIs, etc. So, for each of these, get the documentation, the skills, the experience, etc. E.g., have people write test programs that illustrate and test the tools, and some of the more important limitations on the tools, being learned.
Then start in, say, outline form, the waterfall development process by coming up with a requirements document, that is, what the heck the software is to do.
On writing this document, e.g., what level of detail to include, get some good consulting help.
Then move on to the architecture document. This document is an example of the standard project approach of divide and conquer. So, that division results in pieces, e.g., components of some kind. So, get started on some of the components. And have write some code to test those. Do these pieces one at a time so that a failure is not to expensive.
As long as the project is not struggling at some obstacle, is making good progress, each of the small steps looks okay, then continue on.
Soon begin to get some data on how fast the work is going, at least for each of the stages.
When are well into writing the code for version 1.0, then soon should have some okay estimates of the time to complete version 1.0.
Here see part of the truth: Much of the work is getting relevant skills and experience. Next much of the work, especially the risky part, is getting a clean description of what the heck the system is to do. With those parts done well, we should be talking mostly a nice road downhill from there to a good finish line.
Gee, in the middle ages in Rome there was a big project to move a huge stone obelisk some yards to one side. Big obelisk. Big project. Lots of timbers, ropes, horses, workers. It was successful.
Impressive until ask how the heck the obelisk got there in the first place? Sure, ballpark 1000 years before some of Caligula's slaves went to the upper Nile, cut the obelisk from solid stone, floated it down the Nile, across the Mediterranean, and to Rome, and erected it.
So, Caligula's slaves did some project planning, for a project they were likely mostly doing for the first time. And they got it done.
And now we are struggling with some software project for some Web based order entry and inventory system or some such? Ah, come on, guys!
Yes there is slippage, but you learn to plan for it. In the past week I had an employee have a baby and an employee have a parent pass away and another one have a horrible knee injury and be unable to work for a couple days, meanwhile we lost no time on a project by we are up some of our buffer.
If you don't plan to meet your deadlines even when faced with feature and scope creep ( which managing is another discussion entirely ) or things 100% out of your control there is other people that can and so your not going to be picked for projects that require real time mission critical demands.
And yea, if you figure that out, you get to charge more, and sleep less.
Although it looks to an outsider like the main thing is typing stuff in (or pasting it in :-), the thing that takes all the time is learning what you should type, or learning why the code you already typed in isn't doing what you expected.
If you've done that exact project before then sure, you can estimate how long it will take to do it again. But that isn't often the case - usually we're trying to do something new, or better, or on a different platform. Always learning.
Properly estimating a software project takes just as long as developing the software itself.
When people want estimates they somehow assume they can be given on the spot, or at most with a few hours of work. Nothing further from the truth.
A good software estimate is a project in its own right. It takes time, a deadline and a budget to break apart a software project, design a solution, identify its subtasks, identify its critical path, research alternatives, evaluate frameworks, tentatively assign tasks to teammates, write down an estimation report, discuss it with the customer, etc.
And then you end billing the customer for 120 hours of work to conclude the project can be done in five days, give or take a week.
Business people have put a great deal of worthwhile thought into how to manage these risks in ways that make sense for the company. For some fresh, eye opening ideas on how to do project management for software and other businesses, check out the book The Critical Chain. The short section at the end on metrics for investment is worth the price of admission alone.
The outcome was simple: his estimation was good, the team was lazy.
Let's begin with the first phrase, "Software estimate". An estimate is an approximation. An approximation is an educated guess. So if someone said to you, "Please make an educated guess on when the software you are working on will be completed" To say it would be impossible would be the most absurd thing!
To make an educated guess, You would have to break down the tasks into manageable chunks that you can clearly reason about. Once you have your chunks of task, you will have knows, unknowns, some with risks or not, some that are complex or easy, etc. The problem with estimates is the unknown. If you are constructing software for an entity, then you must not approach it as an R&D, but rather, as a possible Research THEN Development. All your unknowns should be figured out at the research phase, then you estimate the development, else you should fight to remove those unknown tasks from the project.
Think about it, do you get asked for "Software research estimate" or "Software development estimate"? It's the latter, but most developers make the mistake of offering their estimate across both R&D. The really beneficial thing to developers about estimates if they do take it serious, is that it forces you to stop, think, design, and think again before coding.
The biggest concerns for developers is that estimates TURN into deadlines. This is a legitimate concern, but nevertheless this doesn't invalidate your estimate, which is nothing but an approximation. The other concern is that feature/scope creep into the project really puts a wide gap between the estimate and completion date. This again is true, remember your estimate was for the original requirements. So what are you to do as a developer? You must teach your manager if they don't know or whomever you are working for the difference between estimates and deadlines. You must also learn to keep feature creeps at bay or estimate those new features anytime they are added and adjust your estimate accordingly.
If you are a non manager or business. Please do think about this from their point of view. Projects have costs, which we can quickly approximate via cost of people working on it multiplied by the duration of the project. Part of running a business is managing your cash flow, knowing if you should spend or not. Without having an estimate of a duration of the project. How can businesses make decisions on which projects to fund? If you have a startup and $100,000 in cash. No income yet, 5 programmers that you pay $5,000 a month. Would you let them work on a project with no completion date, 1 yr estimate or 2 months estimate?
That's a trite observation. What matters is than an estimate can be made better than not estimating at all.
It's funny how all these problems come up in other disciplines, but they don't throw their hands in the air and quit.
They accept that the Nirvana Fallacy is not an acceptable reason to not estimate and they go ahead and do it anyway.