HN Explorer

How fair do you think time estimate within 30% is? Plus/minus 20% if over 3 days

8 points by b0rsuk ↗ HN
Today our boss came up with a "reorganization" to "improve efficiency". He expects us to estimate time needed with max 30% deviation, and max 20% deviation if it's over 3 days. Software projects.

* * *

Background (SPOILERS)

boss is an old programmer, an American client is outsourcing its work to us. The work is two repos: a) IE-compatible Javascript without Babel or Node (ES5), b) Python 2.7 with no unit tests, no dev database, lots of "except: pass". Both repos are centered around a god class, functions often 100,200,300+ lines , no documentation, very few comments, superficial but nitpicky code reviews, one or maybe two questions answered per day.

16 comments

[ 3.6 ms ] story [ 47.8 ms ] thread
gshdg
I’d say that depends on how much effort he’s willing to let you put into the estimates. It’s often possible to de-risk estimates to that degree, but that basically requires putting in a significant chunk of the work up front to break the problem down and research all the unknowns.

If your boss wants that level of certainty, you should ask him to teach you how to achieve it and see how he thinks such a thing is possible. I’m not suggesting he’s not delusional. Just that if you pick his brain and then ask how to deal with the types of uncertainty you’re concerned about, you’re more likely to get to a place where you and the boss are aligned. (This assumes your boss is an even vaguely reasonable human being, tho. YMMV.)

b0rsuk
FYI I politely asked him, and he snapped ~ "foolhardy" (not directly translatable).
fenier
It's not exactly fair.. and also not likely without extensive time spent doing the estimation, like sure - you can be more precise, but the more precise you need to be, the longer it takes you to frame the estimate.

Two books help with methods of doing this.

Rapid Development: Taming Wild Software Schedules: https://www.amazon.com/Rapid-Development-Taming-Software-Sch...

Software Estimation: Demystifying the Black Art https://www.amazon.com/Software-Estimation-Demystifying-Deve...

This likely does mean you'll need to be allowed to deploy automated test cases, develop documentation and the entire nine yards, because it's going to be far harder to hit any reasonable deadline if you don't have a known development pipeline.

You'll need to reduce your uncertainty as much as you can to even have a chance - and even then, things will still blindside you and double (or more) the actual vs. the original estimate.

b0rsuk
We talked to him, and no. We won't be allowed.
fenier
Then take it up with his boss, fight him on it, or look for work elsewhere. It's unreasonable that you'd be adversely graded on something that industry has historically struggled with.
shoo
How are the estimates used? Are they converted into _quotes_ that are stated to the client for pieces of work? What happens if you give an estimate, then 1 week later you learn new information, then have a new more accurate estimate? How will your boss respond to the new estimate? It is possible your boss is saying estimate but really meaning "quote".

Who owns the codebase? Does it belong to the client or to your boss? Is it in anyone's interest that the quality of the code be improved over time?

There is a conflict between efficiency and accuracy of estimates. If accuracy is a goal, you can improve accuracy at the cost of efficiency by giving very high estimates, then working slowly if the task turns out to be relatively simple.

The codebase sounds like a bit of a tar pit. You may be interested in reading Feathers' book "working effectively with legacy code". This describes patterns and tactics to use to get automated regression tests in place around applications, so that you can start to refactor confidently and add unit tests. You need time and support to work on these if the existing code is bad. It can be very slow going and hard work. If you cannot convince your employer to support some changes to start gradually improving code quality over time, you should look at options for alternative employment.

b0rsuk
I do own the book.

Only the issue here is, strictly Jira-oriented programming. You only ever modify lines that fall into your ticket. You don't lint, don't remove trailing spaces, insert missing semicolons. You DO.NOT.CLEAN.UP. It's not enough to have IE-compatible code that follows naming conventions, README.md and .eslitrc (used very sparingly). You must use arbitrarily chosen existing functions, and on code review you're given very vague suggestions like "use the function we have for that", or "eh". Files commonly over 3000, 5000 lines.

quickthrower2
Id read up on the agile methodologies and not so much the processes but why they exist. Estimates within 30% is waterfall and may not be the best approach for web apps. It might be better for the client and you to use an agile approach. Or if not at least an open dialog where you can say oh X will take longer because of this technical issue, but how about we also do Y and Z, which are things you can do easily the add a lot of value etc. coding to strict estimates all the time sounds like a horrible situation.
b0rsuk
Thanks, but we do have sprints and bi-weekly progress meetings and are ostensibly working using agile. I have my opinion on my boss and the whole situation, but I'm under a heavy reality distortion field and wanted an outsider perspective. The boss basically treats any estimate as a commit. And yes I did point out (not in front of boss) that this smells like Waterfall. Agile for requirements, waterfall for costs and deadlines.
quickthrower2
The only thing I can come up with, if you want to stay in the job, is clandestine overestimation. Anything where you are an expert but no one else is, add some padding, which makes the overall estimate for the job generous.
b0rsuk
Yup, tried that until boss yelled at me. You can say any estimate that's low enough, or you'll get a stern "that's too long", or even "bullshit". After boss left my 3 team members piled on me to make the estimate 5 days instead of my 2 weeks, saying it can be later extended. I said then why not make it 2 weeks and shorten layer. No response.

The task could take 5 days but it's a lottery. A bugfix in a convoluted system.

I would go with the 5 days, but I don't trust the boss. He never takes responsibility for any setback.

bobm_db
Yeah good luck with that.

Problem is, time is just _one_ variable. It's possible to hit every estimate spot-on, if you allow the other variables to change wildly (number of bugs, completeness of features etc). It's kind of like in quantum mechanics, where by knowing one property of a particle, you have to let go of all the others (maybe).

Anyway, I blogged about this a lot here, if it is any use: https://riskfirst.org/Estimates

It seems wrong-headed to be insisting on this, so maybe education is the answer?

muzani
It's pretty easy to demonstrate that when most people are asked to do an estimate with 80% accuracy, they do estimates at 30% accuracy. There are studies on this, but make 5 people in a room do it and it's a lot easier to believe. People are really bad at estimating.

One trick is to make them really small chunks of less than 4 hours. When you add up the chunks they tend to average out quite accurately.

Estimations are quite a pain though. I've spent days doing and negotiating estimations of things that take less than a day to do. And one downside is Parkinson's Law - if you overestimate, it slows down development, and if you underestimate, it's far worse.

While you can make your estimations more accurate, you can't do it without putting a heck lot of effort and manpower... which defeats the purpose of doing it for efficiency.

b0rsuk
That's true as long as the people are not being bullied. In my case, I estimated something I barely worked on in an area of code I barely knew to take 2 weeks. When the boss left, the remaining 3 people kept telling me to estimate it to 5 days "because you know what boss thinks about 2 weeks". Then told me that 5 days is just an initial estimate and it can be later extended. I responded that likewise, 2 weeks can be scaled down and I'll be happy to be proven wrong. No response from them. I kept repeating the question.
heelix
Woof. This is how you continue to expand technical debt until it destroys the project. I'm hoping it is a smaller time window for projection, so you can ... never mind. I see there are no tests, dev environments, docs. Just make sure the check clears while you fly this pig all the way to the crash site.

(We went from the same sad state to having things more or less corrected. Every bit of code we touched, we added sanity testing, automation, etc. It took about two years, however, with 10-20% of the time fixing code/process stink)

b0rsuk
Over the course of the year I worked here, boss went from "yeah we'll eventually get unit tests, they will finally realize..." to downplaying my effort and difficulties, playing the authority card (programmer for 20 years), veiled threats by referring to my personal feedback report, convincing me that not having a single pull request reverted is not an achievement because everyone has one every now and then... not actually true, because early in my work here he called me twice after work about the reverts, only to find out later that it was somebody else's PR. I think he's a deeply manipulative individual and I'm jumping out of this company without a parachute (no another secured job), I have good reasons to afraid he would make me commit to these deadlines, blame for not achieving them, and fire me with a bad opinion. I need to make the first move.

I wanted an outside perspective to see if I'm perhaps overreacting.

I do believe this code is really fixable with some months of effort, but management will have none of it. Refactoring provides no perceived shareholder value for them. And I did try to clean it up, and tried to build support in our team for doing so. It's mostly political/organizational roadblocks. They want code monkeys. Code cleanups are viewed with deep suspicion even by code reviewers.