I've worked places where 6 days would be a miracle. For me, the worst has always been when the database query plan suddenly switched to something with a full table scan (I'm looking at you Sybase) or the always fun full sort merge (nice going Ingres). It always seemed to come down to forcing indexes which tended to get deployed fairly quickly.
Honestly, for code that controls millions of dollars worth of business and could seriously screw over a lot of people if a subtle flaw was introduced, I don't see a problem with doing it this way.
Might be overkill for one line of code, but where do you set the threshold? If you say any change longer than four lines is subject to this process, then people will work on trickling in new stuff in increments of four lines at a time.
Reliability and quality are obviously important, but you have to question whether all these steps are actually helping you to achieve that.
All of these items in the story add overhead and reduce agility, but potentially do very little to improve quality:
- Mandatory paper trail via fields on the issue tracker
- The need for documented internal sign offs
- Standards and policies with no clear reason for the existence
- 'Micro' Code reviews
- Excessive permissions and security processes internally
- No pragmatism with regards to testing the change
This kind of stuff is a constant and substantial overhead on getting anything done. You need to look at these processes very critically and be very sure of the benefits in terms of quality before introducing it.
If these are the proportions then I really disagree. And actually these are the proportions I see (or better). If you have 19 out of 20 changes done correctly, that means probably 1 every week will fail. Now the question is how will it fail - spelling mistake? single request rejected? single request failed? customer data lost?
If you let too many simple issues through, you're likely to find yourself completely failing when a very simple thing breaks. First you'll find that your logging is not correct, so you'll need to fix that first and reproduce, then you'll find that the rollback doesn't quite work and you've got some bad data to fix manually, then you'll find that this is actually a simple error masking some really nasty bug, etc. etc. I've seen that once or twice and I really believe in the broken windows theory now. I'd be more glad with a reasonable slowdown, than people pushing ahead instead of stopping to think about long-term issues.
Yes, I'm the guy who rejects ~4 out of 5 changes during code review initially (on average). Then again, I'm the guy who gets woken up when things fail - not everyone does, but it really gives you the appreciation of why you want to test those exception branches. I hope that people will be as strict about stuff I submit too.
Exactly. The point bengl3rt was making, I assume, is that cavalier avoidance of process is a bad thing because it allows mistakes to happen that would be caught. That much is true.
But the assumption that's wrong is that all processes avoid mistakes. Clearly they don't.
And some of the examples here are just plain cargo cult misapplications of good ideas. The point behind code review is to catch design flaws in new code. You never want to demand refactoring of existing code in order to patch features, that hurts, it doesn't help.
Having just (as in hours ago) been through the wringer of adding new features to a mass of spaghetti that had not been touched in 10 years, the only way I could maintain my sanity (and have some assurance beyond regression testing) that I wasn't breaking something, was to refactor the existing code and then insert my changes.
Sure. That's the point behind refactoring, it makes changes to existing code flow more smoothly. But the case here doesn't fit that at all: the change as described was a change in configuration (that just happened to be stored in a code variable), yet it was being reviewed as if it were a new feature being added through development. That's the "cargo cult" part -- refactoring in the course of development is good. Rules demanding refactoring are bad, because they hit false positives (in this case, a high priority configuration change).
Skilled people without a process will always find a way to get things done. Skill begets process. But process doesn’t beget skill. Following a recipe won’t make you a great chef – it just means you can make a competent bolognese. Great chefs don’t need cookery books. They know their medium and their ingredients so well that they can find excellent combinations as they go. The recipe becomes a natural by-product of their work.
Er... great chefs do need cookery books. They may not refer to them as often, but you won't find many chefs out there without a collection of cookery books. They still use recipes for things they're not familiar with.
Besides, skill may beget product, but it doesn't necessarily beget process.
>> Besides, skill may beget product, but it doesn't necessarily beget process.
Oh boy, talk about not seeing the forest for the trees. For skill to beget a successful product, you must follow a process to get from nothing to product. So in the course of creating a successful product you have automatically created a process -- the process to build the product. That process may be used one time, or multiple times -- but it's still a process.
Not seeing the forest for the trees? Hell, if we're going to use that loose a definition, everything is a process, regardless of skill level. Even if you end up without a product, to get the steaming pile of crap you abandoned, you went through a process.
1) The fields are probably mandatory because so many times routine entries were not filled correctly, and even MORE overhead was required going back and forth between teams to find out all of the details. If there is a way to avoid this overhead, again, how do you stop people abusing it?
2 & 3) Yeah.. bad overhead.
4) A sanity check is ALWAYS a good idea, whether its a fresh intern or the most seasoned programming in the company making a change.
5) They security processes might seem excessive but once again are probably in place due to issues in the past that have required the review/signoff be there to stop major disruptions getting through.
6) As someone who worked in QA for a while, there is no way in a hell someone can say "push this change through" without a strong questioning of why. QA is there (in most cases) to be that last line of defence. Often letting people know just how many people are really going to be affected by this change.
Being on a testing team often gives you a broader oversight of how all of the components of the code work together, while people working on one project can sometimes get tunnel vision on getting their thing out the door.
Edit: Just like to point out, I'm agreeing with you
I really like to see a pragmatic relationship between Dev and QA where you reach agreement on the best way to build confidence in a change.
QA should absolutely be able to push back on Dev with regards to quality and relevant test cases, but likewise Dev often need to be able to direct the testing and mutually agree on the scope that will get the change signed off adequately.
I have to take issue with this one. Code (and Requirements, and Design) reviews, when done properly probably have the greatest impact in improving product quality. No, I don't have time to look up the SEI research on the subject, but it has been consistently shown to be true.
Part of the problem I had with the article was that Shirley was rejecting code for non-specific reasons. You can't say "it's not written down anywhere." If it's important enough to reject the code, it's important enough to write down.
I dunno... the odd bug fix has the potential to really screw things up, when the person fixing it isn't the person who wrote it originally, or it's been months since then.
We have mandatory code reviews for all changes and I think it works quite well. A person making a one or two line bugfix may think it's quite innocuous but it may have a bigger impact on other parts of the system than they realize. Many eyes help reduce the likelihood of that.
I would add that crappy processes and tools are usually in place because the management/leadership in an organization has made them too brittle. While you don't want to wild-west every "critical" issue, bad spots in a process will naturally highlight themselves over time.
Let your tools and processes constantly evolve instead of blindly subscribing to a methodology, and you'll be in much better shape.
I agree that it's worth being careful on mission-critical code, but this isn't being careful, it's just box-ticking. Ed said he'd tested the actual change, but most of the time is taken up by shifting it out to some parameters file, lengthening its name and waiting for permissions.
No one seems to have pointed this out yet, but the code that runs your manufacturing line is indeed "mission critical". Furthermore it's likely that your customers demand that you have strict processes in place for changes, and can actually audit you at their pleasure. So being so bureaucratic about it is not surprising.
The Web 2.0 world works very differently, of course.
To clarify, I'm not suggesting that it's not mission critical, just that I don't believe that that kind of process is actually making things any safer.
"Changing one line of code took 6 days!" is a pretty common WTF story. It comes up every so often, and we all get a chuckle at the ponderous rate of change in large organizations.
However, "Changing one line of code broke everything for 6 days!" is also a common WTF story. As a rule of thumb, you can't avoid both of them, because avoiding one entails the other. I wouldn't fault a company that chooses the first story over the second.
The problem is that things that scale up don't always scale down. The processes implied aren't that bad, for handling large commits to a large process. They're obviously overkill for small changes. But just because the processes are a negative effect on this change in particular does not mean that the processes are a net negative overall. More flexible processes could also end up being more trouble than they're worth, due to allowing (encouraging) sloppy work.
Or you could just hire responsible people and ask them to use their judgement, you don't have to choose one or the other you can instead apply judgement as to whether this is going to screw shit up or not.
Honestly, though the best way to handle this situation is with a phone call to the CEO/IT Director getting him to write you an email allowing you to override all the checks to get this in ASAP.
Once you're known for clearing things with hire ups, QA will start being reasonable. You have to keep in mind that most people in QA are type C personalities and will be a stickler for inane rules.
> Or you could just hire responsible people and ask them to use their judgement
If you know a fail-safe way of only hiring responsible people who always have good judgment... then I hope you're being paid millions of dollars in consulting fees, because nobody has really figured that one out in a general way yet!! :)
Also, those people tend to be expensive... sometimes it's far more cost-efficient to hire less-perfect people, but put rules in place to minimize any catastrophic damage they can create, at the expense of efficiency.
Following the rules to the letter also produces bad outcomes.
The issue is not that using judgement is perfect it's that using judgement usually produces a better outcome than blindly following a rule.
Yes, rules should be put in place for the most extraordinary circumstances or places that good judgement is known to fail. The rule has to prevent the bad thing to be effective while also not creating something worse.
In this case, there was someone with basically good judgement, but they were operating from high within their hierarchical framework, which inherently induces delays. In retrospect, Philip, David, or both, should have been copied on the ticket and directly overseen the policy violation that they deliberately set in motion.
Simply ordering policy violations from the throne doesn't work in practice. Either the person who made the decision or a trusted lieutenant who understands what's going on has to see it through to the end, otherwise the bureaucracy in the middle will simply continue on its preprogrammed course.
For starters, "just hire responsible people" can't be applied retroactively. Not to mention that… most people are responsible, or rather, the road to hell is paved with good intentions.
Secondly, it's an incredibly difficult thing to scale. That works great in startups, but by the time you have things like "HR Directors" you're no longer capable of having insight into the rest of the organization.
There is a lot to be said about how bureaucracies are the formalization of common sense, and a lot more to be said being able to scale processes out such that you don't end up in situations where stakeholders have no visibility into the work performed.
I just feel that the answer is more complicated than "hire A players".
Just because they're smart doesn't mean they don't make mistakes. If you allow one line changes to pass without review, eventually someone's going to make a breaking one-line change.
The code behaves differently after some event happens.
Midnight. Y2K. Daylight Savings Time change. Leap Day. Congress changes when Daylight Savings Time happens and one component in the system didn't know about it. Year 2038 problem.
> As a company, it's great to have a process for incremental changes, but you also need to have a process for critical hot-fixes.
And as soon as you do, everyone thinks that their problem is a good candidate for the critical hot-fix path. At the very least, you now have to sink minutes per day into arguing them down from using it. Worth it? Possibly. But not always.
If you have a single (actually) business critical incident that cannot be resolved, the company will fail. It's called risk management. You spend minutes per day arguing with idiots to not have the company fail in an emergency.
You need a good operations team with the authority and willingness to say "no" with overrides coming only from senior management, the authority to say "yes" for the obvious stuff, and a direct line to senior management for grey areas they're not comfortable making a call on by themselves.
The operations team will spend a few months saying "no" a lot and justifying their decisions to management. Eventually it will slow to a trickle except for a few really stupid people who lack reading comprehension and any sense of pattern detection. Management will eventually get tired of it and forbid the stupid ones from making hot-fix requests.
Indeed. But that doesn't mean that you have to like taking 6 days to write one line of code.
That said, 6 days isn't so bad as long as we're talking about the time it takes to write the code and put it into production. It's another thing to take 6 days to put the code into source control.
Agreed. I think the main problem in this story however is the the red-tape between people and not so much the process itself.
Ask Homer, well Homer is not in so ask Marge, well I don't have access to Marge. Delays like this that come from artificial communication barriers can be easily avoided if they worked on improving communication throughout the organization. It would probably have saved them 1-2 days, if not more.
If you say any change longer than four lines is subject to this process, then people will work on trickling in new stuff in increments of four lines at a time
With proper test automation and continuous deployment, trickling new stuff in increments of four lines at a time is the right way to do it.
And that is how creativity dies. Not all at once with a swift, decisive blow. More like the bites of 10,000 mosquitoes coming 1 at a time, about every 5 minutes so you don't notice that they've stolen a significant amount of your blood until it's too late.
No, this is how creativity is shut out after one too many incidents where she wandered in, decided to be helpful and, due to imperfect understanding how things work, caused massive loss of productivity, money and reputations.
No, creativity is a process to be leveraged and managed like any other. This story is an account of life at a company where the process simply does not support the needs of the business. There are no excuses.
Why aren't you appealing directly to Philip more often? Give him a list of what you need and he could have what he wants with 3 emails. Also I don't understand what "access" you need to Marge? You can't email her or something?
No, don't quit! Embrace! That's 14 hours (if I remember correctly) you could have spent working on your own projects instead of being pissed off on HN. The world is full of this, and it's well accepted and nobody questions it - make the most of it.
I agree with you here. That's a ton of free time. Unfortunately Ed probably posts one disgruntled story each month. I predict he'll post one next month too. He wont quit, he'll just complain on Hacker News.
Good point. When I'm stuck and management takes a couple days to get back to me and I have nothing to do, I take advantage of this time to learn new things. Maybe I'll read the man pages for "find" and discover a few new tricks. Or I'll read some tutorials on vim and learn some new things. I've learned a lot about MongoDB this year while waiting on roadblocks at work.
I understand the advice to quit because of the agonizing process involved, but worse than that for me is the idea that this parameter is hard coded, and even with this change in place that limitation still remains. NO WONDER the change process is so painful, the software design is clearly brain dead. I bet all kinds of unexpected shit happens when changes are rolled out. This is a code smell like a rotting corpse.
Moving parameters somewhere else doesn't change the problem at all, only moves it to a different file. Altering a parameter that effects a large chunk of the system still needs the exact same amount of testing to be done, all you save is a compile.
For what it's worth. I'd also say that anyone who thinks that hardcoded parameters are always bad, probably hasn't worked on anything large or complicated enough yet. There are plenty of cases where you've got parameters which could theoretically change, but the likelihood is small enough that it doesn't make much sense to move it out to somewhere else - not coincidently, these tend also to be parameters which effect large chunks of the system and should never be changed without a solid business reason and a full QA cycle.
Heh, not the same corporate snafu, but my most expensive work was done to change 1 char.
Spent about 3 weeks with a new codebase to find that one number had to be increased. Funny thing is, there was a comment there and somebody had increased it before when they had a similar problem.
I've found that bugs that are very hard to debug often have the smallest fixes. Once you've discovered what tidbit of information you were missing, the fix is trivial. I guess this is a needle/haystack problem.
People don't realize processes, reviews, standards, and audits are there because of painful (costly) incidents and breakdowns in the past. Changing one line of code in the production app is fast and easy. Crashing the system or screwing up the app logic can cause huge disruption to the operation.
After every single crisis, managers got together and vowed to never let this happen again, and thus processes and reviews are born. After couple years, things will get so stifled to the point of what OP was experiencing.
6 days you LUCKY LUCKY Bastard - i have user stories over 6 months old that haven't been actioned yet and had to wait 2 months to get a gwt verification file installed
As good a 'system' as you might have, it is still no match for having a middle manager with a clue (and some freedom to move within the system)...
Sure - this should have still been tested (and thoroughly!), but putting this 'on the queue' when it needs doing now? Making the programmer deal with other issues (rename the variable) while doing it? Not making sure that a testing machine was available? No....
That "legacy code must be brought up to current standards whenever it's touched" policy doesn't seem very wise. I'm all for practical refactoring, but it's a major decision to be treated with care when it comes to legacy code. You don't always want to invite it. I certainly wouldn't want to enforce it.
I mean, it's one thing when your oldest legacy code is two years old, and it's all one program. It's something else entirely when it's, for example, written in Fortran and running on hardware you can't buy anymore. Sometimes you don't want to rock the boat any more than you absolutely have to.
Yes, for an emergency change (change or there will be layoffs), this is a rather dangerous policy. The goal of any emergency fix should be "do as little as possible to solve problem".
I get the feeling that too many people in the chain did not treat this as an emergency. It seems very business as usual. When a company is talking layoffs, it is an emergency.
Yeah, I agree. The first time he told someone "Philip wants this done quickly" and they said, "Sorry, it's policy," the next move -- at least my next move -- would have been to walk to Philip's office and say, "Will you come tell this lady that this change is more important than that policy?"
I mean, they don't have a way to know it's really an emergency. Every engineer thinks everything's an emergency. There's nothing like the guy in charge standing there while they work to communicate THIS IS ACTUALLY AN EMERGENCY.
This is similar to laws that require buildings be brought entirely up to code any time any renovations are done. The intent is to keep buildings up to code, but one of the nasty side effects is that buildings which are already substandard continue to deteriorate because of the massive expense (in time and effort as well as money) that is required. I've seen half a dozen instances with churches and other nonprofits deciding against simple renovations (like adding a wheelchair ramp) because they didn't want to have to redo the entire building's wiring, widen every doorway, and put in an elevator.
If you require a massive overhaul to be tied to a minor fix, one likely outcome is that the minor fix simply doesn't get done.
If you require a massive overhaul to be tied to a minor fix, then you have let the thing rot for too long.
In your example, those buildings that deteriorate will at some point stop being liveable, and lose in value, which will finally mean that the proprietor's failed business approach will become weeded out. I, personally, welcome this sort of development.
"Being out of code" isn't always a result of deterioration or neglect; sometimes it's just a matter of the code changing. Structures built in the 1950s may not have, for example, ADA-compliant accessible restrooms, though the restrooms may be perfectly usable. But putting in something as simple as a wheelchair ramp at the front door might require also remodeling every single restroom in the building to be ADA-compliant.
I've seen half a dozen small religious groups go through this sort of problem -- because they can't afford to make the church/synagogue/etc completely accessible, they aren't able to make incremental improvements. The buildings remain perfectly functional and well maintained and retain all of their value, except in the eyes of the one guy in a wheelchair who can't get a ramp put in because his church can't afford to drop 15 grand on all the other not-very-useful-but-required stuff.
Relating this back to code: if you have a policy that requires any old code must be brought up to new standards any time it's touched, then there are at least some occasions in which someone could have made a useful incremental improvement, but is stopped by a stupid policy decision that would saddle them with burdensome work. The net result is that small, high-value fixes get put off because of large, low-value busy work.
I just went through the same thing. Except I was smart (or so I thought) and had what I needed changed in a config file.
PHB decided that it still counted as a "code" change because someone needed to log onto the box and change the config. Literally it was removing 3 characters from it. The only reason I even had it in the config was to avoid it being a code change to begin with. I argued back that in the time I would take to get through the whole process I could write a screen to do the same thing.
PHB said that would be fine as from that point on its a config not a code change. So off through the whole process, where I present the change to people who don't care, or understand the change and ask for them to approve it. I get that they need awareness, but seriously couldn't a short email saying X is going to happen at time Y suffice?
Im tempted to embed a Python/Ruby interpreter into my next project with a web interface to modify the code. Then I can do these minor changes without going through the 7 day process.
I would argue that in this case where it was designed like that from the beginning IE detailed as how the system works then there should be no issues.
I did forget to mention it did go though a round of testing, which took less then 5 minutes to be verified as correct. It literally was 6 days of process wrapped around 10 minutes work.
This shit happens at my job and leaves me feeling depressed and at times questioning my own skills because I know and my bosses know that on my part, I just had to make a tiny, tiny change. But when you have a dozen such experiences, I feel like something is wrong with me because when the dust settles, this was a one line code change that took me a week to push to production. It also makes my bosses question my estimates. Of course, when I goto the other extreme and give a ridiculous estimate, that also pisses them off and makes them feel like I don't really know what I am talking about. Saying "I cannot provide an estimate" also does not cut it for the boss.
In the longterm, I can feel this killing a part of me. I have always been a start-up guy working for myself. Since getting my first full-time job, stuff that I would expect the start-up me to take an hour to do ends up taking days and weeks, even. I am aware of the (mostly legitimate) excuses to not take too much self-blame but at the end, it still feels shitty because I know, the start-up me knows, this should not have been a one week project.
Ah the perils of having a non-tech boss and coworkers and being the only tech guy. Reading stories like I experience at work is comforting in an odd way.
Here's how my typical story goes...
Boss: So can we make this little analytics thing your #1 priority for tomorrow?
Me: Yeah, as long as we don't have anything else for projectX(my #1 priority)
Boss: Well, do you?
Me: Not right now. But it can pop up any minute Jenny decides she needs something done.
--NEXT DAY--
Jenny: Hey this isn't working in ProjectX. Could you look into it?
Me: Sure. there goes an hour
Boss: I know you're busy but this is very urgent. Can we grab you for few mins for this meeting?
Me: Sure goes another 30 mins
...
Jenny: That thing you fixed. Can we undo it and push to production because I realized blabla?
Me: thinking wtf? Ok sure.
...
Amanda: Hey can you check if these 2 users signed up from same IP?
Me: You can check that in admin
Amanda: Where?
Me: The admin panel you emailed me info to when I joined the company..
Amanda: Oh right. Can you fwd me the email I sent, I can't find it. My gmail's not working right.
Me: Ok.
...
Before I know it, I have spent every day of my job troubleshooting shit to never really get in the flow of doing my own project assigned by my boss.
Boss(after a month): How's that project coming? I know you said it's a one day thing.
Me: Yeah but the next day we changed the specs in the meeting. Also, projectX is getting a huge rehaul.
Boss: But still...you said it's a week after that
Me: Yeah assuming I didn't have work for projectX
Boss: But you'll always have work for projectX
Me: Yeah that's exactly what I told you too
Boss: So where does this leave us? Can you finish it in next week? I don't think there is much work for projectX
Me: But...
Boss: You're the startup guy. We want you to push us to release stuff quick. When you give us deadlines and are so off, it makes us lose faith.
Leave work annoyed, put in a quiet weekend with no coworkers around and wrap up said project
This doesn't sound like a process problem at all, actually. Really it's the reverse: your work environment is a mess, no one has the ability to do their own job except to bug you to do things they need.
And that's a disaster too, but it would be helped by adding process, not hurt.
Also, dude: learn to say no. Learn to prioritize for yourself instead of expecting your slag pile of a management structure to do it for you. i.e. ask: "What things must I get done to make sure I'm successful long term?" (which is entirely different from "What is the highest priority thing I have been asked to do?") and make sure you prioritize your work to get those things done (which is an entirely different thing from "work on those things"). And if your workplace environment really doesn't permit that, move on.
And if your workplace environment really doesn't permit that, move on.
Doing just that :)
It's basically a startup with 10 of us where I am the only in house developer.
I've tried many things like asking folks to Skype me instead of walking over to my desk or calling my name. I've gone to great lengths to get em to understand why skyping to get my attention is different than other ways. None of these guidelines really lasted more than ana few hours.
"It breaks my concentration. That makes things take more time. That makes things cost you more money."
edit: Suggest that he ask you if you're busy over skype. Talking face to face is more effective, and I'd hate to try to discuss something over it, but interrupting flow really sucks.
Because bosses and managers are suppose to support their direct reports by obtaining a work environment and tools that best enable them to be productive... if he refuses to contact you via skype, when he asks you a question verbally, respond by typing a skype message. It's not okay for managers to treat others how they (the manager) wishes or wants to be treated... great managers treat employees the way the employees wish to be treated (as long as that treatment is ethical and productive).
Tim Schafer, of http://www.kickstarter.com/projects/doublefine/double-fine-a... fame, has a note on his door until 11am in the morning saying, don't knock, writing. Could you make progress on projectY if you made a deal to not work on projectX until 11am? To not even open your office door?
(FWIW, this is a classic Quality of Service (QoS) algorithm, where traffic on ProjectY is given dedicated bandwidth (2 hours in the day) so that it isn't swamped by traffic from ProjectX.)
Funny enough I didn't get a private office but we got walls built for a private office when the sales guy joined because he was disturbing everyone. If my non developer coworkers realized how disturbing it is to be in flow writing code to have someone tap your shoulder and start discussing a totally unrelated issue to what you've been coding.
Be a grumpy bastard. Someone taps me on the shoulder, I'm grumpy the first time, I growl the second time and if there is a third time I'll shout at them. Hasn't come to the shouting yet (although one guy got pretty close, but I just glared at him last time he approached and he grasped it).
Also, ask if you can work from home since your office environment is bullshit for writing code.
Work from home was turned down pretty quickly though I'm not sure if that would have addressed the root problem. I think the root issue is they need a developer who just does tech support for the other 10 non tech people. And then a developer who can actually build shit quietly.
If my whole job was to do tech support all day there would be no problem. Of course it wouldn't be fun and I'd prolly end up leaving, the present situation hasn't been much better and I'm on my way out.
Now that I am leaving soon my boss is very interested for advice on how to attract tech talent. A part of me just wants to send him a link to this thread :)
My place of employment is similar to yours, I've moved myself up the ladder in order to start isolating and enabling my dev team the peace and quiet they need to be most productive.
What if, while you code, you think out loud, like really out loud... louder than you would talk in conversation? Perhaps if YOU bothered EVERYONE then maybe they'd put you in a private office? It's worth a try while you are looking for other opportunities...
i.e. "LET ME THINK HERE... IF VARIABLE V EQUALS ENUM ACTIVE THEN CALL DOHICKY ELSE DO NOTHING... WAIT IS THAT RIGHT?! NO NO NO LET ME RETHINK THIS?! I BET IF I REFACTOR THIS CODE I CAN..." All day long...if they tell you to be quiet tell them to "shut up" you're trying to think and they are ruining your thought processes and they risk causing you to slow down and/or enter errors and bugs into the code... "what do you mean me talking and noise in general is disturbing to you and your work? How they hell do you think I feel?!?!?" :)
But if you are planning to leave, what harm will it really do? When you have pocket aces, bet and play them!
EDIT: Maybe you'll inadvertently help the next guy. Yeah they might not care much for you but maybe, just maybe, something will click and the folks will realize... yeah I guess disturbing him was stupid on our part. If not and they don't get it... who gives a crap about people that don't give a crap about you!?
Apparently that passive aggressiveness already exists in his work place thanks to his bosses and co-workers... but some how you all fault him and say he can't share that same love?
I was thinking more about other situations. I thought the salesman was isolated to an office because he has to talk on the phone, constantly, to customers and it was interrupting the developers. I've heard similar situations with startups and their CEOs. If I did that in my open space office, my coworkers would just get irritated and eventually I'll be talked about it by my manager.
unfortunately in environments like the one the OP mentions, the squeaky wheel is usually the one that gets the grease. So, while attached to the trailer (company) you're best to squeak loudly while you look for a better vehicle to get onboard with.
In fact, in my current work environment, when we moved offices there was a "central" open area with a view half built cube walls. Originally our telephone support were going to sit there and the Dev team was going to get placed in an enclosed shared Dev "war room". Support squeaked and the timid dev team got stuck out in the open.
Supposedly since Support is on the phone, they would interrupt everyone (everyone else int he vicinity is sales and they each have private offices) since they talk aloud on the phone. Why's this bad? Because breaking deep thought vs. some noise in the background (which still also exists because of the leakage they have on the phone in a single enclosed room) is much much more unproductive. Worse, the Sales team often forgets to shut their doors so sound leaks out into the open area or between sales calls they congregate a few at a time and have loud conversations disrupting the dev team. Worst of all is if they are in their offices, quietly doing paperwork with their doors open then decide to yell to talk to another salesperson in another office.
I often get up and shut Sales office doors, and constantly have to hush people up. My timid dev team doesn't because, well they are timid and don't want to rock the boat and add friction to inter office relationships. (I can hear, I agree. I need to encourage my dev team to speak up more, perhaps I'm sheltering them too much.)
I sometimes am forced to do this at work. Just to get people to shut up from chit-chatting/small-talking very loudly. But then, I work in a cost-cutting, off-shored, cubicle farm.
I'll frequently work from home in the morning or evening if I want to get stuff done; no coworkers there. Or if I need to concentrate, I'll leave my office and go find a massage chair or cozy nook and curl up with my laptop, or even go to a different building and work alongside a team where nobody knows who I am.
On this subject: Whenever someone chats me in the following manner, I want to kill them. Straight up. Just fucking take an ice pick and jab it right through their skull:
Other Dude: Hi.
Me: Hello
A minute elapses
Other Dude: I'm having a problem
Me: How can I help
A minute elapses
Other Dude: ACTUAL PROBLEM THAT THEY NEEDED HELP WITH
Me: Oh, here is your resolution which took all of 15 seconds to give to you.
WHAT THE FUCK MAN?! WHAT THE FLYING FUCKITY FUCK?! YOU JUST BROKE MY FLOW AND STOLE 2 GOD DAMNED MINUTES! WHAT IS WRONG WITH YOU?! IT'S FUCKING IM, YOU'RE NOT GOD DAMNED STANDING NEXT TO ME! AND WHY THE HELL DOES IT TAKE YOU A MINUTE TO TYPE EACH THING ANYWAY?! DON'T YOU KNOW HOW TO TOUCH TYPE?!
I mean, CHRIST, if this happened once in a blue moon, fine, whatever, but it seems like EVERY motherfucker uses IM the same way. EAT DICKS.
That is why I insist on communication via chat clients when I'm busy. I'm not immediately interrupted as I would be if someone walks up or shouts out as I can ignore the little flash of the chat window on the task tray for a few minutes, sometimes tens of minutes, while I bring my train of thought to a more orderly stop.
If people start with "Hi, I have a problem" and don't state what the problem is I just respond with "drop me the details and I'll have a quick think" and get back to my work. Even if they respond immediately to that it might take me ten or fifteen minutes to get look at their message again - that is their time wasted, not mine.
Some problems require a more urgent response so I'll accept a walk-up or a shout for those, and some problems are best illustrated by showing them than describing them, I just have to trust people to know what they are and what is less urgent. And of course a message will come in at a time when I am eminently interruptible, in which case people get an immediate response anyway.
I find it works except for people who insist on not following the process, but they soon learn I'm far more helpful if they don't irritate me!
> If people start with "Hi, I have a problem" and don't state what the problem is I just respond with "drop me the details and I'll have a quick think" and get back to my work.
If people start with "Hi, I have a problem" and don't state what the problem is I close the chat window.
My work IM's status clearly states "if you have a question, ask it, don't ask if you can ask", time wasters simply get ignored.
> Some problems require a more urgent response so I'll accept a walk-up or a shout for those
> If people start with "Hi, I have a problem" and don't state what the problem is I close the chat window.
In the past I've found that leads to people reporting "I asked him several times over a few hours and got no response" which, if the person being complained to does not know the situation, could look bad for me.
If I feel like being more terse, I just give a one-word response: "Details?".
> In the past I've found that leads to people reporting "I asked him several times over a few hours and got no response" which, if the person being complained to does not know the situation, could look bad for me.
As I noted, my conditions are clearly spelled out as my IM status text, if they can't bother following simple instructions I can't bother with them.
I do this all the time and add, "I like this game. Your turn." It normally gets a smile (at least in person), but doesn't really help with the flow problem.
First of all, you're not bursting into someone's office. Second of all, if you burst into my office while I'm working and start making smalltalk, I will, depending on my mood and whether or not I've previously judged you to be a useless person, either immediately ask you to get to the point, or throw you out.
Stop wasting my time. If sending "Hi" makes you feel better, fine, but the substance had better be on my screen by the time I look at the window, or you've wasted my time.
The same way you know he's not at his desk when you type that useless "Hi": you don't get a reply. How long do you wait after typing a useless "Hi"? Wait that same amount of time after typing your relevant question.
Have you heard of these things called sentences? You don't need to hit return between each one.
"Hi Dude, I've got this thing I need help with. Here is the description of what it is and what I am having a problem with."
It's not an issue if they put it in the same chat message, it's the "Hi", send message, "I have a problem", send message, "This is what the problem is", send message that drives me up a wall.
And if you REALLY need him to get back to let you know if he can help out immediately, you can send a second message along the lines of, "And sorry to be a bother, but I need help pronto. Please let me know if you can take a look at this right now." At that point, it's not a big deal if there's another message, because you haven't wasted time in relaying it.
This is what I'm guarding against. I haven't been a programmer long, but it seems inhuman to not try and be polite or have a tiny lead up to the question.
Ironically, it's the exact same thing that makes me impolite.
> On this subject: Whenever someone chats me in the following manner, I want to kill them.
Then refuse this, set your status to "don't bug me, if you have a question just bloody ask it" and check your IM when you've got a lull[0]. If it's not a question, kill the window it's not relevant. I don't need "social interaction" on IM when I'm coding, if the other guy can't get to the point screw him, I'm not interested.
[0] If IM is specifically a way to get you immediately, start blacklisting people who can't get to the point and waste your time and have they go through email.
>If IM is specifically a way to get you immediately, start blacklisting people who can't get to the point and waste your time and have they go through email.
But of course they won't use email, they need the answer NOW. No, they'll come over to your desk and start tugging on your shirtsleeve.
I much prefer for people to visit, and to visit people. But it requires being able to take 'no' for an answer, and being able to say 'no'. Or at least, "I'm in the middle of something else; I can help you in 10 minutes. Otherwise, talk to Joe."
Hm, I've just started to think, also for my personal use, that maybe it wouldn't be a bad idea to try scripting some automatic 'Hello, how can I help you?' response to /^[^\w](hi|hello)[^\w]($MY_NAME)?[^\w]*$/i :)
YOU JUST BROKE MY FLOW AND STOLE 2 GOD DAMNED MINUTES!
Actually, it's worse than that. The (average) time it takes to get back into flow is 15 minutes[1], so add the time to respond to them to that to get your wasted time. And if you get another interruption before the fifteen minute descent into flow is complete? Start over - another fifteen minutes. If this happens constantly, you will effectively never get any work done. There's more than one way to kill productivity.
It can piss people off and so work negatively (so, evaluate your individual circumstances), but you can begin insisting upon written requests. Amongst other things, this documents the requests. (And you can tell people the written request is e.g. for your email-based "todo" list.)
If/when people start getting into your face (perhaps even formally) about performance, you have documentation to back up your argument. (Although, ultimately, most arguments are kind of moot, unless they end up in court or perhaps before an unemployment insurance evaluation -- not something you really want, anyway).
As for the overall experience. While negative, emotionally, you can still view it as a learning experience. Keep in mind that when you are in a better position, with more control, 90 % of the world with which you are competing is / will be what you are now experiencing. So... if you can cut out that bullsh-t, you will have a very significant competitive advantage.
It can be useful, to get a good, hard, close-up look at what you don't want. Just don't let it go on too long.
P.S. A trick I'll occasionally use, in problematic verbal situations, is to summarize the conversation/request in an email to the other party(s). 'We met. Here's what you asked me to do / how I understand it, what I'm going to do, and my anticipated schedule and potential conflicts / other overriding priorities. Let me know if I've misunderstood or you disagree.'
If no one responds, I proceed accordingly. If they bitch, hey, I laid out my approach and circumstances up front and asked for feedback.
It can put people on the defensive and create a negative vibe. But hey, when people tell you / manipulate you with statements of "We're losing confidence", things are not too positive, to start with. (P.S. That's already the sign/signal to bail. "Bail, bail, bail!")
> Learn to prioritize for yourself instead of expecting your slag pile of a management structure to do it for you
What you are suggesting is easier said than done. I think that most good developers have the mentality you are describing...at first. When you work in an environment like this, however, you get worn down. You can only put up a fight for so long before you start getting all sorts of mixed up and you don't know what the f to do. If you took the best prioritizer in the world and put them in a corrosive environment like this one, the environment would eventually win out. I really feel for guys like the one that posted this original article and some of the subsequent commenters. I have been in that boat before and it is not fun.
Interesting; I've encountered "yak shaving" as meaning something slightly different: it's when your main task depends on a subordinate task which is long, difficult, and stupid--e.g., in programming, it's something which the tools should be doing for you. (Bonus points if you know that, on a different OS, or in a different language, the tools would be doing it for you.)
The original MIT meaning is closer to what I recently heard described as meta-yak-shaving, which is when, before you can start shaving the yak, you discover that each hair of the yak contains a subyak, and each subyak has to be shaved individually.
I see the familiar pattern of programmers not saying "no" and not explaining clearly what they need to work.
None of the other people here are being particularly unreasonable, they're just ignorant. And given that "how to work with programmers, writers and mathematicians" is not part of any educations, it's not really their fault.
Try something like, "Can I get back to you in an hour? Boss wants me to finish this top priority item for him."
And if Boss asks you if you can finish it next week, say no. You're being asked to give an estimate, and it doesn't help to say "Yes, if [this thing doesn't happen which I know it will]". If they want you to be the startup guy, then act like you're the one in charge of your own schedule.
Then you either demand dedicated "anti-office hours" or you tell them that it's not feasible that project B will get done with the way things have been recently. They're not technical, and all they can do is take your word on when you think you can finish it. They're not going to pick up on the caveats you've laced in your estimate. Also, you should let them know ahead of time that things are slipping instead of them finding it "behind schedule" on their own.
Honestly, don't take this as an insult, but it just doesn't sound like you're the right person for the position. If you have trouble pushing back when you need to, then you should find a spot where there is someone pushing back for you.
I agree with you that I'm not the right person. I was brought in to change the culture so we do everything faster(you can imagine the speed at a company with couple offshore devs and half dozen onshore people working on product).
I've, err we, have mostly failed.I can go on about why but at the end, I think we all share the blame and evaluate parts we could have personally done better and apply it next time round.
One of the big realizations for me is that they need someone older and more confident. I'm in my mid 20s and once founded a startup that got 30x the traffic but even months in, the boss would be unsure if I was a good programmer. We tried everything from emailing my code samples to his CTO friends to code reviews by other devs(all of which gave positive feedback) but I feel he could never totally shake that doubt. I called him out on it finally(basically if you can't tell after 8 months if I can program you should let me go). I've been programming since 12 but the lingering doubts actually started making me question my own abilities at which point I decided I need to move.
One day I had an epiphany. I realized as I was programming at my computer that nobody had a freaking clue what I was actually working on. If someone asks how their low-priority project is going, I just say that I'm still working on it. They have no way of knowing if I'm actually working on it right now. In the real world, saying "no" a lot will get you enemies. It's much more diplomatic to only work on some low-priority project an hour or two a day (you don't have to tell them your are only spending an hour or two a day), or say you'll get to their project immediately after you finish this thing your are working on (which can be a small component of your large important project). It's important to know how to say "no", but in practice it is important to minimize the number of times you say "no".
A better trick I discovered when I used to work in an environment closer to this was to always give the person requesting help some trivial but sincerely related task to do before you can help them, e.g. "can you just write down all the steps in the process before you get stuck, it will help me solve the problem", or "Hmm... I'm not quit getting the issue, can you draw me a diagram and then get back to me". This acts as a great filter for people that are just helpless versus people that really need help. And if anyone complains to the boss that you aren't helping, you can say "oh well did you do task X? that's really all I need from you to get started"
I found keeping a timesheet helped me there. When it came to "it is getting to the point where X is not going to be ready on time, and I've got personal things on this week so I really can't work significant overtime", and the question came back "well, what have you been doing for the last week", I could give them a full list of "2 hours helping X with Y", "an hour doing Z", "four hours on that high priority fix client C that no one else was willing to touch" and so forth. Don't play the blame game and point out that person X was being thick on an occasion or otherwise dwell on who is interrupting your key work, just lay it out as matter-of-fact as you can: "This is where my time is being used".
Make damn sure though that people understand a short interruption "for a quick meeting" when you are at the time "powering through" stuff sometimes kills as much as a full hour of development time if you are working on something complex as it can completely break your train of thought (especially if you are tired due to recent overtime!), or worse if you are coordinating with other people and/or are working on something timing sensitive (so the interruption might mean a key coordination point is delayed).
If you can show what the demands on your time are, a good manager will try rearrange things so that you are interrupted less often. Sometimes simply merging those small interruptions into a scheduled period or two each day is enough to make all the difference to your mid/long term work: problems that didn't really need you magically go away (as the person who was going to interrupt you does that 30 seconds of extra thinking/research that was needed to give them the clue) and the others "interrupt" you a few times a day rather than several times over an hours or two.
Here's my suggestion: have your boss talk to Jenn and make her understand that you'll be spending the next day working on the boss' request. Any interruptions would have to go through the Boss, so that all parties are aligned.
This seems like a communication problem between the managers. (I can speak from experience, since I messed this up once when I reallocated our engineers' time without fully being on the same page as the engineering manager)
Ideally that should work but projectX is what the whole company is working on and key pieces of it are being tested throughout the day so it's impractical to have a cut off time.
Well the converse of my suggestion adapted to your particular situation would be this:
Boss: Could you do this thing for me?
Badclient: Well, as you know, I have Project X as my highest priority. I'm happy to oblige, but Jenn has dibs on my time at the moment. If you can talk to her and come to an agreement on how I should distribute my time, I'll be happy to work on your thing.
Whatever happens, your boss is on the same page as Jenn, and all you have to do is execute the agreed upon plan.
ps: Now that I am leaving soon my boss is very interested for advice on how to attract tech talent. A part of me just wants to send him a link to this thread :)
I mainly stuck around because it was my first FT job and I'd promised myself I won't bail on it for one year no matter how bad. In past I've regretted quitting on projects too quickly...at least I won't have that regret here.
1. Never tell anyone how long things take until you got a chance to sit down and make an estimate vs. guesstimate.
2. Never let anyone get you in the position to set priorities yourself (unless you're the boss). Refer them to your manager or whoever is highest in the chain of command and assigned you a task to make this decision.
This is really more a working culture problem. You'd think it's in the management's best interested to make sure nobody ever violates 1. + 2.
Just out of curiosity, and this is something I've been meaning to ask for a while, how does the whole startup only having one tech guy (who isn't a founder) work? I've seen this scenario referenced many times on HN and I'm still having trouble picturing what the incentives are to work for such a company (as opposed to being made a cofounder) when it sounds to me like you're doing at least as much work as they are. I would normally think the sum total of all development (I'm lumping things like specs, coding, documenting, and testing all together) to be at the very least close to 50% of a startup's value, so if there is only one developer, he or she would seem to be worth about half the company. Is this kind of startup less software oriented than the kind I'm describing?
One backlog for new requirements, another backlog for incidents. Track your time working on the different backlogs (or even per-tasks) and pretty soon you will have some numbers to show the boss why new stuff isn't happening snap-snap.
As much as this sounds counterintuitive, say no, and stop being a team player.
Petty requests for emails that someone should have are their responsibility. If you think about it, why should your resourcefulness help them complete their job while it blocks your own progress?
Your boss is doling out reviews, he's #1 priority. Everyone else gets "no" and no access to your time without talking to him first.
"Being a team player" is a fluffy way of saying "do other peoples jobs for them". Sure, help people out and don't be a dick.
It does sound like a mess, but this type of mess exists everywhere. When you meld highly technical people with less technical people you will get these scenarios. Before you know it, you will be expected to replace the toner.
One of the main reasons I took a FT job was to become a better team player so in the long scheme of things, I always wanted to make this work out. Obviously it hasn't worked out too well but I think I have learned some critical lessons for my next gig!
At my previous company we had the same problem, where people from all over the company would come up to us developers constantly asking us to do things which were great, but not really what we were supposed to be doing. Like you, we never got around to doing something that we said would only take a day.
One week we all decided to keep a notepad and paper next to our desks and keep a log of every time we were interrupted and asked to context switch and do something random. It was so bad, before the end of Monday our boss was so dumbfounded he completely re-jigged everything so it wouldn't happen like that again.
We ended up calling one developer pair the "consulting developers" each week, and only that pair could be interrupted. There was a huge sign above their desk advertising them for the week. We sacrificed one for the good of all, and it worked really, really well. AFAIK, they are still doing that.
This is great to hear. It is pretty much the problem/solution that I've diagnosed and stated in another post:
I think the root issue is they need a developer who just does tech support for the other 10 non tech people. And then a developer who can actually build shit quietly.
I will push this stance more aggressively having read your post.
This is something the Scrum methodology is supposed to solve. It places the responsibility of prioritizing tasks at the hands of the stakeholders by making the tradeoffs between scheduled tasks and interruptions very explicit.
If you want something more light weight look into the Pomodoro technique. People can bring issues to you but only in breaks between blocks of uninterupted work.
If you go with either then you can explain away refusing to be distracted as merely following a proven methodology. This gives you an easy out as people will vent any frustration at it rather than you.
If you want a change request processed quickly, you should get someone in upper management to help you herd it through the system. Literally, take your VIP with you, and walk up to the desk of each admin in turn. Say, "Don't wait for Tuesday's meeting--approve this right now." And then walk to the next admin and do it again. And then walk to the test guys and say, "Drop everything and test this now." And then gather up the six essential stakeholders for the CCB meeting and say, "Impromptu CCB meeting, right now in the hallway instead of next month. Approve this."
I've gotten software changes that usually take two months to trickle through the system through in a day. You have to spend all day doing it, though, and so does your VIP.
That works once. Then the next time you go to each desk, the person there has already checked with their boss, and says "my boss says you need to go through the correct channels and if you don't like it, you need to talk to him."
Wishful thinking. Each of those people has a thousand other things to do than to hold your hand all day long. Only works for things that are super-duper-important.
Additionally.. Going to "tell" on your buddies all the time and holding "the grownups" over them is terrible whinery. Everyone on the playground hates kids who do that.
The real WTF is that at the end, all it takes is the president giving a direct order to put the code in production immediately. That is, all the policies and processes just add noise to the interface between strategic decisions and operational tasks. That's how big companies fail.
I find it troubling that they had to make a change at the point of layoffs to create a solution to prevent it. Couldn't they have discovered this trend before it got to that point?
236 comments
[ 421 ms ] story [ 1005 ms ] threadMight be overkill for one line of code, but where do you set the threshold? If you say any change longer than four lines is subject to this process, then people will work on trickling in new stuff in increments of four lines at a time.
Also, I can do a lot of damage in one line.
Especially if it takes a week to revert it. You reap what you sow.
All of these items in the story add overhead and reduce agility, but potentially do very little to improve quality:
This kind of stuff is a constant and substantial overhead on getting anything done. You need to look at these processes very critically and be very sure of the benefits in terms of quality before introducing it.Therefore, process like this unnecessarily slows down the organisation 95% of the time.
Much better to trust people and implement very specific and very carefully designed safety nets such as fast rollbacks.
In most cases, the benefits of just trusting people will massively outway the costs of the occasional slip up.
If you let too many simple issues through, you're likely to find yourself completely failing when a very simple thing breaks. First you'll find that your logging is not correct, so you'll need to fix that first and reproduce, then you'll find that the rollback doesn't quite work and you've got some bad data to fix manually, then you'll find that this is actually a simple error masking some really nasty bug, etc. etc. I've seen that once or twice and I really believe in the broken windows theory now. I'd be more glad with a reasonable slowdown, than people pushing ahead instead of stopping to think about long-term issues.
Yes, I'm the guy who rejects ~4 out of 5 changes during code review initially (on average). Then again, I'm the guy who gets woken up when things fail - not everyone does, but it really gives you the appreciation of why you want to test those exception branches. I hope that people will be as strict about stuff I submit too.
But the assumption that's wrong is that all processes avoid mistakes. Clearly they don't.
And some of the examples here are just plain cargo cult misapplications of good ideas. The point behind code review is to catch design flaws in new code. You never want to demand refactoring of existing code in order to patch features, that hurts, it doesn't help.
Having just (as in hours ago) been through the wringer of adding new features to a mass of spaghetti that had not been touched in 10 years, the only way I could maintain my sanity (and have some assurance beyond regression testing) that I wasn't breaking something, was to refactor the existing code and then insert my changes.
Skilled people without a process will always find a way to get things done. Skill begets process. But process doesn’t beget skill. Following a recipe won’t make you a great chef – it just means you can make a competent bolognese. Great chefs don’t need cookery books. They know their medium and their ingredients so well that they can find excellent combinations as they go. The recipe becomes a natural by-product of their work.
From http://the-pastry-box-project.net/cennydd-bowles/2012-march-...
Besides, skill may beget product, but it doesn't necessarily beget process.
Oh boy, talk about not seeing the forest for the trees. For skill to beget a successful product, you must follow a process to get from nothing to product. So in the course of creating a successful product you have automatically created a process -- the process to build the product. That process may be used one time, or multiple times -- but it's still a process.
1) The fields are probably mandatory because so many times routine entries were not filled correctly, and even MORE overhead was required going back and forth between teams to find out all of the details. If there is a way to avoid this overhead, again, how do you stop people abusing it?
2 & 3) Yeah.. bad overhead.
4) A sanity check is ALWAYS a good idea, whether its a fresh intern or the most seasoned programming in the company making a change.
5) They security processes might seem excessive but once again are probably in place due to issues in the past that have required the review/signoff be there to stop major disruptions getting through.
6) As someone who worked in QA for a while, there is no way in a hell someone can say "push this change through" without a strong questioning of why. QA is there (in most cases) to be that last line of defence. Often letting people know just how many people are really going to be affected by this change.
Being on a testing team often gives you a broader oversight of how all of the components of the code work together, while people working on one project can sometimes get tunnel vision on getting their thing out the door.
Edit: Just like to point out, I'm agreeing with you
QA should absolutely be able to push back on Dev with regards to quality and relevant test cases, but likewise Dev often need to be able to direct the testing and mutually agree on the scope that will get the change signed off adequately.
I have to take issue with this one. Code (and Requirements, and Design) reviews, when done properly probably have the greatest impact in improving product quality. No, I don't have time to look up the SEI research on the subject, but it has been consistently shown to be true.
Part of the problem I had with the article was that Shirley was rejecting code for non-specific reasons. You can't say "it's not written down anywhere." If it's important enough to reject the code, it's important enough to write down.
However, I don't see as much value in reviewing low level changes such as one liners or odd bug fixes.
Making code reviews mandatory is too much overhead and too inflexible for me.
Here are some notes about Firefox's code review process:
https://wiki.mozilla.org/Firefox/Code_Review
https://wiki.mozilla.org/User:GavinSharp/Code_Review
I would add that crappy processes and tools are usually in place because the management/leadership in an organization has made them too brittle. While you don't want to wild-west every "critical" issue, bad spots in a process will naturally highlight themselves over time.
Let your tools and processes constantly evolve instead of blindly subscribing to a methodology, and you'll be in much better shape.
The Web 2.0 world works very differently, of course.
However, "Changing one line of code broke everything for 6 days!" is also a common WTF story. As a rule of thumb, you can't avoid both of them, because avoiding one entails the other. I wouldn't fault a company that chooses the first story over the second.
The problem is that things that scale up don't always scale down. The processes implied aren't that bad, for handling large commits to a large process. They're obviously overkill for small changes. But just because the processes are a negative effect on this change in particular does not mean that the processes are a net negative overall. More flexible processes could also end up being more trouble than they're worth, due to allowing (encouraging) sloppy work.
Honestly, though the best way to handle this situation is with a phone call to the CEO/IT Director getting him to write you an email allowing you to override all the checks to get this in ASAP.
Once you're known for clearing things with hire ups, QA will start being reasonable. You have to keep in mind that most people in QA are type C personalities and will be a stickler for inane rules.
http://www.buzzle.com/articles/type-c-personality.html
If you know a fail-safe way of only hiring responsible people who always have good judgment... then I hope you're being paid millions of dollars in consulting fees, because nobody has really figured that one out in a general way yet!! :)
Also, those people tend to be expensive... sometimes it's far more cost-efficient to hire less-perfect people, but put rules in place to minimize any catastrophic damage they can create, at the expense of efficiency.
The issue is not that using judgement is perfect it's that using judgement usually produces a better outcome than blindly following a rule.
Yes, rules should be put in place for the most extraordinary circumstances or places that good judgement is known to fail. The rule has to prevent the bad thing to be effective while also not creating something worse.
Simply ordering policy violations from the throne doesn't work in practice. Either the person who made the decision or a trusted lieutenant who understands what's going on has to see it through to the end, otherwise the bureaucracy in the middle will simply continue on its preprogrammed course.
For starters, "just hire responsible people" can't be applied retroactively. Not to mention that… most people are responsible, or rather, the road to hell is paved with good intentions.
Secondly, it's an incredibly difficult thing to scale. That works great in startups, but by the time you have things like "HR Directors" you're no longer capable of having insight into the rest of the organization.
There is a lot to be said about how bureaucracies are the formalization of common sense, and a lot more to be said being able to scale processes out such that you don't end up in situations where stakeholders have no visibility into the work performed.
I just feel that the answer is more complicated than "hire A players".
Or, just as easily, you could sprinkle magic fairy dust over your servers to solve provlems.
> It [sic] we don't do this right away, we'll have to have a layoff.
As a company, it's great to have a process for incremental changes, but you also need to have a process for critical hot-fixes.
You're a phone company, and adding a new feature should take months. I get that.
You're a phone company, and because of an unforeseen Cinderella problem, 10% of your customers can't make phone calls. At all.
It is unacceptable to have a one-size-fits-all process, no matter what your business is.
Midnight. Y2K. Daylight Savings Time change. Leap Day. Congress changes when Daylight Savings Time happens and one component in the system didn't know about it. Year 2038 problem.
Mayor: What do you mean, "biblical"?
Dr Ray Stantz: What he means is Old Testament, Mr. Mayor, real wrath of God type stuff.
Dr. Peter Venkman: Exactly.
Dr Ray Stantz: Fire and brimstone coming down from the skies! Rivers and seas boiling!
Dr. Egon Spengler: Forty years of darkness! Earthquakes, volcanoes...
Winston Zeddemore: The dead rising from the grave!
Dr. Peter Venkman: Human sacrifice, dogs and cats living together... mass hysteria!
Mayor: All right, all right! I get the point!
And as soon as you do, everyone thinks that their problem is a good candidate for the critical hot-fix path. At the very least, you now have to sink minutes per day into arguing them down from using it. Worth it? Possibly. But not always.
The operations team will spend a few months saying "no" a lot and justifying their decisions to management. Eventually it will slow to a trickle except for a few really stupid people who lack reading comprehension and any sense of pattern detection. Management will eventually get tired of it and forbid the stupid ones from making hot-fix requests.
Step One is "Act Intelligently."
No process is going to help you skip Step One.
That said, 6 days isn't so bad as long as we're talking about the time it takes to write the code and put it into production. It's another thing to take 6 days to put the code into source control.
Ask Homer, well Homer is not in so ask Marge, well I don't have access to Marge. Delays like this that come from artificial communication barriers can be easily avoided if they worked on improving communication throughout the organization. It would probably have saved them 1-2 days, if not more.
In this case process overrode rationale; and was cheered as a result on its own.
With proper test automation and continuous deployment, trickling new stuff in increments of four lines at a time is the right way to do it.
maintenance programmer words to live by.
edit: d'oh
For what it's worth. I'd also say that anyone who thinks that hardcoded parameters are always bad, probably hasn't worked on anything large or complicated enough yet. There are plenty of cases where you've got parameters which could theoretically change, but the likelihood is small enough that it doesn't make much sense to move it out to somewhere else - not coincidently, these tend also to be parameters which effect large chunks of the system and should never be changed without a solid business reason and a full QA cycle.
Spent about 3 weeks with a new codebase to find that one number had to be increased. Funny thing is, there was a comment there and somebody had increased it before when they had a similar problem.
Time to find out what charater to change: 3 weeks
After every single crisis, managers got together and vowed to never let this happen again, and thus processes and reviews are born. After couple years, things will get so stifled to the point of what OP was experiencing.
That memory is hard to maintain with procedural changes in modern times, as employees move around quite frequently.
Sure - this should have still been tested (and thoroughly!), but putting this 'on the queue' when it needs doing now? Making the programmer deal with other issues (rename the variable) while doing it? Not making sure that a testing machine was available? No....
I mean, it's one thing when your oldest legacy code is two years old, and it's all one program. It's something else entirely when it's, for example, written in Fortran and running on hardware you can't buy anymore. Sometimes you don't want to rock the boat any more than you absolutely have to.
If you make one exception, then a month later, you discover that nobody is ever bringing old code up-to-date ever.
I get the feeling that too many people in the chain did not treat this as an emergency. It seems very business as usual. When a company is talking layoffs, it is an emergency.
I mean, they don't have a way to know it's really an emergency. Every engineer thinks everything's an emergency. There's nothing like the guy in charge standing there while they work to communicate THIS IS ACTUALLY AN EMERGENCY.
If you require a massive overhaul to be tied to a minor fix, one likely outcome is that the minor fix simply doesn't get done.
In your example, those buildings that deteriorate will at some point stop being liveable, and lose in value, which will finally mean that the proprietor's failed business approach will become weeded out. I, personally, welcome this sort of development.
I've seen half a dozen small religious groups go through this sort of problem -- because they can't afford to make the church/synagogue/etc completely accessible, they aren't able to make incremental improvements. The buildings remain perfectly functional and well maintained and retain all of their value, except in the eyes of the one guy in a wheelchair who can't get a ramp put in because his church can't afford to drop 15 grand on all the other not-very-useful-but-required stuff.
Relating this back to code: if you have a policy that requires any old code must be brought up to new standards any time it's touched, then there are at least some occasions in which someone could have made a useful incremental improvement, but is stopped by a stupid policy decision that would saddle them with burdensome work. The net result is that small, high-value fixes get put off because of large, low-value busy work.
PHB decided that it still counted as a "code" change because someone needed to log onto the box and change the config. Literally it was removing 3 characters from it. The only reason I even had it in the config was to avoid it being a code change to begin with. I argued back that in the time I would take to get through the whole process I could write a screen to do the same thing.
PHB said that would be fine as from that point on its a config not a code change. So off through the whole process, where I present the change to people who don't care, or understand the change and ask for them to approve it. I get that they need awareness, but seriously couldn't a short email saying X is going to happen at time Y suffice?
Im tempted to embed a Python/Ruby interpreter into my next project with a web interface to modify the code. Then I can do these minor changes without going through the 7 day process.
I did forget to mention it did go though a round of testing, which took less then 5 minutes to be verified as correct. It literally was 6 days of process wrapped around 10 minutes work.
In the longterm, I can feel this killing a part of me. I have always been a start-up guy working for myself. Since getting my first full-time job, stuff that I would expect the start-up me to take an hour to do ends up taking days and weeks, even. I am aware of the (mostly legitimate) excuses to not take too much self-blame but at the end, it still feels shitty because I know, the start-up me knows, this should not have been a one week project.
Ah the perils of having a non-tech boss and coworkers and being the only tech guy. Reading stories like I experience at work is comforting in an odd way.
Here's how my typical story goes...
Boss: So can we make this little analytics thing your #1 priority for tomorrow?
Me: Yeah, as long as we don't have anything else for projectX(my #1 priority)
Boss: Well, do you?
Me: Not right now. But it can pop up any minute Jenny decides she needs something done.
--NEXT DAY--
Jenny: Hey this isn't working in ProjectX. Could you look into it?
Me: Sure. there goes an hour
Boss: I know you're busy but this is very urgent. Can we grab you for few mins for this meeting?
Me: Sure goes another 30 mins
...
Jenny: That thing you fixed. Can we undo it and push to production because I realized blabla?
Me: thinking wtf? Ok sure.
...
Amanda: Hey can you check if these 2 users signed up from same IP?
Me: You can check that in admin
Amanda: Where?
Me: The admin panel you emailed me info to when I joined the company..
Amanda: Oh right. Can you fwd me the email I sent, I can't find it. My gmail's not working right.
Me: Ok.
...
Before I know it, I have spent every day of my job troubleshooting shit to never really get in the flow of doing my own project assigned by my boss.
Boss(after a month): How's that project coming? I know you said it's a one day thing.
Me: Yeah but the next day we changed the specs in the meeting. Also, projectX is getting a huge rehaul.
Boss: But still...you said it's a week after that
Me: Yeah assuming I didn't have work for projectX
Boss: But you'll always have work for projectX
Me: Yeah that's exactly what I told you too
Boss: So where does this leave us? Can you finish it in next week? I don't think there is much work for projectX
Me: But...
Boss: You're the startup guy. We want you to push us to release stuff quick. When you give us deadlines and are so off, it makes us lose faith.
Leave work annoyed, put in a quiet weekend with no coworkers around and wrap up said project
And that's a disaster too, but it would be helped by adding process, not hurt.
Also, dude: learn to say no. Learn to prioritize for yourself instead of expecting your slag pile of a management structure to do it for you. i.e. ask: "What things must I get done to make sure I'm successful long term?" (which is entirely different from "What is the highest priority thing I have been asked to do?") and make sure you prioritize your work to get those things done (which is an entirely different thing from "work on those things"). And if your workplace environment really doesn't permit that, move on.
Doing just that :)
It's basically a startup with 10 of us where I am the only in house developer.
I've tried many things like asking folks to Skype me instead of walking over to my desk or calling my name. I've gone to great lengths to get em to understand why skyping to get my attention is different than other ways. None of these guidelines really lasted more than ana few hours.
People do what works. Reinforce their bad behavior, and you have a problem.
edit: Suggest that he ask you if you're busy over skype. Talking face to face is more effective, and I'd hate to try to discuss something over it, but interrupting flow really sucks.
(FWIW, this is a classic Quality of Service (QoS) algorithm, where traffic on ProjectY is given dedicated bandwidth (2 hours in the day) so that it isn't swamped by traffic from ProjectX.)
Also, ask if you can work from home since your office environment is bullshit for writing code.
If my whole job was to do tech support all day there would be no problem. Of course it wouldn't be fun and I'd prolly end up leaving, the present situation hasn't been much better and I'm on my way out.
Now that I am leaving soon my boss is very interested for advice on how to attract tech talent. A part of me just wants to send him a link to this thread :)
That's probably the single best thing you could do for him, if he is serious about wanting to attract (and retain!) good technology people.
Been there, done that. The only difference is that they call me on the phone instead of coming up to my desk.
What if, while you code, you think out loud, like really out loud... louder than you would talk in conversation? Perhaps if YOU bothered EVERYONE then maybe they'd put you in a private office? It's worth a try while you are looking for other opportunities...
i.e. "LET ME THINK HERE... IF VARIABLE V EQUALS ENUM ACTIVE THEN CALL DOHICKY ELSE DO NOTHING... WAIT IS THAT RIGHT?! NO NO NO LET ME RETHINK THIS?! I BET IF I REFACTOR THIS CODE I CAN..." All day long...if they tell you to be quiet tell them to "shut up" you're trying to think and they are ruining your thought processes and they risk causing you to slow down and/or enter errors and bugs into the code... "what do you mean me talking and noise in general is disturbing to you and your work? How they hell do you think I feel?!?!?" :)
EDIT: Maybe you'll inadvertently help the next guy. Yeah they might not care much for you but maybe, just maybe, something will click and the folks will realize... yeah I guess disturbing him was stupid on our part. If not and they don't get it... who gives a crap about people that don't give a crap about you!?
Supposedly since Support is on the phone, they would interrupt everyone (everyone else int he vicinity is sales and they each have private offices) since they talk aloud on the phone. Why's this bad? Because breaking deep thought vs. some noise in the background (which still also exists because of the leakage they have on the phone in a single enclosed room) is much much more unproductive. Worse, the Sales team often forgets to shut their doors so sound leaks out into the open area or between sales calls they congregate a few at a time and have loud conversations disrupting the dev team. Worst of all is if they are in their offices, quietly doing paperwork with their doors open then decide to yell to talk to another salesperson in another office.
I often get up and shut Sales office doors, and constantly have to hush people up. My timid dev team doesn't because, well they are timid and don't want to rock the boat and add friction to inter office relationships. (I can hear, I agree. I need to encourage my dev team to speak up more, perhaps I'm sheltering them too much.)
Seriously, most of these problems are due to lack of communication, set up a ticketing system, teach people how to use it.
Only thing I miss is my 3 24" monitors.
I mean, CHRIST, if this happened once in a blue moon, fine, whatever, but it seems like EVERY motherfucker uses IM the same way. EAT DICKS.
...
I might have went a little overboard on this.
If people start with "Hi, I have a problem" and don't state what the problem is I just respond with "drop me the details and I'll have a quick think" and get back to my work. Even if they respond immediately to that it might take me ten or fifteen minutes to get look at their message again - that is their time wasted, not mine.
Some problems require a more urgent response so I'll accept a walk-up or a shout for those, and some problems are best illustrated by showing them than describing them, I just have to trust people to know what they are and what is less urgent. And of course a message will come in at a time when I am eminently interruptible, in which case people get an immediate response anyway.
I find it works except for people who insist on not following the process, but they soon learn I'm far more helpful if they don't irritate me!
If people start with "Hi, I have a problem" and don't state what the problem is I close the chat window.
My work IM's status clearly states "if you have a question, ask it, don't ask if you can ask", time wasters simply get ignored.
> Some problems require a more urgent response so I'll accept a walk-up or a shout for those
Absolutely.
In the past I've found that leads to people reporting "I asked him several times over a few hours and got no response" which, if the person being complained to does not know the situation, could look bad for me.
If I feel like being more terse, I just give a one-word response: "Details?".
As I noted, my conditions are clearly spelled out as my IM status text, if they can't bother following simple instructions I can't bother with them.
Would you send an email that says "Hi", then another that says "I have a question", then another with your question?
Stop wasting my time. If sending "Hi" makes you feel better, fine, but the substance had better be on my screen by the time I look at the window, or you've wasted my time.
"Hi Dude, I've got this thing I need help with. Here is the description of what it is and what I am having a problem with."
It's not an issue if they put it in the same chat message, it's the "Hi", send message, "I have a problem", send message, "This is what the problem is", send message that drives me up a wall.
And if you REALLY need him to get back to let you know if he can help out immediately, you can send a second message along the lines of, "And sorry to be a bother, but I need help pronto. Please let me know if you can take a look at this right now." At that point, it's not a big deal if there's another message, because you haven't wasted time in relaying it.
Then refuse this, set your status to "don't bug me, if you have a question just bloody ask it" and check your IM when you've got a lull[0]. If it's not a question, kill the window it's not relevant. I don't need "social interaction" on IM when I'm coding, if the other guy can't get to the point screw him, I'm not interested.
[0] If IM is specifically a way to get you immediately, start blacklisting people who can't get to the point and waste your time and have they go through email.
But of course they won't use email, they need the answer NOW. No, they'll come over to your desk and start tugging on your shirtsleeve.
Actually, it's worse than that. The (average) time it takes to get back into flow is 15 minutes[1], so add the time to respond to them to that to get your wasted time. And if you get another interruption before the fifteen minute descent into flow is complete? Start over - another fifteen minutes. If this happens constantly, you will effectively never get any work done. There's more than one way to kill productivity.
[1] - See "Peopleware", Chapter 10.
If/when people start getting into your face (perhaps even formally) about performance, you have documentation to back up your argument. (Although, ultimately, most arguments are kind of moot, unless they end up in court or perhaps before an unemployment insurance evaluation -- not something you really want, anyway).
As for the overall experience. While negative, emotionally, you can still view it as a learning experience. Keep in mind that when you are in a better position, with more control, 90 % of the world with which you are competing is / will be what you are now experiencing. So... if you can cut out that bullsh-t, you will have a very significant competitive advantage.
It can be useful, to get a good, hard, close-up look at what you don't want. Just don't let it go on too long.
P.S. A trick I'll occasionally use, in problematic verbal situations, is to summarize the conversation/request in an email to the other party(s). 'We met. Here's what you asked me to do / how I understand it, what I'm going to do, and my anticipated schedule and potential conflicts / other overriding priorities. Let me know if I've misunderstood or you disagree.'
If no one responds, I proceed accordingly. If they bitch, hey, I laid out my approach and circumstances up front and asked for feedback.
It can put people on the defensive and create a negative vibe. But hey, when people tell you / manipulate you with statements of "We're losing confidence", things are not too positive, to start with. (P.S. That's already the sign/signal to bail. "Bail, bail, bail!")
What you are suggesting is easier said than done. I think that most good developers have the mentality you are describing...at first. When you work in an environment like this, however, you get worn down. You can only put up a fight for so long before you start getting all sorts of mixed up and you don't know what the f to do. If you took the best prioritizer in the world and put them in a corrosive environment like this one, the environment would eventually win out. I really feel for guys like the one that posted this original article and some of the subsequent commenters. I have been in that boat before and it is not fun.
The original MIT meaning is closer to what I recently heard described as meta-yak-shaving, which is when, before you can start shaving the yak, you discover that each hair of the yak contains a subyak, and each subyak has to be shaved individually.
None of the other people here are being particularly unreasonable, they're just ignorant. And given that "how to work with programmers, writers and mathematicians" is not part of any educations, it's not really their fault.
Try something like, "Can I get back to you in an hour? Boss wants me to finish this top priority item for him."
And if Boss asks you if you can finish it next week, say no. You're being asked to give an estimate, and it doesn't help to say "Yes, if [this thing doesn't happen which I know it will]". If they want you to be the startup guy, then act like you're the one in charge of your own schedule.
"So I know your project is important but projectX is still our #1 priority so to the extent Jenny needs you please prioritize her work"
Honestly, don't take this as an insult, but it just doesn't sound like you're the right person for the position. If you have trouble pushing back when you need to, then you should find a spot where there is someone pushing back for you.
I've, err we, have mostly failed.I can go on about why but at the end, I think we all share the blame and evaluate parts we could have personally done better and apply it next time round.
One of the big realizations for me is that they need someone older and more confident. I'm in my mid 20s and once founded a startup that got 30x the traffic but even months in, the boss would be unsure if I was a good programmer. We tried everything from emailing my code samples to his CTO friends to code reviews by other devs(all of which gave positive feedback) but I feel he could never totally shake that doubt. I called him out on it finally(basically if you can't tell after 8 months if I can program you should let me go). I've been programming since 12 but the lingering doubts actually started making me question my own abilities at which point I decided I need to move.
Let the managers fight it out.
Make damn sure though that people understand a short interruption "for a quick meeting" when you are at the time "powering through" stuff sometimes kills as much as a full hour of development time if you are working on something complex as it can completely break your train of thought (especially if you are tired due to recent overtime!), or worse if you are coordinating with other people and/or are working on something timing sensitive (so the interruption might mean a key coordination point is delayed).
If you can show what the demands on your time are, a good manager will try rearrange things so that you are interrupted less often. Sometimes simply merging those small interruptions into a scheduled period or two each day is enough to make all the difference to your mid/long term work: problems that didn't really need you magically go away (as the person who was going to interrupt you does that 30 seconds of extra thinking/research that was needed to give them the clue) and the others "interrupt" you a few times a day rather than several times over an hours or two.
This seems like a communication problem between the managers. (I can speak from experience, since I messed this up once when I reallocated our engineers' time without fully being on the same page as the engineering manager)
Boss: Could you do this thing for me?
Badclient: Well, as you know, I have Project X as my highest priority. I'm happy to oblige, but Jenn has dibs on my time at the moment. If you can talk to her and come to an agreement on how I should distribute my time, I'll be happy to work on your thing.
Whatever happens, your boss is on the same page as Jenn, and all you have to do is execute the agreed upon plan.
I mainly stuck around because it was my first FT job and I'd promised myself I won't bail on it for one year no matter how bad. In past I've regretted quitting on projects too quickly...at least I won't have that regret here.
2. Never let anyone get you in the position to set priorities yourself (unless you're the boss). Refer them to your manager or whoever is highest in the chain of command and assigned you a task to make this decision.
This is really more a working culture problem. You'd think it's in the management's best interested to make sure nobody ever violates 1. + 2.
You don't look bad for declining requests. You will get your shit done. Just learn to say no. Really.
Petty requests for emails that someone should have are their responsibility. If you think about it, why should your resourcefulness help them complete their job while it blocks your own progress?
Your boss is doling out reviews, he's #1 priority. Everyone else gets "no" and no access to your time without talking to him first.
"Being a team player" is a fluffy way of saying "do other peoples jobs for them". Sure, help people out and don't be a dick.
It does sound like a mess, but this type of mess exists everywhere. When you meld highly technical people with less technical people you will get these scenarios. Before you know it, you will be expected to replace the toner.
One week we all decided to keep a notepad and paper next to our desks and keep a log of every time we were interrupted and asked to context switch and do something random. It was so bad, before the end of Monday our boss was so dumbfounded he completely re-jigged everything so it wouldn't happen like that again.
We ended up calling one developer pair the "consulting developers" each week, and only that pair could be interrupted. There was a huge sign above their desk advertising them for the week. We sacrificed one for the good of all, and it worked really, really well. AFAIK, they are still doing that.
I think the root issue is they need a developer who just does tech support for the other 10 non tech people. And then a developer who can actually build shit quietly.
I will push this stance more aggressively having read your post.
If you want something more light weight look into the Pomodoro technique. People can bring issues to you but only in breaks between blocks of uninterupted work.
If you go with either then you can explain away refusing to be distracted as merely following a proven methodology. This gives you an easy out as people will vent any frustration at it rather than you.
I've gotten software changes that usually take two months to trickle through the system through in a day. You have to spend all day doing it, though, and so does your VIP.
But if it's really that important . . .
And as you said, a smart VIP is there in the room, suggesting things get done. Explicitly not writing things down in emails.
Additionally.. Going to "tell" on your buddies all the time and holding "the grownups" over them is terrible whinery. Everyone on the playground hates kids who do that.