Good article, I've always been on the lookout for the "silver bullet" when trying to manage a development team, but there's really no concise way to get good metrics and evaluate the team's burn rate and productivity efficiently.
Thanks! When humans are involved, things get way more complicated than expected, that's true. It's all about finding a good balance and a set of metrics that makes sense for your business & team.
Daily stand-ups were the best thing I ever did for this. We had our task list on big post-it notes on the wall. We talked about which ones we were going to accomplish, what we had done the day before, and what we expected to do that day.
That way, when I looked at their check-ins and their project updates, I had context for that.
Even more so when you're working on extremely complex systems like software.
Consider a simple example of two developers, Alice and Bob.
Alice spends 4 hours a day, at work, on reddit and pinterest, but she does excellent work. When she writes code it is of the highest quality. Her code very rarely has defects and is often extremely extensible and easy to use or modify. She only completes about 2 or 3 work items in a given week but usually they are very critical, core functionality components.
Bob, on the other hand, is very different. He works 12 hours a day and is often in the office on Saturdays. Unlike Alice he doesn't goof off at work, he's got too much on his plate. He usually cranks out 3 or 4 work items a day. However, his code isn't nearly as good as Alice's, often he'll have little design errors or implementation defects that creep into his work, but overall his code is still pretty good.
On paper Bob seems like a vastly more productive employee than Alice, and thus vastly more worthwhile having around. And in terms of naive measures those facts are abundantly clear. Bob puts in more hours. Bob completes more work items. Bob checks in more lines of code.
But none of those metrics matter, what matters is the improvement in the quality and capability of the product. And in those aspects Alice beats the crap out of Bob. Alice's checkins may have fewer lines but they are better designed and they are often in more crucial systems than Bob's. Moreover, a lot of Bob's checkins and work items are just bug fixes for defects that he himself introduced in earlier checkins.
Imagine two wood workers. One takes a single block of wood and spends a week laying out a plan and then meticulously carves out various pieces from the block then assembles them into an incredibly well made chair. The other takes half a dozen blocks of wood and cuts them down into pieces that vaguely resemble the parts of a chair using a chainsaw, leaving a giant pile of sawdust in his wake. Then he assembles the pieces into a chair-like structure and uses duct tape to hold it together. After repeated instances of the chair falling apart when used he applies more and more duct tape to certain parts of the chair until finally it stops falling apart and mostly works as a chair.
Again, if you were incapable of understanding the actual work that either worker did you may end up with a false impression of who is a more valuable worker based on the simplistic observable evidence. One worker spent a week to build a chair and most of the time they weren't even doing anything with the wood. The other worker built the chair in much less time, they started working almost immediately, and there was a flurry of activity related to chair making relative to the almost sedate minimalism of effort the first worker put in.
This is why it's absolutely vital that management has to have the technical chops to be able to tell the difference between an Alice and a Bob, and between high quality craftsmanship and duct-tape run-and-gun enthusiasm. If you end up promoting the Bobs and firing the Alices you end up in a very sad place indeed.
Congratulations, a new way to fire people quantitatively. I imagine this to actually be useful for call center owners, staffing managers and whomever that sells fixed units of activity for money.
The very first reaction from the founder at my company: "Seems a little big-brother-ish?"
I couldn't agree more.
The same thing can easily happen in all metric tracking for productivity. For instance, if you estimate points-per-story in an agile estimation process (or apples or oranges), you will naturally have an incentive to give higher point values to stories, and mark the stories complete. This puts incentive on the process, not the output.
The same goes for these distinct productivity applications. Peak serves as a high-level glimpse into data collected by tools that are supposed to aid in productivity; this data most certainly does not directly translate to "work being done".
Case in point - I've been working on a RubyMotion application. A lot of my work has been wading through StackOverflow posts to learn and overcome barriers. I don't have a problem saying that. But what tracks my "learning" metric? Absolutely nothing. Not the number of lines of code I write, not my emails, not even my web history (though that may be a better picture).
If you want to know what work is being done in our office, have a 2 minute conversation with the person who is doing the work. Verify what they say, and move on.
Productivity in web development isn't trackable by metrics. It's trackable by product goals and people sharing solutions to problems. If problems aren't being solved in a reasonably predictable and acceptable time period, that's where you can start to see cracks in productivity.
Don't start by looking at my inbox. Start with a conversation.
7 comments
[ 2.8 ms ] story [ 24.2 ms ] threadThat way, when I looked at their check-ins and their project updates, I had context for that.
EDIT: NP->AI :)
Consider a simple example of two developers, Alice and Bob.
Alice spends 4 hours a day, at work, on reddit and pinterest, but she does excellent work. When she writes code it is of the highest quality. Her code very rarely has defects and is often extremely extensible and easy to use or modify. She only completes about 2 or 3 work items in a given week but usually they are very critical, core functionality components.
Bob, on the other hand, is very different. He works 12 hours a day and is often in the office on Saturdays. Unlike Alice he doesn't goof off at work, he's got too much on his plate. He usually cranks out 3 or 4 work items a day. However, his code isn't nearly as good as Alice's, often he'll have little design errors or implementation defects that creep into his work, but overall his code is still pretty good.
On paper Bob seems like a vastly more productive employee than Alice, and thus vastly more worthwhile having around. And in terms of naive measures those facts are abundantly clear. Bob puts in more hours. Bob completes more work items. Bob checks in more lines of code.
But none of those metrics matter, what matters is the improvement in the quality and capability of the product. And in those aspects Alice beats the crap out of Bob. Alice's checkins may have fewer lines but they are better designed and they are often in more crucial systems than Bob's. Moreover, a lot of Bob's checkins and work items are just bug fixes for defects that he himself introduced in earlier checkins.
Imagine two wood workers. One takes a single block of wood and spends a week laying out a plan and then meticulously carves out various pieces from the block then assembles them into an incredibly well made chair. The other takes half a dozen blocks of wood and cuts them down into pieces that vaguely resemble the parts of a chair using a chainsaw, leaving a giant pile of sawdust in his wake. Then he assembles the pieces into a chair-like structure and uses duct tape to hold it together. After repeated instances of the chair falling apart when used he applies more and more duct tape to certain parts of the chair until finally it stops falling apart and mostly works as a chair.
Again, if you were incapable of understanding the actual work that either worker did you may end up with a false impression of who is a more valuable worker based on the simplistic observable evidence. One worker spent a week to build a chair and most of the time they weren't even doing anything with the wood. The other worker built the chair in much less time, they started working almost immediately, and there was a flurry of activity related to chair making relative to the almost sedate minimalism of effort the first worker put in.
This is why it's absolutely vital that management has to have the technical chops to be able to tell the difference between an Alice and a Bob, and between high quality craftsmanship and duct-tape run-and-gun enthusiasm. If you end up promoting the Bobs and firing the Alices you end up in a very sad place indeed.
I couldn't agree more.
The same thing can easily happen in all metric tracking for productivity. For instance, if you estimate points-per-story in an agile estimation process (or apples or oranges), you will naturally have an incentive to give higher point values to stories, and mark the stories complete. This puts incentive on the process, not the output.
The same goes for these distinct productivity applications. Peak serves as a high-level glimpse into data collected by tools that are supposed to aid in productivity; this data most certainly does not directly translate to "work being done".
Case in point - I've been working on a RubyMotion application. A lot of my work has been wading through StackOverflow posts to learn and overcome barriers. I don't have a problem saying that. But what tracks my "learning" metric? Absolutely nothing. Not the number of lines of code I write, not my emails, not even my web history (though that may be a better picture).
If you want to know what work is being done in our office, have a 2 minute conversation with the person who is doing the work. Verify what they say, and move on.
Productivity in web development isn't trackable by metrics. It's trackable by product goals and people sharing solutions to problems. If problems aren't being solved in a reasonably predictable and acceptable time period, that's where you can start to see cracks in productivity.
Don't start by looking at my inbox. Start with a conversation.