Scrum and Technical Debt
We follow scrum practices and use Atlassian's (pretty good) JIRA software. Most of scrum makes sense to me: morning standups so we all stay in the loop, a scrum master to keep some of the process from distracting us, short stories so we can digest the workflow smoothly.
But one thing makes no sense to me: There's no place in our processes for cleaning up technical debt. Bad code is pretty much written in stone. I've explicitly been told not to clean up code (I sneak some in occasionally when I can get away with it). It's bad enough that I've also been told not to trust the names of functions. :\
"User stories" seem to exclude any code cleanup by definition. I searched a little and, for example, the Wikipedia article on Scrum doesn't even mention technical debt.
Does scrum just ignore code cleanup? If so, why does anyone even do scrum? Does anyone else do scrum and still clean up technical debt? How do you incorporate it? Create technical debt "user stories" or as part of other user stories?
7 comments
[ 3.5 ms ] story [ 31.8 ms ] threadSprint Planning Meeting - The team is responsible for selecting the amount of work they feel they can implement without accruing technical debt. The team “pulls” work from the Product Backlog to the Sprint Backlog.
Failure to change old habits leads to technical debt and eventual design death
Pg 2 http://scrumreferencecard.com/ScrumReferenceCard.pdf
The reference card above is part of this free series http://scrumtrainingseries.com/
That's the thing with agile, it only works well if you have 2 things:
1) "An agile codebase" with good tests, and little technical debt. It's very hard to adopt an agile process on existing code which doesn't meet these standards.
2) A process that has a Definition-of-Done which includes that there is no added tech debt. Ideally code should be peer reviewed and the commits rejected if there is any smell/debt added.
Remember: in scrum developers are responsible for estimating the development cost of the stories. If you know that the story X will involve touching a piece of code that has some wrinkles, it's your job to set the effort estimate including the cleanup time required to make the code good enough to implement the new feature on. At the top level, it's fine to leave horrible code until there is a bug reported against it. On lower levels however, you must avoid basing new code on old bad code. An example can be in OO, if a base class has a bad design, don't add yet another subclass to make a new feature. As part of that new feature refactor the base class and all existing subclasses if necessary. The counterexample: if one subclass smells, leave it until there is a bug in that particular subclass. The same reasoning can be said for anything that builds on anything else. Don't add new calling code to a smelly API, fix the API first.
The problem is just to actually make the developers take the time they need when making estimates. The time it takes to analyze existing code, investigate test coverage, write new tests, refactor old code and THEN implement new code, (and then probably do more refactoring and add further tests) is underestimated. You know what happens. The feature (barely) gets implemented, but there are some loose ends, substandard testing etc. Again: it's the developers responsibility to make these estimates big enough. Next time you estimate a trivial thing: list the subtasks in your head and sum the hours (read old code 1h, refactor old code 1h, fix old tests 1h, fix old technical debt 2h, write new tests 1h, implement new feature 2h, ...). Those 2h you initially thought it would take now doesn't seem like a lot! If your peers disagree with your 10h estimate for the trivial feature, mention the tasks to them and make them estimate+sum themselves.
The development speed will initially go down, in terms of new features added. Managers will be sweating and deadlines will slip. Don't worry, if you have a smelly codebase that slowdown would happen later otherwise. It's (normally but not always) better to keep the code clean and miss an early deadline (or even cancel the project) rather than miss a late deadline, when there is a lot invested already. This is still a management decision. Tech debt is called tech debt because that what it is. But not all debt is bad. Sometimes you have to take a loan and repay it (150%...) later. It's no good to ship a game with nice code if you miss your christmas deadline. Better to make it work with smelly code, and patch it 3 weeks later. That's just a reality (anyone who has bought the 1.0.0 of a recent game knows this). If your manager says to cut corners, cut corners, but do write the technical debt fixing story, or add it as a task to another story. Don't write // TODO: Fix this in the code and then forget about it. Make the project owner know that saving 8h today will cost them 16h in a month. Do not accept a continuously decreasing code quality. Quality can be a sawtooth pattern where you sometimes rush and then clean up (just a business reality), it just can't be a continuous slope. If your managers don't accept this reasoning you know what to do.
> How do you incorporate it? Create technical debt "user stories" or as part of other user stories?
Both. For larger refactoring efforts, it may be worth having se...
As Uncle Bob points out, when code doesn't have tests, programmers are rightfully afraid of touching it. One approach to legacy systems that limits the risk of breaking working code is Eric Evans "anti-corruption-layer" [0] approach.
Good luck.
[0] http://allegrotech.io/working-with-legacy-architecture.html