@Matt, thanks for including this link. I'd read it a while back (maybe a link from you!) but couldn't find it when I was putting this together last night.
So what? If you have paying customers and the software "works" this isn't a huge downfall. New, young and challenged developers will gladly follow in the foot steps of seasoned coders before them.
> Recruiting will get harder
This is a problem for every company with developers. As long as you have paying customers, recruiting isn't as big of a problem as people make it out to be. Is this a problem for places like Google and Facebook? Sure. For you start-up, probably not. Software challenges tend to trail off towards the latter part of the software's life cycle. If there is a new piece of functionality required then smart developers will simply find the tools, frameworks, libraries, they need to solve that problem. They don't need this dictated to them.
> Productivity won’t improve
You also are inherently decreasing their productivity by teasing them with new technologies that may never be used. You can also clearly quantify their loss in productivity when they spend time on new technologies. I'd prefer to measure the measurable and manage that, rather than half guess a "innovation debt" that I'll never be able to manage properly.
> Your software will get stale
Are your customers still paying? Good, get over this aspect then. Software that doesn't change always goes stale.
It happens when the team is too busy putting out fires and finishing up features to keep up to date with advances in languages, frameworks, libraries, tools and processes.
What about the increased costs (real actual currency HR costs) that are included in constantly taking your resources off the software maintenance and delivery for a product/service that real people are paying for. There are some serious implications with letting people's innovative brains wander too far when given a specific task to accomplish. This is why hackathons and "FedEx" days are so popular. It's a mutual agreement between employer and employee that says: "You are allowed to hack something together with new exciting technology but I cannot guarantee it will reach the production stack"
I think this idea of innovation debt is a clear non-problem for me and most web based startups in my situation.
This idea is not aimed at startups. It's a huge problem for large enterprises, albeit less so for product companies than for captive IT orgs creating internal systems & utilities. If you, or anyone else, would like specific examples from one company -- along with discussion of remediation attempts -- feel free to email me. When the economy went south in 2007/2008 we dismissed about 18% of our global IT org and didn't start to rebuild until late 2011. Those 3+ years of stagnation nearly killed several business critical systems.
Fair enough -- loads of enterprise IT and IT software experience on my back...
captive IT orgs creating internal systems & utilities.
Again, so what? These are _costs_ centers and therefore treated as such. As long as the accounting team can add to the general ledger and the trucks ship the goods, the company doesn't go broke and people get paid. If you need innovation for your company to survive, then you do the innovator's solution - create an organisation outside the rules and realms of your existing structure to innovate.
This is all too wishful and pie in the sky type thinking.
I should have been clearer: innovation debt when combined with technical debt is the issue. When the internal systems stagnate and become a) unstable, b) un-supportable, c) un-upgradable, there is a direct impact to business operations, and escaping from the cycle is time consuming and painful.
I did exactly as you suggested, btw, and created a team whose explicit charter is innovation & directed R&D.
Thinking in terms of skimping on 'cost centers' is stupid. From the development side nothing changes when managing software for internal use vs. selling that same software. What people forget is if your internal software improves in functionality you capture 100% of that value, the only problem is assigning that savings to the software vs the shipping department or whatever.
What is often forgotten is time is the hardest thing to gain in software development. 1 person for 10 years can get a lot more done than 10 people for 1 year.
I don't want to get into an extended discussion, but in my opinion, most web based start ups don't do innovation. They build a small service people want and then 'coast' quite profitably and happily on that.
The heavy duty R&D and innovation requirements start coming into play for different sorts of shops. E.g., enterprises or r&d-driven consulting shops. If you don't have R&D/innovating driving improvements to your products, you will wind up dead in the water.
Which is funny, because they often bill themselves as 'changing the world.'
There is a distinct dearth of R&D-to-product shops, and our industry suffers greatly for it. Intellectually, some people need more than displaying a DB on a web page, no matter how profitable it is at this time.
This is a very short-sighted view. While losing great devs will not put your company out of business over night, it may start a death spiral that it will be very difficult, if not impossible, to recover from. Hiring may be easy for you now, but once your start up starts falling behind the curve, if will suddenly look a lot less attractive to new talent.
> If there is a new piece of functionality required then smart developers will simply find the tools, frameworks, libraries, they need to solve that problem. They don't need this dictated to them.
What smart developers? They have all left to work for your competitor. Besides finding the right tools, frameworks, etc takes time, which is precisely what this article is arguing for.
> Are your customers still paying? Good, get over this aspect then
Your customers will stop paying when something better comes along. And it will come along because, unlike your company, others are innovating.
> You also are inherently decreasing their productivity by teasing them with new technologies that may never be used.
Maybe, but not by as much as you'd think. Looking into new technologies may often improve productivity with the technology currently being used (by bringing in ideas from the new tech).
> What about the increased costs (real actual currency HR costs) that are included in constantly taking your resources off the software maintenance
You don't have to constantly take resources off. Just occasionally. A day-long hackathon a year, and a couple of lunchtime sessions a moth is really not that much.
> I think this idea of innovation debt is a clear non-problem for me and most web based startups in my situation.
The thing about innovation dept, like architectural debt, is that it doesn't become a problem until it is too late to do something about it.
That, in turn, gets (eventually) impacted by innovation debt.
> And do you believe this is seriously something that is killing companies left and right?
No, not at all; and I don't believe the author of the post does either. There is a whole bunch of things that can kill a company (especially a startup), and a lot of them are going to be bigger and more urgent then innovation dept. You should definitely focus on dealing with those. However, innovation dept should not be dismissed entirely, especially since it doesn't take much to offset it.
If you have code that is difficult to maintain and build upon and as a result are loosing a lot of your top development talent then this will eventually kill the majority of companies.
But like you said if you have little to no cashflow right now then that is going to kill the business faster. It's all about finding a balance. You need to make sure that you have the cashflow to get to the point where having a sustainable code base and talent matters. If you worry too much about innovation debt off the bat you'll never get out the front door, but if you don't worry about it at all you will likely reach a point where everything eventually comes crashing down.
I certainly lean heavily in the direction you describe; cashflow or better still, net profit is the lifeblood of any real business.
So if you're barely staying alive, anything that doesn't soon lead to revenue and acceptable gross profit is a luxury you can't afford.
On the other hand, if you are an owner with a steady profit-generating machine you want to keep it running as smoothly as possible as long as possible. Investing a relatively small amount directly in your employees' professional development is a way to help this happen.
Employees like to stay current and feel like you care enough to keep them current. Going from spending $0/yr on someone for training to $1000/yr will probably make them a lot happier than a $1000/yr raise. That might be as simple as giving a group a collective budget for books and courses on anything related to your business.
There is a definite cost when a core member of your team, no matter which level, moves on due to boredom. This is one way to keep them a little happier. Some aspects will show up in your company's resources beyond just happier employees too.
I'm currently working at a company that is facing this problem. The programming team has stagnated on repairing legacy systems. Innovation and modernization is a business objective, but the philosophical issue of losing 'expert business knowledge' due to technological evolution presents barriers in the process.
Our code ranges from 10 to 20 years old. Most of the principles involved in that based are from those years too. Moving to any framework, language or SOA means training 16 people for years. Not because the language, framework, or services are too hard to grasp, but because the overall arching backbone is missing. We haven't reached 3-layered programming, let's jump straight into 5-layered.
We're still in waterfall functional code with weak modularization and few OOP principles. Yet, we're trying to skip the whole OOP/modularization schematic and jump right into fully decoupled SOA. Without understanding the hardships of the middle, I find it difficult to believe we can transition to the end without losing half of our team.
Yeah - that's a really hard change. For anyone dealing with a legacy system, I always recommend Mike Feathers' book on "working effectively with legacy code". He talks about lots of patterns for dealing with this.
I'd probably see if I could start to break off a piece of a system, bring in a consultant or new dev for a couple of months, pair them with one of your team and rotate every few weeks. You won't lose too much capacity in terms of working on the core system and you'll get a chance to start to slowly introduce members of your team to new best practices. Sometimes single page web apps can help with some of this as you can just drop in backbone on a page or two and use it to federate data from both your main app and a new app you're adding some functionality to.
You probably will end up losing half the team over time, but honestly if they put up with the current state of affairs for so long, that might not end up being a bad long term outcome. Just don't big bang any changes - cultural or technical. And try to ignite curiosity rather than "mandating innovation"
Testers feel that pain, too. (Not talking about dev/testers but dedicated testers and white-box ones, in particular). Testers get bored when they're testing repeatedly testing something which does not change at all technologically. And since the productivity is usually lower with the legacy technologies, testers can't even count on many new features in each release. Yes, testers get jealous, too.
Another way to keep sharp, which surely Peter knows (as he runs several meetups in NYC), is to encourage your team to get involved with local meetups, both as an attendee and a presenter, or do other community-oriented stuff like teaching classes (e.g. skillshare). This has two positive effects – increases the flow of new ideas, and (when speaking) helps your company / team to get more visibility in the community.
While I am very supportive of learning and using cutting edge technologies when it makes sense, it does need to balanced with technical risk assessment, which your post alludes to at the end. For example, say you want to try the Rust language and think it would give you some advantage. Things like maturity / community, etc, definitely come into play.
Innovation comes not just from using new technologies, but also from using new techniques with old boring technologies. I’d even argue that working on cutting-edge problems forces you to innovate, alleviating some of the innovation debt issue (not everybody has that issue).
You can innovate on nonfunctional areas, such as devops, logging, metrics, etc, if your core business needs to be more conservative.
One of the best talks I went to in the last year was by CTO of Etsy, where he talked about how they use “the most boring technologies they can find” – php, mysql, etc. At the same time, Etsy has a very strong reputation in the tech community.
http://www.infoq.com/presentations/Etsy
Agreed 100% - both with the value of meetups and balancing the technical risk assessment. I'm doing a keynote at NFJS New York on "how to select and adopt technologies" that talks a lot about community focus, where in the technology adoption lifecycle the technology is, etc.
You also have to look out for how many technologies you're managing. I'm a fan of polyglot programming and polyglot persistence, but that doesn't mean you should have five server side languages and seven different NoSQL data stores.
Agreed re: innovating on peripheral apps. Only thing I'd add is select technologies that could be great for your core, innovate peripherally, and once you have some experience you can revisit the risks of introducing that tech into your core stack.
26 comments
[ 2.7 ms ] story [ 63.2 ms ] threadSo what? If you have paying customers and the software "works" this isn't a huge downfall. New, young and challenged developers will gladly follow in the foot steps of seasoned coders before them.
> Recruiting will get harder
This is a problem for every company with developers. As long as you have paying customers, recruiting isn't as big of a problem as people make it out to be. Is this a problem for places like Google and Facebook? Sure. For you start-up, probably not. Software challenges tend to trail off towards the latter part of the software's life cycle. If there is a new piece of functionality required then smart developers will simply find the tools, frameworks, libraries, they need to solve that problem. They don't need this dictated to them.
> Productivity won’t improve
You also are inherently decreasing their productivity by teasing them with new technologies that may never be used. You can also clearly quantify their loss in productivity when they spend time on new technologies. I'd prefer to measure the measurable and manage that, rather than half guess a "innovation debt" that I'll never be able to manage properly.
> Your software will get stale
Are your customers still paying? Good, get over this aspect then. Software that doesn't change always goes stale.
It happens when the team is too busy putting out fires and finishing up features to keep up to date with advances in languages, frameworks, libraries, tools and processes.
What about the increased costs (real actual currency HR costs) that are included in constantly taking your resources off the software maintenance and delivery for a product/service that real people are paying for. There are some serious implications with letting people's innovative brains wander too far when given a specific task to accomplish. This is why hackathons and "FedEx" days are so popular. It's a mutual agreement between employer and employee that says: "You are allowed to hack something together with new exciting technology but I cannot guarantee it will reach the production stack"
I think this idea of innovation debt is a clear non-problem for me and most web based startups in my situation.
captive IT orgs creating internal systems & utilities.
Again, so what? These are _costs_ centers and therefore treated as such. As long as the accounting team can add to the general ledger and the trucks ship the goods, the company doesn't go broke and people get paid. If you need innovation for your company to survive, then you do the innovator's solution - create an organisation outside the rules and realms of your existing structure to innovate.
This is all too wishful and pie in the sky type thinking.
I did exactly as you suggested, btw, and created a team whose explicit charter is innovation & directed R&D.
c) un-upgradable, there is a direct impact to business operations
I'd love to know how to measure this. Any VP of Tech/CIO would be fired if this happened.
What is often forgotten is time is the hardest thing to gain in software development. 1 person for 10 years can get a lot more done than 10 people for 1 year.
The heavy duty R&D and innovation requirements start coming into play for different sorts of shops. E.g., enterprises or r&d-driven consulting shops. If you don't have R&D/innovating driving improvements to your products, you will wind up dead in the water.
There is a distinct dearth of R&D-to-product shops, and our industry suffers greatly for it. Intellectually, some people need more than displaying a DB on a web page, no matter how profitable it is at this time.
> If there is a new piece of functionality required then smart developers will simply find the tools, frameworks, libraries, they need to solve that problem. They don't need this dictated to them.
What smart developers? They have all left to work for your competitor. Besides finding the right tools, frameworks, etc takes time, which is precisely what this article is arguing for.
> Are your customers still paying? Good, get over this aspect then
Your customers will stop paying when something better comes along. And it will come along because, unlike your company, others are innovating.
> You also are inherently decreasing their productivity by teasing them with new technologies that may never be used.
Maybe, but not by as much as you'd think. Looking into new technologies may often improve productivity with the technology currently being used (by bringing in ideas from the new tech).
> What about the increased costs (real actual currency HR costs) that are included in constantly taking your resources off the software maintenance
You don't have to constantly take resources off. Just occasionally. A day-long hackathon a year, and a couple of lunchtime sessions a moth is really not that much.
> I think this idea of innovation debt is a clear non-problem for me and most web based startups in my situation.
The thing about innovation dept, like architectural debt, is that it doesn't become a problem until it is too late to do something about it.
And do you believe this is seriously something that is killing companies left and right?
You know what kills companies? Cashflow.
That, in turn, gets (eventually) impacted by innovation debt.
> And do you believe this is seriously something that is killing companies left and right?
No, not at all; and I don't believe the author of the post does either. There is a whole bunch of things that can kill a company (especially a startup), and a lot of them are going to be bigger and more urgent then innovation dept. You should definitely focus on dealing with those. However, innovation dept should not be dismissed entirely, especially since it doesn't take much to offset it.
But like you said if you have little to no cashflow right now then that is going to kill the business faster. It's all about finding a balance. You need to make sure that you have the cashflow to get to the point where having a sustainable code base and talent matters. If you worry too much about innovation debt off the bat you'll never get out the front door, but if you don't worry about it at all you will likely reach a point where everything eventually comes crashing down.
So if you're barely staying alive, anything that doesn't soon lead to revenue and acceptable gross profit is a luxury you can't afford.
On the other hand, if you are an owner with a steady profit-generating machine you want to keep it running as smoothly as possible as long as possible. Investing a relatively small amount directly in your employees' professional development is a way to help this happen.
Employees like to stay current and feel like you care enough to keep them current. Going from spending $0/yr on someone for training to $1000/yr will probably make them a lot happier than a $1000/yr raise. That might be as simple as giving a group a collective budget for books and courses on anything related to your business.
There is a definite cost when a core member of your team, no matter which level, moves on due to boredom. This is one way to keep them a little happier. Some aspects will show up in your company's resources beyond just happier employees too.
The irony, of course, is the tagline used at ignitelab.net is: "Ignite Your Innovative Internet Business Idea Right Now"
Our code ranges from 10 to 20 years old. Most of the principles involved in that based are from those years too. Moving to any framework, language or SOA means training 16 people for years. Not because the language, framework, or services are too hard to grasp, but because the overall arching backbone is missing. We haven't reached 3-layered programming, let's jump straight into 5-layered.
We're still in waterfall functional code with weak modularization and few OOP principles. Yet, we're trying to skip the whole OOP/modularization schematic and jump right into fully decoupled SOA. Without understanding the hardships of the middle, I find it difficult to believe we can transition to the end without losing half of our team.
I'd probably see if I could start to break off a piece of a system, bring in a consultant or new dev for a couple of months, pair them with one of your team and rotate every few weeks. You won't lose too much capacity in terms of working on the core system and you'll get a chance to start to slowly introduce members of your team to new best practices. Sometimes single page web apps can help with some of this as you can just drop in backbone on a page or two and use it to federate data from both your main app and a new app you're adding some functionality to.
You probably will end up losing half the team over time, but honestly if they put up with the current state of affairs for so long, that might not end up being a bad long term outcome. Just don't big bang any changes - cultural or technical. And try to ignite curiosity rather than "mandating innovation"
That last line really shifted my outlook on this experience.
While I am very supportive of learning and using cutting edge technologies when it makes sense, it does need to balanced with technical risk assessment, which your post alludes to at the end. For example, say you want to try the Rust language and think it would give you some advantage. Things like maturity / community, etc, definitely come into play.
Innovation comes not just from using new technologies, but also from using new techniques with old boring technologies. I’d even argue that working on cutting-edge problems forces you to innovate, alleviating some of the innovation debt issue (not everybody has that issue).
You can innovate on nonfunctional areas, such as devops, logging, metrics, etc, if your core business needs to be more conservative.
One of the best talks I went to in the last year was by CTO of Etsy, where he talked about how they use “the most boring technologies they can find” – php, mysql, etc. At the same time, Etsy has a very strong reputation in the tech community. http://www.infoq.com/presentations/Etsy
Agreed 100% - both with the value of meetups and balancing the technical risk assessment. I'm doing a keynote at NFJS New York on "how to select and adopt technologies" that talks a lot about community focus, where in the technology adoption lifecycle the technology is, etc.
You also have to look out for how many technologies you're managing. I'm a fan of polyglot programming and polyglot persistence, but that doesn't mean you should have five server side languages and seven different NoSQL data stores.
Agreed re: innovating on peripheral apps. Only thing I'd add is select technologies that could be great for your core, innovate peripherally, and once you have some experience you can revisit the risks of introducing that tech into your core stack.
Nice link re: the infoQ etsy talk - thanks!