Couldn't agree more. I've spent a lot of time trying to explain this concept.
I do think the author has overlooked another common reason that problem definition gets overlooked: people simply conflate problems and solutions. Most of the "problems" people bring to me are actually solutions to some assumed and unacknowledged problem, e.g. "the button should be bigger." This preempts all critical evaluation of the problem and all of the alternative solutions.
PMs spending too much time on specific features is the key reason I advocate for problem-based roadmaps rather than feature roadmaps. All data points and feedback is often retrofitted as support for the initial feature solution (aka the feature you thought of first) rather than being used to diagnose the underlying problem.
Very interesting article! My doubt is about how to use a "problem roadmap" in practice.
It seems to me that a client problem is rarely completely "solved". So how to decide when it is time to move on to the next problem? It is clear when a feature is "done", it is less clear when a problem is done.
Your problems usually should be measurable, and sometimes %80 of the work is creating systems that can measure it. By thinking of how do I solve the problem first, instead of features, you might skip making a lot of stuff that is unnecessary in actually solving the problem.
One example is application startup time. Or revenues. User survey scores and so on.
You might not ever close a distinct problem if it’s core to your products purpose. It just keeps accreting evidence and pushing you towards creating better solutions.
Think about this problem as an example: Jane wants to listen to music when she’s mobile.
Solutions: iPod, iPod Mini, iPhone, ???
Each iteration is making a better way of listening to music when you are mobile. As technology improves better solutions can be built to solve the same problem.
Edit: I hate Apple scenarios but the iPod line up is the best example I’ve seen where iterating on solving the same problem vs solution had a remarkable result.
Interesting point of view. I would challenge this article by asking you what you are using the roadmap for. For me, a roadmap is mostly a communication tool. Off course, if I say I'll do something there's a commitment to that. But I always try to create an understanding that my roadmap is more a vision open to refinement and improvement than a set of solutions.
In this context, I find a feature roadmap easier to share most of the time. Discussing which problem I'm solving is always difficult for many reasons:
1) unless I'm really unfocused, the big problem I'm solving should be the same. I could tackle different parts of the problem, but it is more difficult to communicate
2) when you present a priority between problems you are solving, people develop a "negative" mood. Every problem is a priority. If you focus more on general features, there's more the issue of creating "pet projects".
That said, I always use a problem point of view when working on prioritization and approaching a product.
Some of this depends on what a "product manager" does. Someone needs to prioritize, define, and design the migration off of the Jabberwocky servers, which haven't had an OS upgrade in ten years, but have their IPs hardcoded all over the code base.
Maybe you have product managers tracking technical debt and strategic technical priorities, but that would be an exception rather than a commonplace thing.
But, even in those problems, focusing on and managing one predefined solution is usually an antipattern.
But when this category of problems is ignored (because it is green field development, often), the build phase blows up because you need to be neck deep in code or deployment on some client facing problem to discover these sorts of issues.
This is critically important. As a PM, I view it as one of our responsibilities to keep pushing to address the problems - even as we focus on external facing features.
In practice, what will sometimes (often?) happen is that someone will come up with a solution (e.g. elsewhere in this thread "The button on the right needs to be bigger") and then realize that it doesn't fit into the user story box. So they'll think about it for a second and write it as a user story: "As a user, I want the button on the right to be bigger"
yup. 9x out of 10 that’s what happens. personally i’m constantly pushing back on this type of story, or design doc requirement, even to principal level people.
I think my favourite example was "As a user, I want the $COMPANY logo to be displayed prominently in the top-left corner of the application"
Hell, I even accept that marketing is a valid stakeholder in the product. But just be honest about what it is! "For brand consistency, the marketing department would like the logo to be displayed in the top-left corner of the application, following standard company branding documentation." That's a perfectly fine "user" story (stakeholder story) I think. It answers the "why does the person want this?" question much better.
This is a great example. But I still have a bone to pick with the edited user story. It's not really a user story and crosses the line into implementation.
I'd propose a further revision:
"As a user, I'd like the home page to reflect a consistent set of branding guidelines so it reinforces the company's identity for me."
Logo display isn't covered by the branding guidelines? The company doesn't even have a branding guide?
Now you've uncovered a deeper problem. Fix that and hopefully you'll avoid a lot of aimless subjective debates over cosmetic issues in the future.
Generally speaking, as a user, I couldn't give a shit about the company's branding guidelines and how they display the logo, so long as it's not taking up 1/3 of the screen ;).
But I get your point, and agree about digging into those aimless debates. "As a user, I'd like a consistent visual experience across $COMPANY's products"
How do you handle people that simply don't do well on this? I have for example a front-end guy who's not really good in UX and always jumps in to code instead of discussing a mockup first.
I've been thinking about this a fair bit lately. How do you get from where you're at right now to something better? I haven't yet figured out any kind of holy grail approach that works, unfortunately :)
What I have been thinking about quite a bit is the idea of "initial conditions" in software. If a project starts out in a super cowboy way (let's not think about requirements, bring on the code!) and proceeds relatively successfully that way for a while, it's damned hard to reign that back in to a process where you do requirements and think about the problem before starting to write code. Best I can tell, the way to keep things like this from getting out of hand is to nip them in the bud early before they become habits.
That, unfortunately, doesn't help teams that are in the "nothing is thought out" swamp...
Edit: I just remembered the Scott Adams article on HN today too, talking about systems. That's exactly what this is all about: making systems where it's just assumed that you're going to do the requirements work up front. You can still do this in an Agile way, you just don't immediately pluck a card off and start writing code; you normalize the fact that the person is going to spend some time thinking about the solution first and writing those thoughts down.
That, too, is easier said than done. One team that I've worked with was heavily resistant to a 4 point checklist that was automatically added to their pull requests... It was not a burden and was specifically tailored to catching common mistakes they'd been making, but there was still a lot of resistance and ultimately the idea was abandoned.
Thank you for reply. I think it also depends a lot on the individual. You need more motivation to invest some thoughts instead of just getting it over with.
Is this front-end guy an employee? Or a contractor? That changes things a little bit, but I think things below still apply.
> You need more motivation to invest some thoughts instead of just getting it over with.
That's where your team's standards help, especially if you get them set up early. If, at the point this employee was hired, it was standard to (per the article's example) spend 40% of the task time up front thinking about the problem and writing it out, then the motivation is simple: If I don't do it this way, then I won't fit in with the team that I joined and I'll probably get fired.
Onto the situation you're in right now, where this person has been on your team for a while and not doing this... Do the other teammates do this? Is there code review? Do they flag the fact that this person is implementing features sloppily and push back on that at review time?
I joined the team half year ago and since then we started code review (but quite technical, so sloppy UX won't be caught here), I write actual concepts (instead of just 1/2 page of a rough draft before implementation starts), we discuss and adjust the concepts together and there is a roadmap (not super agile, but suits what we currently do best). So there is little culture to build upon, but we take steps in the right direction (though time pressure from upper management makes progress plateau a bit here).
About the (sole) front-end guy: he's an employee and a little less independent than the others.
Thank you for the encouraging me to develop the culture a further. Currently I try to get the devs to write propper tickets themselves at least (in a way that lets QA do their job afterwards).
> I write actual concepts (instead of just 1/2 page of a rough draft before implementation starts), we discuss and adjust the concepts together and there is a roadmap (not super agile, but suits what we currently do best)
> "That’s it. That’s why most companies don’t do it. It’s hard work, it’s wrongly viewed as unexciting, it can be hard to justify the initial output."
I have to disagree slightly here. While yes it's hard, unexciting and hard to measure, that still isn't why organizations don't do it.
The reality is that it just takes an incredible amount of time. This is especially true for startups and companies trying to move fast. You have to ask yourself — should I spend 2 sprints talking with customers about their problems, or 2 sprints building a POC that I can then test directly with them. In most cases, good PMs, engineers and designers have some basic idea for a solution that usually gets you 50% of the way there. So while I agree with the article is spirit, it's not a one size fits all process.
I've worked at 2 startups for a combined 14 years. One as an engineer and the other as a co-founder. I hate to say it but you and the author are both right.
That's the incredibly frustrating thing about startups. Where to spend your time depends on the context.
At first, you're desperate to validate your product and understand your customers. Spending more time in the lab won't save much time because you have little overhead and changes can happen quickly. Prototypes don't always convey a product's value. Imagine looking at mockups of Dropbox before it was released. I'd be the skeptic saying "So I can only sync one folder? That's stupid!", yet I love Dropbox.
I think Intercom's opinion is based on mature, profitable, companies that build ambitious new products. The proportion of time invested in user research, design and prototyping costs a fraction of changes downstream. My current company is in its sixth year and we have millions of users. In the first six months, users could tweet feature requests and we'd release that week. Building new features today requires a 3-month lead just to be considered for the roadmap.
PM done right can be magical. I don't believe in Smurfs, fairies or 100x devs but I believe in 100x Project Managers.
IME "100x devs" have been those that to a certain extent can do their own PM'ing (as well as people managing, have substantial technical chops in their own right, an insane work ethic...).
I should have mentioned that this magical 100X PM chooses the right solution to the biggest problem. I'm not referring to the PM who manages a project he or she was given. You can't manage your way to glory if you're solving a problem no one cares about.
Think Drew Houston deciding that magically syncing files on save without an explicit "Do you want to sync" step was a good idea. Magic doesn't happen often, but it exists in the world.
If so much time is spend on problem definitions, wouldn't it be less time for engineers to actually build the thing? While possibly refining and reducing the scope of your product, I'd say that approach would reduce velocity (at least in the short run? In the long run having less product surface = easier iteration)
I think that the point is that direction is more important than velocity. After all if you travel away from the target then the faster you go the more time you lose.
Perhaps the engineers already working on something else, for which the problem definition and prioritisation has already been completed? Just because you spend more time on definition and prioritisation, it doesn't mean your backlog shrinks to zero.
This article includes two images taken from “the (fantastic) book given out [to Facebook employees] the day of the IPO” of two pages, one about “ruthless prioritization” [0] and another about “code wins arguments” [1]. Does anyone have info on this book? What is the title? How many pages? Anything else in it of interest? I tried searching using these expressions and only got the @padday post. I’m interested in what seems like a set of guiding principles that was given out just as Facebook was starting out with its IPO.
I agree with the post that code wins arguments doesn’t really matter if the problem is misunderstood, but want to learn more as I frequently witness PM-types arguing over technology solutions and deciding without ever building anything to test and present evidence.
When decisions are heavily sales-driven--be it the loudest voices in the room are the sales team, or a big deal is coming down the pipeline requiring custom work--individual solutions start overriding problem-based thinking. It leads to a nasty cycle where clients just blindly ask for features and changes, but no one actually sits down to discuss or think about the "why" behind them.
Honestly the best way to deal with this is to train sales and client-facing teams to start defining the problem in the first place, and to set expectations with clients that the conversation needs to be about problems instead of proposed solutions. Product then needs to gather all the requirements (scoping how much dev work it takes, metrics on how much a change affects all clients vs. individual ones) to make good decisions on what the best solution should be. It's cheap to say "product teams need to be more problem-focused" because the idea pipeline rarely starts on the product team in the first place. Client-facing teams need to play a stronger role in defining the problem itself.
(This is mostly relevant for B2B, by the way. May not be as true for consumer tech.)
"When decisions are heavily sales-driven--be it the loudest voices in the room are the sales team, or a big deal is coming down the pipeline requiring custom work--individual solutions start overriding problem-based thinking. It leads to a nasty cycle where clients just blindly ask for features and changes, but no one actually sits down to discuss or think about the "why" behind them."
Very true. I have seen this plenty of times. At some point you need someone who keeps the big picture in mind.
In the time before app stores, I gave up on the mobile space because I saw this pattern happen at every B2B place I worked and the places by colleagues had worked.
Every network was a unique snowflake and we would bend over backward to get customer 3, creating a piece of code that was so difficult to land customer 5 that we started burning through money and people.
The worst one was the place where management and the sales people thought we should go after the biggest fish first (because they were looking for an exit). This created a problem where we had given sweetheart deals (low or in many cases negative margins) to the people who we should have had the easiest time getting cashflow positive off of. If only we'd waited to land them until we had more experience.
Don't land the whale until you know what to do with it.
Given the relative weighting of the sections of the product development lifecycle, I would expect Intercom to have approximately equal numbers of engineers and product managers. But based on a brief search of LinkedIn the number looks to be in a much more traditional range (I counted 16 product managers and stopped counting engineers at ~60).
49 comments
[ 2.5 ms ] story [ 122 ms ] threadI do think the author has overlooked another common reason that problem definition gets overlooked: people simply conflate problems and solutions. Most of the "problems" people bring to me are actually solutions to some assumed and unacknowledged problem, e.g. "the button should be bigger." This preempts all critical evaluation of the problem and all of the alternative solutions.
I wrote an article about this at https://medium.com/product-managers-at-work/the-product-road... if you want to learn more.
It seems to me that a client problem is rarely completely "solved". So how to decide when it is time to move on to the next problem? It is clear when a feature is "done", it is less clear when a problem is done.
One example is application startup time. Or revenues. User survey scores and so on.
The problem roadmap informs the feature roadmap.
Think about this problem as an example: Jane wants to listen to music when she’s mobile.
Solutions: iPod, iPod Mini, iPhone, ???
Each iteration is making a better way of listening to music when you are mobile. As technology improves better solutions can be built to solve the same problem.
Edit: I hate Apple scenarios but the iPod line up is the best example I’ve seen where iterating on solving the same problem vs solution had a remarkable result.
In this context, I find a feature roadmap easier to share most of the time. Discussing which problem I'm solving is always difficult for many reasons: 1) unless I'm really unfocused, the big problem I'm solving should be the same. I could tackle different parts of the problem, but it is more difficult to communicate 2) when you present a priority between problems you are solving, people develop a "negative" mood. Every problem is a priority. If you focus more on general features, there's more the issue of creating "pet projects".
That said, I always use a problem point of view when working on prioritization and approaching a product.
You can combine the two if you think of the problem as a “theme of solutions” you’re going to go build features for in your feature roadmap.
Thanks for this challenge - I think it deserves to be the topic of a follow up blog post :)
Maybe you have product managers tracking technical debt and strategic technical priorities, but that would be an exception rather than a commonplace thing.
But, even in those problems, focusing on and managing one predefined solution is usually an antipattern.
But when this category of problems is ignored (because it is green field development, often), the build phase blows up because you need to be neck deep in code or deployment on some client facing problem to discover these sorts of issues.
In practice, what will sometimes (often?) happen is that someone will come up with a solution (e.g. elsewhere in this thread "The button on the right needs to be bigger") and then realize that it doesn't fit into the user story box. So they'll think about it for a second and write it as a user story: "As a user, I want the button on the right to be bigger"
Hell, I even accept that marketing is a valid stakeholder in the product. But just be honest about what it is! "For brand consistency, the marketing department would like the logo to be displayed in the top-left corner of the application, following standard company branding documentation." That's a perfectly fine "user" story (stakeholder story) I think. It answers the "why does the person want this?" question much better.
I'd propose a further revision:
"As a user, I'd like the home page to reflect a consistent set of branding guidelines so it reinforces the company's identity for me."
Logo display isn't covered by the branding guidelines? The company doesn't even have a branding guide?
Now you've uncovered a deeper problem. Fix that and hopefully you'll avoid a lot of aimless subjective debates over cosmetic issues in the future.
But I get your point, and agree about digging into those aimless debates. "As a user, I'd like a consistent visual experience across $COMPANY's products"
What I have been thinking about quite a bit is the idea of "initial conditions" in software. If a project starts out in a super cowboy way (let's not think about requirements, bring on the code!) and proceeds relatively successfully that way for a while, it's damned hard to reign that back in to a process where you do requirements and think about the problem before starting to write code. Best I can tell, the way to keep things like this from getting out of hand is to nip them in the bud early before they become habits.
That, unfortunately, doesn't help teams that are in the "nothing is thought out" swamp...
Edit: I just remembered the Scott Adams article on HN today too, talking about systems. That's exactly what this is all about: making systems where it's just assumed that you're going to do the requirements work up front. You can still do this in an Agile way, you just don't immediately pluck a card off and start writing code; you normalize the fact that the person is going to spend some time thinking about the solution first and writing those thoughts down.
That, too, is easier said than done. One team that I've worked with was heavily resistant to a 4 point checklist that was automatically added to their pull requests... It was not a burden and was specifically tailored to catching common mistakes they'd been making, but there was still a lot of resistance and ultimately the idea was abandoned.
> You need more motivation to invest some thoughts instead of just getting it over with.
That's where your team's standards help, especially if you get them set up early. If, at the point this employee was hired, it was standard to (per the article's example) spend 40% of the task time up front thinking about the problem and writing it out, then the motivation is simple: If I don't do it this way, then I won't fit in with the team that I joined and I'll probably get fired.
Onto the situation you're in right now, where this person has been on your team for a while and not doing this... Do the other teammates do this? Is there code review? Do they flag the fact that this person is implementing features sloppily and push back on that at review time?
About the (sole) front-end guy: he's an employee and a little less independent than the others.
Thank you for the encouraging me to develop the culture a further. Currently I try to get the devs to write propper tickets themselves at least (in a way that lets QA do their job afterwards).
Keep doing what you're doing! That's awesome!
I have to disagree slightly here. While yes it's hard, unexciting and hard to measure, that still isn't why organizations don't do it.
The reality is that it just takes an incredible amount of time. This is especially true for startups and companies trying to move fast. You have to ask yourself — should I spend 2 sprints talking with customers about their problems, or 2 sprints building a POC that I can then test directly with them. In most cases, good PMs, engineers and designers have some basic idea for a solution that usually gets you 50% of the way there. So while I agree with the article is spirit, it's not a one size fits all process.
That's the incredibly frustrating thing about startups. Where to spend your time depends on the context.
At first, you're desperate to validate your product and understand your customers. Spending more time in the lab won't save much time because you have little overhead and changes can happen quickly. Prototypes don't always convey a product's value. Imagine looking at mockups of Dropbox before it was released. I'd be the skeptic saying "So I can only sync one folder? That's stupid!", yet I love Dropbox.
I think Intercom's opinion is based on mature, profitable, companies that build ambitious new products. The proportion of time invested in user research, design and prototyping costs a fraction of changes downstream. My current company is in its sixth year and we have millions of users. In the first six months, users could tweet feature requests and we'd release that week. Building new features today requires a 3-month lead just to be considered for the roadmap.
PM done right can be magical. I don't believe in Smurfs, fairies or 100x devs but I believe in 100x Project Managers.
100x Project Manager? I don’t see it. Not that they don’t bring value, but project managers just don’t have enough leverage in the process to be 100x.
Think Drew Houston deciding that magically syncing files on save without an explicit "Do you want to sync" step was a good idea. Magic doesn't happen often, but it exists in the world.
I agree with the post that code wins arguments doesn’t really matter if the problem is misunderstood, but want to learn more as I frequently witness PM-types arguing over technology solutions and deciding without ever building anything to test and present evidence.
[0] http://blog.intercomassets.com/wp-content/uploads/2017/05/08...
[1] http://blog.intercomassets.com/wp-content/uploads/2017/05/08...
Honestly the best way to deal with this is to train sales and client-facing teams to start defining the problem in the first place, and to set expectations with clients that the conversation needs to be about problems instead of proposed solutions. Product then needs to gather all the requirements (scoping how much dev work it takes, metrics on how much a change affects all clients vs. individual ones) to make good decisions on what the best solution should be. It's cheap to say "product teams need to be more problem-focused" because the idea pipeline rarely starts on the product team in the first place. Client-facing teams need to play a stronger role in defining the problem itself.
(This is mostly relevant for B2B, by the way. May not be as true for consumer tech.)
Facebook can have a rule, code wins arguments because they can do A/B testing. Enterprise software can not.
Very true. I have seen this plenty of times. At some point you need someone who keeps the big picture in mind.
Every network was a unique snowflake and we would bend over backward to get customer 3, creating a piece of code that was so difficult to land customer 5 that we started burning through money and people.
The worst one was the place where management and the sales people thought we should go after the biggest fish first (because they were looking for an exit). This created a problem where we had given sweetheart deals (low or in many cases negative margins) to the people who we should have had the easiest time getting cashflow positive off of. If only we'd waited to land them until we had more experience.
Don't land the whale until you know what to do with it.
A (2017) tag might be needed.
"We go to great lengths to apply science to this heavy upfront process. We use our:
Customer Support team (who tag every single conversation they have with customers for our PMs to review. We naturally use Intercom to do this btw)
Sales team and Marketing team (who build a product/market matrix based on inputs from the field)
Research team (who we invested heavily in since the very start)
Analytics team (who we are investing very heavily in now)"