Sometimes I would almost advocate a little confusion because who is to say you've correctly identified the priority. Who's to say the bug is correctly triaged?
I disagree. If a lot of people know about the bug, and actually take time to reach out and tell you, it must be more severe than you thought.
People these days are used to bugs. And bugs that arent as severe often go completely unreported; Its so common we shrug it be off. People tend to report bugs when they see no other option.
There is certainly a difference beween the two, but to say that they are completely different things is going too far, as can be seen by some of the examples used to justify that claim - e.g.:
Something that is Critical severity doesn’t necessarily mean its a P0. That crashing bug in your bio screen? Face it, nobody really cares to read it and few will ever click the button, so its not something that should be a P0 ship-stopper.
In what way is that an issue of 'critical severity'? The quoted text contains an explanation of why it is not.
The real difference is that severity is a measure of the harm done, while priority is a plan for to responding to it. Clearly they are different, but clearly they will likely be correlated, at least when the severity is high.
> In what way is that an issue of 'critical severity'? The quoted text contains an explanation of why it is not.
The only reason would be this: given a large enough enterprise you have one person categorizing a bug as a crash and another making the call on whether that crash is actually worth prioritizing, a decision made on another day at another pay grade in another time zone. In this case it might be clear, but in the next case it might be "the configuration page crashes if the region is France". I as a developer don't know and I don't care that we haven't even launched the product in France so I just want to mark the issue as a very severe crash. Somewhere above my head the can call this low prio because the product launch in France is still a year out. This is enterprise backlog massage for enterprise workflows.
This is flirting with redefining 'severe' as something it is not. Severity is about impact on the user.
In your scenario, you don't know whether it is severe or not, so you just want to provisionally mark it as severe, and have someone with the appropriate domain knowledge make the final call. Until that happens, decisions made on the basis of that provisional call could be bad decisions.
And what might those decisions be? Priority, for one! To say that the severity should stay at 'severe', and the adjustment should be made only in the priority, is to confuse the two distinct concepts. I see a lot of that going on wherever data are used.
If you want to characterize bugs by the sort of technical outcome they produce, then do just that, instead of trying to bend 'severity' to try and encompass it, while actually hiding the issue that you wanted to characterize on. Again, while technical outcome is an aspect of severity, they are not the same thing (and, FWIW, crashing is rarely the worst thing that could happen (unless the domain is vehicle control.))
No I have the Domain knowledge and know that it’s a crash that occurs on odd weeks in France. I noticed it while developing and I can confirm it in the code. This to me is exactly as severe as the same crash in the US (as far as I’m aware). I must assign a high severity.
However I have little to no knowledge of the business (markets, release schedule). I’m not even aware that we have zero users in this region. A manager I have never met assigns a low priority, two months later.
Domain and business overlaps, but for example if one doesn’t mark this “severe” because the business hasn’t launched for affected customers, then a year later when it is launched in France you have a low severity bug that should be high.
The priority can be constantly re-evaluated against business needs while the severity is fixed.
Yes this is a simple technical characterization. It’s not useless to separate the business-stopping bugs from the cosmetic. And importantly: It’s not my call to say what is important.
Someone might say that the font size is more important than a deadlock. I don’t care and I don’t want to care. But I’m rather sure that those that do make those decisions want to be able to sort the bugs I wrote by a severity.
There is no "severe to you" as a dev in this sense. Severe to you is when your IDE is not working and you can't work on the priority tasks.
The crash has no severity to you. It is either severe or not to the client. If you don't know whether it is severe to them, sure you can guess that a crash in france is severe, but that's an incorrect guess, not an independent dimension of severity.
There is no "techical measure of severity", everything is goal oriented. It's not severe unless it's important to the client, or is impeding something that is important to the client.
In that case, I don't see this as very relevant as a counter-example to my original point, though you haven't made it clear what your organization's concept of severity is, and why this example counts as such. Does your definition convey any information, to the person setting the priority, that is not redundant with respect to the other information about the bug that they must necessarily take into account when making that decision?
Do you call a bug that corrupts data more or less severe than a bug that crashes a program? In whichever order you put them, is the severity always necessarily in that order?
>I’m rather sure that those that do make those decisions want to be able to sort the bugs I wrote by a severity.
Why would they do that, if, as you say, "It’s not my call to say what is important"? In what way is this not you signalling that you suspect, provisionally, this has a severe impact, as I suggested in my previous post? I think you are being a bit inconsistent here.
Google has this concept. P0 S0 is the most urgent. P2 S2 is “standard”. P3/4 S3/4 is pretty much code for “this will never happen”. 99% of the time Pn = Sn. When it doesn’t it’s almost always off by 1. So practically I never saw the point in this distinction.
It's not a bug report if there's no one consuming the backlog. It's a wish list.
In my experience P3's capture tech debt, cut features, and design flaws. Those should be addressed for what they are, not shoehorned into a bug prioritization system that will never even glance at them.
> a P0 (Blocker) bug is a ship-stopper which has to be fixed before the next release can happen. P1s (Critical) are important but not something we’d stop a release for, while P2s (Medium,Low) represent pretty much anything that will only be fixed when our development team has cleared all P0 and P1 bugs
> It’s important to note that a bug’s priority shouldn’t be confused with its severity, which exists as an entirely different dimension... ‘Severity’ to represent a (somewhat) objective assessment of a bug’s impact to the functionality of the system. From Critical representing major crashes or hangs, to Minor functional issues or purely Cosmetic blemishes. While a QA engineer will classify a bug’s Severity at creation time, it’s the PM who assigns the Priority at the time of triage based on their knowledge of the client’s business and product requirements
I don't see the benefit in having severity and priority as 2 distinct dimensions. Given a specific priority, what additional benefit do you get from having a severity label as well? Based on the author's description, I can't think of any.
If the goal is to better communicate what "type" of bug it is, using an enum classification (eg, cosmetic vs crash vs data-corruption) would be more appropriate than a numeric scale.
Because as the quote said, it depends on the client requirement.
"if you click anywhere in the left half of the screen during the 5th screen all the settings get deleted" might be a very severe/critical bug, but if the client says "thats fine for me don't worry, but can you fix the wrong color on screen one where the logo is purple instead of fuschia that's not acceptable for my needs" then this one has a higher priority, even though it's much less severe.
Your job is not to build the perfect/best product, it's to build the one your client want and is willing to pay money for.
The mixup between priority and severity only comes from the modern era when so many product get made for internal consumption, or where the user is not not the buyer.
The severity is a technical assessment of the issue that the PM, who doesn't necessarily have a technical background, weighs against business needs to establish the priority. Once there is a priority, the severity has already served its purpose.
Why would you weigh the technical "severity" ("what happened?") against the actual severity ("business needs")? The severity, by hypothesis, has no influence on whether you're going to work on the bug or not. You want to weigh the business needs against how easy the bug is to fix, not against what happened as a result of the bug. "What happened as a result of the bug" is purely within the domain of business needs.
I'm really confused by all this arguing against simply having more information.
Is the information about the criticality of the bug (whether or not the app crashes) relevant to the business decision?
Yes, of course: "When I press the 'more' button in the developer's bio the app crashes" is a more important bug than "When I press the 'more' button in the developer's bio I don't see more text."
Might the criticality of the bug not be relevant to the business decision? Again, yes: "When I press the 'more' button in the developer's bio -- ok, stop, I don't care about this bug."
In some cases the criticality is relevant to the business decisions, in others it may not be. It probably usually will be. The app shouldn't crash, because even if it only happens to 1% of people, they're the ones writing reviews on the app store.
So the criticality is a big red "Hey! This should probably be high-priority!"-flag, but it's not the only piece of information the PM is going to use.
> I'm really confused by all this arguing against simply having more information.
I think the argument is that it's not actually more information; it's just smearing out the same information in a way that makes communication and decision making harder. (Note I'm not completely convinced by either side here, just trying to explain.)
> I'm really confused by all this arguing against simply having more information.
What people are pointing out is that you're not arguing for having more information. The "severity" is a marginal zero information on top of the priority. There is no context where you would want to use "severity" instead of priority.
So...
GIVEN that priority trumps "severity" 100% of the time,
AND GIVEN that priority is already being recorded,
THEREFORE "severity" has no use, and shouldn't be recorded.
What's happening in the post is that someone wondered why there were two fields, and made up a justification for the second field instead of realizing it was a mistake to have two.
You can see the mistake happening whenever people choose a word that ordinarily refers to the concept of importance as their label for "severity", while simultaneously acknowledging that "severity" doesn't affect the importance of the issue.
They're saying that severity informs the decision of how to set priority. I'm not sure why that's necessary since severity is implicit in the description of the bug.
> In some cases the criticality is relevant to the business decisions, in others it may not be. It probably usually will be. The app shouldn't crash, because even if it only happens to 1% of people, they're the ones writing reviews on the app store.
> So the criticality is a big red "Hey! This should probably be high-priority!"-flag, but it's not the only piece of information the PM is going to use.
The point is that PM may be inundated with lots of bugs and new features and other stories that all need prioritization before the next release. If they had a perfect understanding of all of them, they wouldn't need any metadata at all (not even "bug" vs "feature" -- just read the long-form description!), but metadata helps people make decisions.
High-severity bugs should generally be prioritized higher, except in cases where PMs have gained a deep understanding of the situation and realize it's not business-critical, and override that.
At the point that they set the priority, the criticality is no longer important, just as the "bug" vs "feature" tag is no longer that important once it's been prioritized and assigned.
> I'm really confused by all this arguing against simply having more information.
As with all information gathering, you need to weigh the cost of gathering and storing vs the gains of having the information.
People are saying there are no gains from having this information; which, if true means if there's any cost, the cost is too high.
People are saying it is hard to determine, which means the cost to gather it is high, it may include concensus building.
In my personal professional experience, there hasn't been a whole lot of retrospective analysis on the bugtracker data. So information gathering that doesn't immediately help with fixing the bugs, or communicating which bugs need to be fixed sooner is time wasted. Use of non-critical fields in the tracker is spotty and inconsistent, so analysis would require an expensive data cleaning phase.
Sure, you can say I didn't work in healthy organizations if they didn't do this sort of analysis. But that's the root cause of most of the bugs --- there were never any formal specifications and so many things didn't meet the specs; plus occasionally we wrote crashy code.
> Yes, of course: "When I press the 'more' button in the developer's bio the app crashes" is a more important bug than "When I press the 'more' button in the developer's bio I don't see more text."
Of course? Really?
I mean, the upshot for the user is the same - they wanted to see more info about the developer, they clicked the button, they didn't see more info.
An app crash on most devices isn't actually a particularly dangerous problem. Start the app up again.
Arbitrarily deciding that a particular bug is worse because it causes more harm to the software rather than because it causes more harm to the user is precisely the problem. The importance of fixing those two bugs is probably exactly the same (unless the app crashing causes data loss or corruption or is exploitable or... you get it. Of course, the same could be true of the 'no more info appears' bug - behind the scenes it might be starting an infinitely retrying string of HTTP requests to download the developer bio that gradually leak memory and drain the user's battery. Maybe that's really the more serious bug?)
There isn't some separate measure of 'severity' that independently makes the crash bug magically worse than the functional failure. There's only the actual consequences of the bug. For the user, their data, their time their resources, and your business goals.
> Your job is not to build the perfect/best product, it's to build the one your client want and is willing to pay money for.
So why have severity and priority as two distinct dimensions then? If customer impact is the only relevant metric for your process, why would a bug about which the client says "that's fine for me" ever be considered severe or critical?
If the only metric you care about is customer impact, why track two of them?
On the other hand, when you have multiple different users..
One of them sometimes suffers a crash that also loses all their work, and it really pisses them off. That's got to be bad, right? I'd like to call it severe based on its impact to that user.
But then the vast majority of your users have never seen that bug, and the bug is damn difficult to track down, so maybe you can't treat it like a stopper.
Then you are back classifying it by impact x probability... What is a different set of dimensions and make much more sense to use basically because filling those 2 dimensions tends to be easier than filling a severity dimension.
But that says nothing about why one would want 2 unrelated priority dimensions.
Because severity is usually a general measure (eg, to use an example from application security: XSS is usually a sev:med), but priority can be subject to a bunch of forces not known to the person who found the bug. You might find eg an exploitable dangerouslySetInnerHTML, but if that’s on a separate domain (in the authn and the origin sense), in some back-office page somewhere, it may not be particularly exploitable, the impact of exploitation may not be particularly high, and so may justify a lower priority. Meanwhile, lack of CSRF protection on an endpoint that half the internet is engaged with may be a higher priority, even though CSRF is typically lower sev.
Maybe the page with the bad bug is getting shut down next month anyway and we’re just accelerating that instead. Doesn’t make the bug any less bad: definitely makes fixing it less of a priority.
(In hindsight I should have used SSRF or SQLi or RCE as examples of the former style bug and XSS as the lower severity of the two to emphasize the difference.)
Well, good, but if customer impact is the only metric that counts, what relevance does severity have?
If you find a RCE vulnerability that never affects your customer, and how much it affects your customer is the only metric by which bugs are prioritized, what relevant information does severity add?
Either it affects your customer, in which case you prioritize a bug's fix by its severity (i.e. simply by how much it affects your customer), or it doesn't affect your customer, in which case you prioritize is by... how much it affects your customer. Either way, you end up prioritizing bugs just by customer impact, which is either measured by the bug's severity (if it affects a customer), or zero (if it doesn't, regardless of severity).
Just to be clear, I am absolutely not arguing that severity shouldn't be tracked, I'm arguing against the idea that purely functional customer requirements should be the only criteria used for prioritizing bugfixes and development tasks.
It's certainly true that "your job is not to build the perfect/best product, it's to build the one your client want and is willing to pay money for", as the parent mentioned. But it's also your responsibility to do the building in a sustainable manner.
Decoupling severity from priority is a great way to run a codebase into the ground. RCEs that are unexploitable today will be expoitable five years from now, after sufficient requirement changes accrue -- and at some point in the future, you're going to have to deal with five years' worth of previously unexploitable RCEs that you have to fix yesterday.
Edit: I've worked for several years on two codebases where customer impact has been the primary metric used for bug prioritization for 15+ years and 7+ years respectively. Both were absolute catastrophes. You could literally crash devices by blowing hot air on them (literally, there was an out-of-bounds access bug in the code that handled overheating alerts and they crashed after a few alerts). An entire team was there just to put out fires, virtually all of which were known bugs that had at one point been deferred because they had "no customer impact" -- until they did.
It's been a very useful experience. Disentangling these things is a very useful skill to have for an expensive, and highly sought-after on the market.
You need both because severity and priority aren't necessarily set by the same people who have the same information. They're (of course) not completely orthogonal: generally there's a 1:1 map between severities and priorities. I think we're in violent agreement: severity is also a close proxy for how much I think you should care about something. (We don't assign severities to hardening tasks, since they're, well, hardening, not bugs--but there are plenty of hardening tasks that I think are more important than vulnerabilities and should be prioritized sooner. E.g. "unencrypted EBS volume" is a finding I will totally let you ignore, but I will also be on your case every day if you're not using aws-vault.)
So, to rephrase: you need both precisely because that's often the way you can even start having the prioritization conversation.
We're in perfectly peaceful agreement, it's not even violent :-). As I mentioned in my post above:
> Just to be clear, I am absolutely not arguing that severity shouldn't be tracked, I'm arguing against the idea that purely functional customer requirements should be the only criteria used for prioritizing bugfixes and development tasks.
> "if you click anywhere in the left half of the screen during the 5th screen all the settings get deleted" might be a very severe/critical bug, but if the client says "thats fine for me don't worry, but can you fix the wrong color on screen one where the logo is purple instead of fuschia that's not acceptable for my needs" then this one has a higher priority, even though it's much less severe.
The problem here is the made-up definition of severity.. basically, you decided that crash or losing settings equals severe and cosmetic stuff equals not severe. Why?
I could easily give it a different definition. For example, a crash that nobody cares about is not severe. A small "cosmetic" slip up that can cause big damage to brand is very severe.
So I agree with the GP here; why do we need to have a definition of severity that does not align with the things that we care about when it comes to actually fixing things?
They're both measures of importance, but evaluated at different times by different parties. In the article's formulation, the QA engineer sets the severity during the initial investigation based on technical criteria. This is then one of the datapoints that the PM uses during triage to set the priority, which is the controlling value from that point in the process onward.
I guess the underlying source of confusion is that they're both measures of severity, but priority is severity from business viewpoint while severity is severity from the affected user's viewpoint.
A high priority bug that isn't fixed presumably has severe consequences for business (or it was prioritized wrong).
A high severity bug that isn't prioritized presumably has low consequences (severity) for business, but it probably really sucks for the unfortunate user who is affected.
Here in pivotal-tracker-land the product manager controls priority by ordering stories in the backlog. There is no need for a separate 'priority' field because that is implicit.
FWIW, there's also no explicit 'severity' field; the PM is expected to understand all the factors that make issues important and order the backlog appropriately. If you need more categorization, you can apply arbitrary labels.
I think it's helpful to think of how bad a cosmetic issue could be. For example, imagine that it's a big launch of a product people will be using in stores worldwide and the cosmetic issue is that the logo now consistently looks like a sex act because of missing letter or development art left inside the release. Also imagine that that act is relevant to a recent revelation in the news about the CEO's personal life.
Which do you think the CEO cares about more, the logo or the fact that it crashes sometimes on an infrequently accessed menu?
> basically, you decided that crash or losing settings equals severe and cosmetic stuff equals not severe. Why?
Because you, as a developer, are clueless regarding business needs, and thus are unaware of why "cosmetic stuff" might be far more important than the risk of deleting someone's user profile. For example, perhaps the color scheme clashes with the one used by your client's main competitor and therefore might leave him vulnerable to lawsuits.
Yes but what is the point of severity _and_ priority? Why not one field that's first estimated by QA and then updated by project manager when the clients needs are known?
So that they can be tracked independently and reviewed later. As I explain in a comment elsewhere, my company uses an app to generate severity, and it may not be adjusted outside of that. We can then track the number of low or high severity bugs in a delivery regardless of how the customer perceives the impacts of the bugs, using a more-or-less objective measure. We can compare that to the customer's view of the quality of the delivery by using the number of low or high priority bugs.
Makes total sense and my team does this as well. I think calling the value completely perpendicular to fix priority is hyperbolic. Fix priority should be some combination of severity, frequency, effort and stakeholder desire.
We have dozens of customers worldwide for our software packages, and each package is highly customised for each customer's business. The severity measure lets us compare release quality across customers using an objective measurement defined and managed by us. The priority measure lets us refine that comparison per customised packaged. Generally, a release with a lot of high-severity issues will have a lot of high-priority issues (since by a default an S1 is a P1, an S2 is a P2, etc.), but some customers have different requirements, different custom features, and some are just more fussy.
If a base release that was customised for multiple customers has an expected number of P1, P2, P3, P4 issues for most customers but a high number of P1 and P2 issues for one particular customer, off of the same number of issues in the base release as measured by severity, then that will stand out in our measurements and we'll dive deeper into that customisation to see what's going on.
(Edited mid-stream... accidentally submitted too early.)
FTA, severity reflects "a bug’s impact to the functionality of the system", while priority reflect the
"client’s business and product requirements".
The point of this system is that high-severity bugs might have lower priority than low-severity bugs if you take business requirements into consideration. Yet, this does not mean that severity should be ignored.
You nailed it. Stick with priority and the right stuff will get fixed. Severity just encourages more debate about what needs to be fixed vs. deferred. Inevitably you will end up with a list of defects which will never be fixed.
The person you are responding to is saying, yes, "cosmetic stuff" might be far more important. So it's more important! Why have another dimension of assessment where we label it less important? Why not only have the dimension of assessment that actually matches the clients' needs?
Because there is expressed client needs and real needs. They say they care about this cosmetic thing now so it better be fixed. However you well know that if you don't get this other thing fixed soon internal politics at the client will mean they throw you out. Thus you fix the thing they demand be fixed now (it may only take a few minutes), but you ensure the other things are fixed in the next release even though the client doesn't know they care yet.
Sure, but you don't need separate priority and severity scales to do that: it's just one priority scale but you just assign the priority not entirely on the clients expressed needs but rather also factoring in your own assessment of their needs.
You don't need that, but you are not everybody. When you have a large organization having a simple way to capture this type of thing and make it clear what you are talking about matters.
Of course it does add complexity. It is the call of each organization which things are important enough to be worth the extra complexity and which are not. Maybe for yours it isn't worth the extra cost - there is nothing wrong with that - but other people have different needs and so they will have different answers.
In short, you are wrong in thinking there is a universal right answer.
Like I said in my other comment, because it makes the difference between controlled and uncontrolled.
Eg "the color is wrong because we specified it wrong" and "the color is wrong because the app doesn't respect what we ask it to display" both ends up with the same bug (wrong color), same priority, but not the same severity because the second case is uncontrolled.
Severity is a dev tool, priority is a business tool.
So instead of a severity rating, you are saying severity is encoded in the language of the description? Using descriptors a potentially non-technical PM can understand unambiguously?
I'm not saying this is the wrong approach by the way, it's just interesting how people approach this differently.
It's not another dimension. It's a classification. Some issues matter more from one perspective but might not justify allocating resources to address them from other perspectives. To be able to do an adequate job prioritizing issues, you need to take in consideration their classification. You're arguing about a desired outcome without paying any attention to the process that leads you to that outcome.
that's a strong argument for having a single priority... Even (generously, IMO) assuming severity represents a tangible thing like a threat to code quality or system stability/debugging, a developer should not be the one trying to balance those internal demands against a customers priorities. The important thing here is that devs know what needs to be done. Distributing that arbitrarily across two fields obscures that.
> Because you, as a developer, are clueless regarding business needs
Then that's the problem we should fix. Instead of creating an extra database field to capture somebody's incorrect opinion, the people who understand priority should be helping developers know enough to have useful opinions.
Hiring good programmers who are not too far out on Asperger/ introvert scale is an issue. So you can fix it by letting programmers only worry about the tech part and letting PM prioritise things.
I think motivation will not be as high but it is a way to get things shipped and profitable.
Speaking as someone with Asperger's who considers himself both a good programmer and capable of navigating / leading cross-functional prioritization discussions, and who likes knowing the context behind his work: maybe you should re-evaluate your assumptions about neuroatypical people.
(...and if you're indeed in a position where you're responsible for hiring decisions or performance reviews: strike "maybe" from the preceding sentence.)
Agreed. Non-neurotypical people may or may not need to approach understanding the context differently than neurotypicals. But it's not like we're incapable of understanding the context!
I work in a niche field. Not so niche that there isn't a lot of money in it, but niche enough that we need to explain to all of our new hires what we do as a company.
For us, it is absolutely 100% necessary to hire domain experts to prioritize bugs and features. It's not a question of incompetent or dense developers, it's a question of things that are not obvious to someone who doesn't have tons of experience in the field.
It's a problem that I imagine developers working on Chrome, Call of Duty, iTunes, or Outlook don't have. You can hire recent college grads and expect them to understand what the software does, have reasonably good instincts how to prioritize bugs and put together the right user experience even if the description in the feature request is sparse on details.
By the way, I heartily recommend working for such a company. My company works very, very hard to retain people. Someone who's spent ten years getting used to the weird stuff our customers expect is far more valuable than someone with half the pay who needs someone to hold their hand through every single issue. Everyone has their own office, management is extremely permissive about the shit that doesn't matter, there's never deadlines or crunch time, everyone chooses their own work/life balance. (We're hourly, and the expectation is that you work more than forty hours a week and need manager approval if you want to work more than 60 for more than three pay periods in a row) If we want more vacation, we can bank hours and spend it on supplemental vacation. Everything's great.
There's nothing to fix. Developers assess severity but the project manager defines the priority. As the PM calls the shots, they pick which issue should be addressed first, and obviously it's expected that some low-severity issues should be given priority over some high-severity issues.
In fact, the only thing that needs fixing is any potential misunderstanding on behalf is some developers on how low-severity issues shall have priority over high-severity issues.
Because a crash is different than a button rendering with the wrong color, and although priority assessment might change with time, a crash is still a crash.
It seems that a recurrent theme in this thread is that some developers have a hard time understanding that PMs might have solid reasons to prioritize low-severity issues over high-severity issues. It's like these developers are stuck with a mindset where the forest doesn't exist and only the trees they personally have direct contact with should.be worth any consideration.
Why set a severity if you're not going to use it? A crash is still a crash if you don't set the severity and just write that it's a crash in the bug description.
I know some people look at me like I have three heads every time I say this. But if a project is dropping so many balls that it's hard to keep track of them all, I think the real solution is to work smaller and close feedback loops faster, so the sheer number of bugs is not overwhelming.
Have you worked at small companies all your life. There are teams with 50 devs in some places and shipping huge multi-hundreds of million dollars pieces of software that is in a lot of cases, legacy software. You dont get a call in deciding to work smaller with closed feedback loops all the time.
Your opinion on this strikes me as you being a junior in the software dev world.
> Why set a severity if you're not going to use it?
Your question misses the point entirely.
The point is that severity is an element that's used to classify an issue with regards to priority. Severity does not exist in a vacuum, and priority is given depending on context. Severity is an attribute, other attributes, that's used to determine the priority.
It's only used to determine it in a way that's divorced from the business context. If everybody understands the business context, that's no longer useful. Ditto if people are collaborating with actual discussion, rather than trying to mediate communications via a database.
> It's only used to determine it in a way that's divorced from the business context. If everybody understands the business context, that's no longer useful.
That's the point, everyone does not understand the business context. Nor are they expected to. That's the job of the PM it's his main responsibility, and it's the reason PMs are hired to manage teams of developers.
I understand some organizations work that way. I'm saying it's bad.
The point of developers is to make software that people use. So if we want to do our jobs well, we have to understand how we are creating value. Product managers may manage that information flow, and they may get the final say in decisions. But if they are specifying software in enough detail that developers can be 100% ignorant, then developers can (and should!) be automated out of that work.
I think this thread is interesting and kind of funny, because it reminds me of work where I maintain some systems that keep track of projects for PMs, and I originally thought my job was to make everything consistent. But there are a whole slew of ways to express the "closedness" or "openness" of a project, and the PMs have evolved conventions where they want to be inconsistent and resist all efforts to make it all make sense. You have a project status which may be in progress or closed or something else. You have a closeout milestone which may be in progress or closed or something else. And you have a time entry code which may be open or closed. But it turns out there is no simple way to make these consistent, because people use inconsistent combinations to express things...but it's hard to tell what.
You guys are missing the corollary that low-severity bugs being escalated to high-priority is the edge case.
The point is that severity is not ignored; it does inform the priority. Most of the time there may even be a direct correlation between severity and priority.
But other (real-world business) factors also inform the priority; while severity will never change in the absence of new information about the bug, those other factors may change frequently. It doesn't make sense for a PM to reread every single ticket and reassess each one's severity when adjusting priorities, when the developer can just determine that once and record it in the ticket from the start.
Complexity sounds to me like more of an implementation-level concern.
e.g. A bug might be critical severity if it wipes the entire production database, but low complexity if the fix is to delete one line of code. And maybe its priority is P1 instead of P0 because the customer said they'll remember how to avoid triggering the behavior but they really need that logo color changed asap for an important demo.
The point i was trying to make is that the severity hardly changed the priority, if its user impact is low. But then that means the severity isn't high either!
Where are you getting "hardly" from? In this example, it normally would have been P0 (release-blocker) but was downgraded to P1 (still the second-highest priority) because of a special consideration on the customer's end.
The point of severity is that it's an objective metric determined in isolation based on specific technical guidelines that any developer on the team can follow (such as https://www.chromium.org/developers/severity-guidelines). Whereas priority is a higher-level determination that isn't purely technical.
It's like the difference between body fat percentage and attractiveness. Any "engineer" (doctor with a DEXA scanner) can tell you your BF%, and attractiveness will typically correlate with BF%, but ultimately your "customers" (romantic partners and targets) decide how attractive you are. Not a perfect analogy (priority is still something you'd decide internally, not your customers directly), but hope that clarifies things.
> Then that's the problem we should fix. Instead of creating an extra database field to capture somebody's incorrect opinion, the people who understand priority should be helping developers know enough to have useful opinions.
The priority database field is how you communicate this factor, but there is a point where reasonable people can disagree and the organization needs a way to make decisions clear.
To draw out the example even more: you could have a $600K invoice riding on customer acceptance, when the person at the customer site who signs off won't do so until the customer's logo color is correct. While that crasher when you enter a non-numeric character in a numeric field of a new feature? "We accept the functionality meets the milestone so will sign off but we don't plan to roll it out until next quarter after we have begun training personnel on the new feature."
Sure, every good organization should want everybody, not just developers, to understand the customer's business and such but sometimes you just need to get it shippable.
Why? What is the business value of having severity as essentially a protest field logged by people who don't understand business impact? You are basically in all your points explaining the business value of priority which nobody ever disagreed with and then going "and that's why we need two fields".
It could be useful if the folk prioritizing things are dealing with non-specific complaints about the software being unreliable or not working correctly.
> Why? What is the business value of having severity as essentially a protest field logged by people who don't understand business impact?
I don't understand where you could possibly get the "protest field" idea. Severity is an objective statement regarding the known impact of a bug as verified by a developer. It summarizes the technical impact of a software defect. Stating that bug X is high-severity because it crashes is not a protest, and just because the PM decides to give priority to other more pressing issues it doesn't mean you should throw a tantrum.
What is the 'technical impact' of a defect, and how can you divorce it from the user impact? How can it be stated objectively?
Crash bugs aren't bad because crashes are inherently bad, they're bad because they have negative user impact - if the program crashes and loses user context, or data, or takes time to restart... those are bad things. If it crashes a little untidily when it receives a shutdown event from the operating system... maybe not so much.
Same goes for performance issues, scalability problems, security flaws, even badly structured code - they don't have technical impact unconnected to their user (or business, at least) impact.
> What is the 'technical impact' of a defect, and how can you divorce it from the user impact?
TFA provides a concrete definition and also the method to classify bugs based on severity.
Severity does not divorce a bug from "the user impact". There is, however, also the problem of low-severity bugs or even tasks having low user impact but high business impact.
But that's a contradiction. Unless the users aren't important (and the business is another entity, e.g, a CxO that has clout and demand a fix for a thing that users don't care about).
Databases are a very bad communications medium. So if that's the major way devs and and product people are conversing about issues, it's no wonder the devs lack sufficient understanding of business context to understand what the real priorities are.
I do get that people have all sorts of adaptations to dysfunctional working conditions. So if a severity field is one of them, fine. But I don't want people to mistake that for healthy collaboration.
Are they? That's how majority (all?) systems that are asynchronous work. The data to be communicated has to be persisted. I think asynchronous is a good communication method.
Playing devil's advocate: Why does the developer need to know why one bug is more important than another, if the priorities are clearly defined? I.e., if the backlog manager sets the priorities according to the customer's/business' needs, then the developer just needs to know that the cosmetic bug has a higher priority than the crash bug, but they don't need to know why the priorities are ordered that way to accomplish their tasks. And if they do want to know for their own knowledge, they can just ask someone who understands the needs, without the need for a more complex set of bug report attributes.
I agree with your overall point I think, but I find that people in general just work better when they have some context for their task and its priority/relevance. Absent that, they sometimes -- consciously or not -- decide "this is stupid" and either rush to just check it off or slow-walk it by allowing themselves to be distracted by other things.
If the color is indeed the one set by the dev/designer/... and it's juste not the right one, that's a controlled thing, it's not severe. The app is doing what you want it to, you just gave it the wrong instructions.
The crash on the other hand is uncontrolled. That's severe, because it's a case where the app is not doing what you want it to.
Severity is something for the dev team internally, priority is something for the product manager and business people that the dev team follows.
When you start mixing the two in a single thing, the dev specific needs are always what goes out the window first.
Devs inform a ticket with a severity as a counter balance to bug fixing being purely complaint driven. The business benefits from having someone with a technical eye look at a bug and determine if it really is just a cosmetic issue or may allow system misuse - then the PM should take that severity into consideration along with client complaints when handing out priority.
> For example, a crash that nobody cares about is not severe.
That's a bad example, something not severe is not severe... can't argue with that. A crash nobody cares about can be quite severe though, there's quite a bit of crash that has been used to make some of the most important security vulnerabilities. When I tried to reproduce the Dirty CoW vulnerability over a VM in a school setting made the teacher VM crash multiple time, he didn't care about the crash either ;), yet that single vulnerability allowed me to skip 90% of the vulnerabilities he wanted us to try to find.
I think your comment point something important that you miss too, different client may have different priority, and you too may. I think severity is closer to YOUR own priority. The severity is the impact this MAY have on your business if it wasn't fixed. Like you said, a small "cosmetic" slip up may be important for our brand, thus severe for us. That "cosmetic" slip up could still not be important for all of our client, like over a one page made to sell, yet still be on top in both priority and severity, because it matters for the next client or the sale team, which does matter to you.
> So I agree with the GP here; why do we need to have a definition of severity that does not align with the things that we care about when it comes to actually fixing things?
You could say that about many information in a ticket. It is still informative though, maybe it's no longer needed, but at one point it did was relevant and in many case, still is.
A ticket with an higher severity is something that you need to put more time on to decide whether it deserve an higher priority. I'll go back to your crash example, why would you works on a crash that nobody care about, why would you see "crash" in the ticket with so many steps to reproduce and believe that this deserve an high priority when it never going to happen? In an ideal world sure you would know 100% of the system and know 100% of the tickets and can understands 100% of the impact of this kind of crash and be able to fit it well in the priority, but let be honest, that ideal world doesn't exist. You have a limited time that you can put on each ticket to decide their priority, you have a limited capacity to understands the impact and everything, you have hundreds of tickets to go through, but something that you can do is see that the high severity set by the developer means that you may need to put more time on that ticket to choose its right priority.
Sure once the priority has been set, does the severity matter? That's a much bigger question that I won't try to answer, probably not, but did it matter at one point? It did. Would you remove any information from a ticket because it no longer matter? I sure hope not.
Your job is not to build the perfect/best product, it's to build the one your client want and is willing to pay money for.
If you tell the client that they're wrong and that you should fix the critical bug first how they react is a really good test of whether or not they're going to be a good client. If they insist on fixing the lower priority bug first then you should try to stop working with them, because at some point in the future they're going to demand more unreasonable and stupid things that blow up in their face, and you'll lose that client anyway. Save yourself the stress of working with bad clients and get rid of them at the earliest possible opportunity.
If they accept that you know what you're talking about and give you agency to fix things in the order that you think they should be fixed, do your absolute best to keep that client happy because they'll be a joy to collaborate with for years.
> If they insist on fixing the lower priority bug first then you should try to stop working with them
I'm all for firing bad clients. Fire the crazy ones. Fire the ones that don't pay. Fire the ones who make ridiculous demands at the last moment.
But the client wants you to deploy a fix to the logo ASAP, you do it. That's the client's priority. As long as they accept the disruption to the rest of the work, it's no problem.
I think ultimately it boils down to what you want to glean from the data. Not all crashes are mustfix, and some cosmetic issues really are mustfix. If you're trying to assess the priority of two bugs, both are p1, but in your world are marked crash and cosmetic, you would choose the crash. But it might be that the cosmetic but is related to a sponsor/third party/branding/legal issue, which warrants fixing immediately. Having a "cosmetic" tag on that doesn't help you, but a severity of 1 vs a severity of 3 on an edge case crash on shutdown _does_ tell you.
I agree. My experience with the issue is that, while the distinction may exist academically, the reality is that most people don't grok it. The result being that they are so frequently misused that, in practice, they are indistinguishable dimensions.
Basically, if what you are trying to do is too complex/nuanced for everybody to understand - don't do it.
The two values are good to separate if they come from different people/groups.
A developer or tester might be good at specifying how severe a bug is: impact (does everyone get it, or just some users) multiplied by frequency (does it happen in commonly used functionality?) multiplied by a category factor (is it a crash, cosmetic, dataloss...) will give you the severity.
Some completely different person can come along and give this a high priority because of business reasons, and to prioritize fixing the bug compared to other functionality. Two bugs with the same assigned severity might impact two different markets where the business knows one is critical for the survival of the business, and one is already a safe market. Then it's a priority to fix it.
A second reason might be due to ROI (easier severe bugs are better to prioritize than difficult bugs of the same severity).
Yes exactly -- it's two different sources of information that one party is not likely to have access to. Severity from a technical-only perspective is important to have. The sense of whether something is easy to fix (which should be size) is also important.
Ultimately you could have a low-severity bug, but it impacts a really important customer who cares about it, so it's high priority. This is the most common case I've seen.
Severity also can impact whether something gets prioritized at all. If it's low-severity and the business side isn't there either, maybe it just doesn't get done. Whereas something with high severity but low business impact still probably needs to be done regardless or it will lead to bigger problems.
Usually the other orthogonal assessment in addition to severity is frequency. If the bug is unlikely to be seen during regular use then even if severity is high the priority might be lowered. This is usually how risk is managed in my experience.
Agreed. 1 dimension can represent the impact to an individual user, and the other dimension represents the pervasiveness (frequency) in the user base/market.
We have applied the same metric to feature request prioritization. Severity can be thought of as necessity for an individual customer (must have, should have, nice to have, could have, won’t have) and frequency represents an estimation of demand in the market.
We even created our own Jira plugin to calculate the product of these and use that number as a self prioritizing index.
> I don't see the benefit in having severity and priority as 2 distinct dimensions
Working directly with PO's both "Technical POs" and "Product POs" I've found that creating a strong distinction between the two as well as what the two dimensions define has a significant impact on the quality of the product over time.
In the original post we see that both Priority and Severity are two different dimensions each representing a scale. This is what I think is causing a lot of the discussion in this thread.
Because both of these values are a scale we are creating a foundation over which we can get confused and we find ourselves needing to differentiate these two in some way. This comment is a good example of the confused conversations I would often see. All of variations on the same flavour:
> If the goal is to better communicate what "type" of bug it is, using an enum classification (eg, cosmetic vs crash vs data-corruption) would be more appropriate than a numeric scale.
To get around this in Agile contracts I've always advocated the following:
"Severity" scale. Severity of the issue on the user from 1-5. (For Jira users imagine this as your 'Priority' field renamed to severity.
1: The user is unable to complete the journey or part of the journey
2: The user is unable to perform an action, but can complete the journey
3: The user is able to perform an action but...
4: The user is able to perform an action but...
5: The user is able to perform an action but...
Of course, you can put in your own definitions. The important thing being that this is defined from the 'users' point of view rather than attempting to define the nature of the issue (crash, freeze, etc...). The nature of the issue can be discussed within the ticket itself.
The second dimension 'Priority' instead being thought of as 'Ranking' and is not a field which is added to the ticket. This defines the order in which the issues must be dealt with. Thus allowing a PO to appreciate the severity of an issue and also negotiate with the team it's relative important through ranking.
Priority/Severity is a good, useful way of describing things.
Buying clothes detergent because you ran out and have no clean clothes is high priority, low severity. Booking cancer screening sometime this year because you're reaching that age and there's a history of cancer in your family is low priority, high severity.
As explained, the severity is the impact on the system. For example, a null pointer that causes the system to crash will have a critical severity.
But maybe this bug occurs very rarely and in very specific circumstances.
On the other hand you have also found another bug that makes your app use bright green Comic Sans. It is not severe as it is only cosmetic.
But for some reason, on your last release this happens every time the app is started.
It seems likely that the second issue will have a much higher priority and will have to be fixed immediately, while the null pointer can wait.
Severity is purely technical. Priority is a project/commercial decision (which is informed by severity but also other considerations). They generally do not align.
Severity can give you a prioritization within priority categories. Fix the severe high-priority bugs first, then the less severe ones, then on to medium-priority.
Of course, scheduling is often dominated by other factors, but it gives you a default at least.
It's always a scrummaster type who gets back from some training and wants to add a ton of JIRA overhead like separating severity and priority. I've ignored it for 15 years now and it hasn't bit me once.
I agree. To assign a priority, the PM has to know exactly what happens in context. "Severity" doesn't help with that or with any other stage of reporting, tracking, fixing, and testing.
I could see where a severity dimension could be useful retrospectively if paired with classification. For example, you could look back and say, why did we have a big spike in severity 1 security bugs in Q4? However, I think I'd rather not have the dimension available than be forced to set it on every single bug on the off chance it will help with analysis later. "I bet we can do cool stuff with this someday" is never a good reason to add process overhead. If somebody retrospectively categorizes and summarizes tickets by some dimension and proves that it yields useful information, that could be grounds for asking engineers to track it in the future.
Priority is useful for... Prioritization. One can autoassign a priority for an enum classification.
I've never seen impact/urgency/severity as being anything other than different ways to say priority or overcomplicated ways to derive priority. Then again I've never wrangled an absolutely massive bug/ticket queue.
Lets say you have 10 bugs and they are prioritized in order from 1 to 10. Bugs with the priority 2, 3, and 6 have been given a severity of P0 (a ship-stopper, as the original post described it). Bugs 1, 4, 5, and 7-10 are not considered P0 bugs.
Given those priorities and their relative severity, that means the team is committing to deliver, at least, bugs 1-6 prior to the next release. That makes severity a pretty useful dimension to consider on its own and not just as consideration when determining priority.
On top of what others have said, one case where I've seen value in them being separate is to workaround problems in long gaps between deployments.
A bug might be Major or Minor but there is a high risk to fixing it. You might want to do it early in the development process, ahead of higher severity issues, so you get the most runtime on the fix before the next deployment.
At my work, crashes and even data loss situations are often given low priority and high severity. That's because we just tell the user the workaround and he can keep doing his job.
Low severity and high priority can be:
- it breaks the software for just one person, but that person is on the critical path.
- if breaks users workflow. For example a keyboard shortcut that is used every second stops working.
- the bug is an easy, high value fix, so why wait?
They provide some distinction between the impact and how soon something is to be dealt with. But in many cases, they tend to go together in a proportional manner, at least for the top values of severity and priority. And in all the cases I've seen, people understand them as directly linked together and never try to assign different values (that seem to crisscross if you put them in a two column table).
- "severity" as level of importance the tester assigned it when they found the bug and
- "priority" as level of importance a developer assigned to it after triage.
That first piece of information is important when you need to determine which bugs to look at first for triage, but how important is it after that point? Can it not just be replaced after the developer's judgement?
In other words, could you not have a single priority field? The tester uses a heuristic to assign an initial priority (e.g., crashes are P0, cosmetic are P4). The dev uses this to prioritize which bugs to triage first, and once they've determined a new priority based on customer experience combined with app behaviour, they replace the old one.
If you really need to go back and check what the tester assigned, then I assume you can just use the "history" or "revision" feature in your bug tracking app.
Additionally, as suggested in a different comment, you can add a label for the bug's type if you feel that's important (crashing, lagging, cosmetic, etc.).
Perhaps the message here is that the app's behaviour in a vacuum is not the sole determinant of its priority. But then that should be the message, rather than claiming there is another metric which needs to be separately tracked when evaluating bugs.
I think the formula should be - severity * (how widespread is this) - ease of workaround = priority. So if any of those measures change (e.g. an easy workaround is discovered, it's determined that the bio page that is crashing is also almost never viewed) then the priority should be adjusted. Having just severity without a measure of 'how many people does this impact?' and 'just how badly does this impact them?' seems like it's missing part of the picture.
But I think it should be similar to risk calculation, multiply by the "impact" or "damage". A typo might offend a small number of people but have a big publicity impact
Honestly, I've found that testers are poor judges of priority. They get emotionally involved in their bug and don't really make a good judgement of the priority to the business.
That's why the split between priority and severity exists: The tester is telling you the subjective view of the bug as if a user encountered it; and the triage process is all about judging the priority of fixing the bug in this particular release cycle.
Testers are not supposed to judge priority, that's a business decision (that could be based on technical know-how) and as such should be decided by business people aka PMs
> when you need to determine which bugs to look at first for triage
If you can't at least do a rough triage of all the new bugs in one sitting, you're either not allocating enough time for triage, or letting yourself get way too bogged down with ancillary conversation during the triage meeting, or it's just time to scrap the product and go home.
If you can stay on top of your triage, then there's not really any need to worry about what order you do them in.
Severity is how the reporter sees the problem: if it interferes with their particular use case, it's of the highest severity. Almost all bugs end up being of the highest severity.
Priority is how the developer (or, more likely, the project management team) sees the bug. Highest priority bugs are the ones in the project manager's spreadsheet and ended up there because of the most vocal complainers (paying customers, an inconvenienced CEO, etc). The rest will never get fixed.
It's impractical to use the severity of the bug to rank the priority, because they're not strongly correlated.
My gut reaction to the headline and the article was that it's a mistake to make such a distinction. However, if you look at QA/Support owning severity, and PM owning priority, it potentially reduces conflict--what do you mean this crash isn't critical?!
I think they are highly correlated, but they occasionally diverge.
This is confusing. It amounts to saying ‘bugs have an inherent property called ‘severity’. We explicitly track it in our system. It has no effect on how we prioritize work’
If there are people on the team who insist on objectively classifying bugs with no reference to customer impact, you don’t need to indulge them by giving them a meaningless drop down to play with in Jira, you need to get them up to speed with understanding your product and release priorities.
My company's measure of severity is measured by impact to the end user as well as to the customer (retail enterprises). However, the customer's priority may vary from the severity based on a number of factors.
I work on an embedded system. We had found a major bug and labeled it as "stop ship" because we didn't want anything more getting out. It is better for the factory to build nothing (paying all the assembly line workers to stand around if required - though more likely build a different product) than to have one more customer see this product and encounter this issue. I don't know the details on this issue, but there have been software issues that could kill people in the worst case.
Anyway, the developer got this stop ship the day before Thanksgiving, so he worked all weekend and Monday morning turned in a fix. At which time the factory manager told him "thanks, but one of the parts to make this machine is on back order for 4 months so we weren't planing to make that product anyway".
Google's internal bug tracker has both priority and severity, and it's kind of a running joke that severity is meaningless (because it isn't meaningfully different from priority). On my team for example we leave it at its default value and ignore it completely.
We switched to P1, P2, P3. Why? A company that bought us used P1, P2, and P3, and they just couldn't comprehend that most of our "P1" bugs were really P2 bugs under their system.
My take is not to split priority and severity, but priority and tags. For example P0 typo,UI and P2 crash,UI ...the crash might be P2 if if's not severe for the product itself, as in the example, if it is on bio or whatever page.
Tags/label have been working quite well for me. Priority can be replaced with milestones, but you have to create them for each releases to ensure P0 are quickly enough.
The post doesn't answer the biggest question, which is "What would I do with this additional information"? In concrete terms, what actions would your team take for a P2S0 that would be different from what they would do for a P2S3?
If anyone has a workflow where "severity" is important for your decision-making, I'm curious to learn about it.
Low severity bugs that are hard to fix sometimes get closed as "won't fix". It is unlikely any customer will hit the area where we discovered to breaks and after several weeks we still don't have a clue - why spend more money fixing this issue?
Unless I'm missing something no one has mentioned the effort to fix a problem? I've seen cosmetic issues (using the wrong masculine / feminine form in French place names for example) which would have taken a huge amount of time to fix correctly and internal server errors which where the result of someone accidentally including a comma after a value in python, turning it into a tuple where one was not expected. At some point that should be factored into the order of priority.
The effort may indeed affect the priority, and it also is potentially a factor in severity. If a customer knows that a high-severity issue will take a long time to fix, they may decide on a mitigation for a particular delivery and lower the priority so that the fix comes in a patch or later delivery. The severity measure still counts in determining the overall quality of the release, though.
In my area of software, we provide customised solutions to large enterprises. We have an app that takes in a lot of dimensions about each issue report and spits out an "objective" severity. This app has been honed over 20 years of reviewing issues. That's the only severity that's allowed into JIRA. By default, the priority is set the same as the severity. When we review the issue list with the customers, however, they are free to adjust the priority according to their unique business needs, and developers work on issues in priority order only, not severity. It's a good system, and it ensures that we're working on what the customer needs and not our isolated view of severity. Granted, we will consult with the customer during triage and give them a steer based upon our experience, but they have the final say.
I wish I could get my organization to explore this a little more. We tend to use a single priority value. But it doesn't feel like prioritization to me. If you have 100 issues that all have the same priority level, that means you don't care what order they're dealt with in - which is never really the case. There's invariably another level of "prioritization" that has to happen within each level. But instead of being codified, it gets left to casual discussion, or worse, it never gets discussed.
This has led me to think that a hierarchical priority structure would be useful. So we could say that these 100 Level 3 issues are generally of the same priority, but there's a ranking within that. But even the highest among them is still less than the lowest Level 2 issue.
That's not to say that it shouldn't also be 2-dimensional though.
I have worked in a development team that had both priority and severity. The main difficulty I encountered was that people would silently factor one into the other.
Some would factor severity into priority (This is a crashing-type bug, so we should prioritize it), or priority into severity (This bug only causes minor harm, so it's not a high priority for us). But this was done silently and unconsciously, and different people had different directions for how they would do it. Due to this experience, I think one metric is best.
I think the issue here is that "Severity", as used here, isn't a good label. It sounds like they're mapping the kind of effect the bug has on the system to "Severity" values. E.g, a crash and hang maps to Severity "critical" and typos map to "minor".
The problem is that "Severity" is being used for something that doesn't really match its intuitive meaning.
Maybe call the field "kind of impact" (or something similar) and have enumerated values or tags like "crash", "hang", "typo", etc. Now the field is intuitive and you don't need to retrain (or write blog posts) to get people to understand your new meaning for the word "severity".
I've preferred Urgency and Impact as my terminology, rather than Priority and Severity, respectively. But yes, I've been frustrated that these concepts are often combined when they shouldn't be.
185 comments
[ 8.2 ms ] story [ 305 ms ] threadIt was just extra work/mind cycles for no gain. I don't feel like I'm missing out but happy to hear where this makes sense
Something that is Critical severity doesn’t necessarily mean its a P0. That crashing bug in your bio screen? Face it, nobody really cares to read it and few will ever click the button, so its not something that should be a P0 ship-stopper.
In what way is that an issue of 'critical severity'? The quoted text contains an explanation of why it is not.
The real difference is that severity is a measure of the harm done, while priority is a plan for to responding to it. Clearly they are different, but clearly they will likely be correlated, at least when the severity is high.
The only reason would be this: given a large enough enterprise you have one person categorizing a bug as a crash and another making the call on whether that crash is actually worth prioritizing, a decision made on another day at another pay grade in another time zone. In this case it might be clear, but in the next case it might be "the configuration page crashes if the region is France". I as a developer don't know and I don't care that we haven't even launched the product in France so I just want to mark the issue as a very severe crash. Somewhere above my head the can call this low prio because the product launch in France is still a year out. This is enterprise backlog massage for enterprise workflows.
In your scenario, you don't know whether it is severe or not, so you just want to provisionally mark it as severe, and have someone with the appropriate domain knowledge make the final call. Until that happens, decisions made on the basis of that provisional call could be bad decisions.
And what might those decisions be? Priority, for one! To say that the severity should stay at 'severe', and the adjustment should be made only in the priority, is to confuse the two distinct concepts. I see a lot of that going on wherever data are used.
If you want to characterize bugs by the sort of technical outcome they produce, then do just that, instead of trying to bend 'severity' to try and encompass it, while actually hiding the issue that you wanted to characterize on. Again, while technical outcome is an aspect of severity, they are not the same thing (and, FWIW, crashing is rarely the worst thing that could happen (unless the domain is vehicle control.))
However I have little to no knowledge of the business (markets, release schedule). I’m not even aware that we have zero users in this region. A manager I have never met assigns a low priority, two months later.
Domain and business overlaps, but for example if one doesn’t mark this “severe” because the business hasn’t launched for affected customers, then a year later when it is launched in France you have a low severity bug that should be high.
The priority can be constantly re-evaluated against business needs while the severity is fixed.
Yes this is a simple technical characterization. It’s not useless to separate the business-stopping bugs from the cosmetic. And importantly: It’s not my call to say what is important.
Someone might say that the font size is more important than a deadlock. I don’t care and I don’t want to care. But I’m rather sure that those that do make those decisions want to be able to sort the bugs I wrote by a severity.
The crash has no severity to you. It is either severe or not to the client. If you don't know whether it is severe to them, sure you can guess that a crash in france is severe, but that's an incorrect guess, not an independent dimension of severity.
There is no "techical measure of severity", everything is goal oriented. It's not severe unless it's important to the client, or is impeding something that is important to the client.
Do you call a bug that corrupts data more or less severe than a bug that crashes a program? In whichever order you put them, is the severity always necessarily in that order?
>I’m rather sure that those that do make those decisions want to be able to sort the bugs I wrote by a severity.
Why would they do that, if, as you say, "It’s not my call to say what is important"? In what way is this not you signalling that you suspect, provisionally, this has a severe impact, as I suggested in my previous post? I think you are being a bit inconsistent here.
In my experience P3's capture tech debt, cut features, and design flaws. Those should be addressed for what they are, not shoehorned into a bug prioritization system that will never even glance at them.
> It’s important to note that a bug’s priority shouldn’t be confused with its severity, which exists as an entirely different dimension... ‘Severity’ to represent a (somewhat) objective assessment of a bug’s impact to the functionality of the system. From Critical representing major crashes or hangs, to Minor functional issues or purely Cosmetic blemishes. While a QA engineer will classify a bug’s Severity at creation time, it’s the PM who assigns the Priority at the time of triage based on their knowledge of the client’s business and product requirements
I don't see the benefit in having severity and priority as 2 distinct dimensions. Given a specific priority, what additional benefit do you get from having a severity label as well? Based on the author's description, I can't think of any.
If the goal is to better communicate what "type" of bug it is, using an enum classification (eg, cosmetic vs crash vs data-corruption) would be more appropriate than a numeric scale.
"if you click anywhere in the left half of the screen during the 5th screen all the settings get deleted" might be a very severe/critical bug, but if the client says "thats fine for me don't worry, but can you fix the wrong color on screen one where the logo is purple instead of fuschia that's not acceptable for my needs" then this one has a higher priority, even though it's much less severe.
Your job is not to build the perfect/best product, it's to build the one your client want and is willing to pay money for.
The mixup between priority and severity only comes from the modern era when so many product get made for internal consumption, or where the user is not not the buyer.
« Given a specific priority, what additional benefit do you get from having an additional severity label on the bug? »
not
« Given a specific severity, what additional benefit do you get from having an additional priority label on the bug? »
Is the information about the criticality of the bug (whether or not the app crashes) relevant to the business decision?
Yes, of course: "When I press the 'more' button in the developer's bio the app crashes" is a more important bug than "When I press the 'more' button in the developer's bio I don't see more text."
Might the criticality of the bug not be relevant to the business decision? Again, yes: "When I press the 'more' button in the developer's bio -- ok, stop, I don't care about this bug."
In some cases the criticality is relevant to the business decisions, in others it may not be. It probably usually will be. The app shouldn't crash, because even if it only happens to 1% of people, they're the ones writing reviews on the app store.
So the criticality is a big red "Hey! This should probably be high-priority!"-flag, but it's not the only piece of information the PM is going to use.
I think the argument is that it's not actually more information; it's just smearing out the same information in a way that makes communication and decision making harder. (Note I'm not completely convinced by either side here, just trying to explain.)
What people are pointing out is that you're not arguing for having more information. The "severity" is a marginal zero information on top of the priority. There is no context where you would want to use "severity" instead of priority.
So...
GIVEN that priority trumps "severity" 100% of the time,
AND GIVEN that priority is already being recorded,
THEREFORE "severity" has no use, and shouldn't be recorded.
What's happening in the post is that someone wondered why there were two fields, and made up a justification for the second field instead of realizing it was a mistake to have two.
You can see the mistake happening whenever people choose a word that ordinarily refers to the concept of importance as their label for "severity", while simultaneously acknowledging that "severity" doesn't affect the importance of the issue.
> In some cases the criticality is relevant to the business decisions, in others it may not be. It probably usually will be. The app shouldn't crash, because even if it only happens to 1% of people, they're the ones writing reviews on the app store.
> So the criticality is a big red "Hey! This should probably be high-priority!"-flag, but it's not the only piece of information the PM is going to use.
The point is that PM may be inundated with lots of bugs and new features and other stories that all need prioritization before the next release. If they had a perfect understanding of all of them, they wouldn't need any metadata at all (not even "bug" vs "feature" -- just read the long-form description!), but metadata helps people make decisions.
High-severity bugs should generally be prioritized higher, except in cases where PMs have gained a deep understanding of the situation and realize it's not business-critical, and override that.
At the point that they set the priority, the criticality is no longer important, just as the "bug" vs "feature" tag is no longer that important once it's been prioritized and assigned.
As with all information gathering, you need to weigh the cost of gathering and storing vs the gains of having the information.
People are saying there are no gains from having this information; which, if true means if there's any cost, the cost is too high.
People are saying it is hard to determine, which means the cost to gather it is high, it may include concensus building.
In my personal professional experience, there hasn't been a whole lot of retrospective analysis on the bugtracker data. So information gathering that doesn't immediately help with fixing the bugs, or communicating which bugs need to be fixed sooner is time wasted. Use of non-critical fields in the tracker is spotty and inconsistent, so analysis would require an expensive data cleaning phase.
Sure, you can say I didn't work in healthy organizations if they didn't do this sort of analysis. But that's the root cause of most of the bugs --- there were never any formal specifications and so many things didn't meet the specs; plus occasionally we wrote crashy code.
Of course? Really?
I mean, the upshot for the user is the same - they wanted to see more info about the developer, they clicked the button, they didn't see more info.
An app crash on most devices isn't actually a particularly dangerous problem. Start the app up again.
Arbitrarily deciding that a particular bug is worse because it causes more harm to the software rather than because it causes more harm to the user is precisely the problem. The importance of fixing those two bugs is probably exactly the same (unless the app crashing causes data loss or corruption or is exploitable or... you get it. Of course, the same could be true of the 'no more info appears' bug - behind the scenes it might be starting an infinitely retrying string of HTTP requests to download the developer bio that gradually leak memory and drain the user's battery. Maybe that's really the more serious bug?)
There isn't some separate measure of 'severity' that independently makes the crash bug magically worse than the functional failure. There's only the actual consequences of the bug. For the user, their data, their time their resources, and your business goals.
So why have severity and priority as two distinct dimensions then? If customer impact is the only relevant metric for your process, why would a bug about which the client says "that's fine for me" ever be considered severe or critical?
If the only metric you care about is customer impact, why track two of them?
One of them sometimes suffers a crash that also loses all their work, and it really pisses them off. That's got to be bad, right? I'd like to call it severe based on its impact to that user.
But then the vast majority of your users have never seen that bug, and the bug is damn difficult to track down, so maybe you can't treat it like a stopper.
But that says nothing about why one would want 2 unrelated priority dimensions.
Maybe the page with the bad bug is getting shut down next month anyway and we’re just accelerating that instead. Doesn’t make the bug any less bad: definitely makes fixing it less of a priority.
(In hindsight I should have used SSRF or SQLi or RCE as examples of the former style bug and XSS as the lower severity of the two to emphasize the difference.)
If you find a RCE vulnerability that never affects your customer, and how much it affects your customer is the only metric by which bugs are prioritized, what relevant information does severity add?
Either it affects your customer, in which case you prioritize a bug's fix by its severity (i.e. simply by how much it affects your customer), or it doesn't affect your customer, in which case you prioritize is by... how much it affects your customer. Either way, you end up prioritizing bugs just by customer impact, which is either measured by the bug's severity (if it affects a customer), or zero (if it doesn't, regardless of severity).
Just to be clear, I am absolutely not arguing that severity shouldn't be tracked, I'm arguing against the idea that purely functional customer requirements should be the only criteria used for prioritizing bugfixes and development tasks.
It's certainly true that "your job is not to build the perfect/best product, it's to build the one your client want and is willing to pay money for", as the parent mentioned. But it's also your responsibility to do the building in a sustainable manner.
Decoupling severity from priority is a great way to run a codebase into the ground. RCEs that are unexploitable today will be expoitable five years from now, after sufficient requirement changes accrue -- and at some point in the future, you're going to have to deal with five years' worth of previously unexploitable RCEs that you have to fix yesterday.
Edit: I've worked for several years on two codebases where customer impact has been the primary metric used for bug prioritization for 15+ years and 7+ years respectively. Both were absolute catastrophes. You could literally crash devices by blowing hot air on them (literally, there was an out-of-bounds access bug in the code that handled overheating alerts and they crashed after a few alerts). An entire team was there just to put out fires, virtually all of which were known bugs that had at one point been deferred because they had "no customer impact" -- until they did.
It's been a very useful experience. Disentangling these things is a very useful skill to have for an expensive, and highly sought-after on the market.
So, to rephrase: you need both precisely because that's often the way you can even start having the prioritization conversation.
> Just to be clear, I am absolutely not arguing that severity shouldn't be tracked, I'm arguing against the idea that purely functional customer requirements should be the only criteria used for prioritizing bugfixes and development tasks.
The problem here is the made-up definition of severity.. basically, you decided that crash or losing settings equals severe and cosmetic stuff equals not severe. Why?
I could easily give it a different definition. For example, a crash that nobody cares about is not severe. A small "cosmetic" slip up that can cause big damage to brand is very severe.
So I agree with the GP here; why do we need to have a definition of severity that does not align with the things that we care about when it comes to actually fixing things?
A high priority bug that isn't fixed presumably has severe consequences for business (or it was prioritized wrong).
A high severity bug that isn't prioritized presumably has low consequences (severity) for business, but it probably really sucks for the unfortunate user who is affected.
Here in pivotal-tracker-land the product manager controls priority by ordering stories in the backlog. There is no need for a separate 'priority' field because that is implicit.
FWIW, there's also no explicit 'severity' field; the PM is expected to understand all the factors that make issues important and order the backlog appropriately. If you need more categorization, you can apply arbitrary labels.
Which do you think the CEO cares about more, the logo or the fact that it crashes sometimes on an infrequently accessed menu?
Because you, as a developer, are clueless regarding business needs, and thus are unaware of why "cosmetic stuff" might be far more important than the risk of deleting someone's user profile. For example, perhaps the color scheme clashes with the one used by your client's main competitor and therefore might leave him vulnerable to lawsuits.
If a base release that was customised for multiple customers has an expected number of P1, P2, P3, P4 issues for most customers but a high number of P1 and P2 issues for one particular customer, off of the same number of issues in the base release as measured by severity, then that will stand out in our measurements and we'll dive deeper into that customisation to see what's going on.
(Edited mid-stream... accidentally submitted too early.)
The point of this system is that high-severity bugs might have lower priority than low-severity bugs if you take business requirements into consideration. Yet, this does not mean that severity should be ignored.
Of course it does add complexity. It is the call of each organization which things are important enough to be worth the extra complexity and which are not. Maybe for yours it isn't worth the extra cost - there is nothing wrong with that - but other people have different needs and so they will have different answers.
In short, you are wrong in thinking there is a universal right answer.
Eg "the color is wrong because we specified it wrong" and "the color is wrong because the app doesn't respect what we ask it to display" both ends up with the same bug (wrong color), same priority, but not the same severity because the second case is uncontrolled.
Severity is a dev tool, priority is a business tool.
1. We have the wrong .png asset in the database
2. Our entire rendering infrastructure is suspect
I'm not saying this is the wrong approach by the way, it's just interesting how people approach this differently.
Then that's the problem we should fix. Instead of creating an extra database field to capture somebody's incorrect opinion, the people who understand priority should be helping developers know enough to have useful opinions.
(...and if you're indeed in a position where you're responsible for hiring decisions or performance reviews: strike "maybe" from the preceding sentence.)
For us, it is absolutely 100% necessary to hire domain experts to prioritize bugs and features. It's not a question of incompetent or dense developers, it's a question of things that are not obvious to someone who doesn't have tons of experience in the field.
It's a problem that I imagine developers working on Chrome, Call of Duty, iTunes, or Outlook don't have. You can hire recent college grads and expect them to understand what the software does, have reasonably good instincts how to prioritize bugs and put together the right user experience even if the description in the feature request is sparse on details.
By the way, I heartily recommend working for such a company. My company works very, very hard to retain people. Someone who's spent ten years getting used to the weird stuff our customers expect is far more valuable than someone with half the pay who needs someone to hold their hand through every single issue. Everyone has their own office, management is extremely permissive about the shit that doesn't matter, there's never deadlines or crunch time, everyone chooses their own work/life balance. (We're hourly, and the expectation is that you work more than forty hours a week and need manager approval if you want to work more than 60 for more than three pay periods in a row) If we want more vacation, we can bank hours and spend it on supplemental vacation. Everything's great.
There's nothing to fix. Developers assess severity but the project manager defines the priority. As the PM calls the shots, they pick which issue should be addressed first, and obviously it's expected that some low-severity issues should be given priority over some high-severity issues.
In fact, the only thing that needs fixing is any potential misunderstanding on behalf is some developers on how low-severity issues shall have priority over high-severity issues.
It seems that a recurrent theme in this thread is that some developers have a hard time understanding that PMs might have solid reasons to prioritize low-severity issues over high-severity issues. It's like these developers are stuck with a mindset where the forest doesn't exist and only the trees they personally have direct contact with should.be worth any consideration.
Your opinion on this strikes me as you being a junior in the software dev world.
Your question misses the point entirely.
The point is that severity is an element that's used to classify an issue with regards to priority. Severity does not exist in a vacuum, and priority is given depending on context. Severity is an attribute, other attributes, that's used to determine the priority.
That's the point, everyone does not understand the business context. Nor are they expected to. That's the job of the PM it's his main responsibility, and it's the reason PMs are hired to manage teams of developers.
The point of developers is to make software that people use. So if we want to do our jobs well, we have to understand how we are creating value. Product managers may manage that information flow, and they may get the final say in decisions. But if they are specifying software in enough detail that developers can be 100% ignorant, then developers can (and should!) be automated out of that work.
The point is that severity is not ignored; it does inform the priority. Most of the time there may even be a direct correlation between severity and priority.
But other (real-world business) factors also inform the priority; while severity will never change in the absence of new information about the bug, those other factors may change frequently. It doesn't make sense for a PM to reread every single ticket and reassess each one's severity when adjusting priorities, when the developer can just determine that once and record it in the ticket from the start.
e.g. A bug might be critical severity if it wipes the entire production database, but low complexity if the fix is to delete one line of code. And maybe its priority is P1 instead of P0 because the customer said they'll remember how to avoid triggering the behavior but they really need that logo color changed asap for an important demo.
So what's the point of severity?
The point of severity is that it's an objective metric determined in isolation based on specific technical guidelines that any developer on the team can follow (such as https://www.chromium.org/developers/severity-guidelines). Whereas priority is a higher-level determination that isn't purely technical.
It's like the difference between body fat percentage and attractiveness. Any "engineer" (doctor with a DEXA scanner) can tell you your BF%, and attractiveness will typically correlate with BF%, but ultimately your "customers" (romantic partners and targets) decide how attractive you are. Not a perfect analogy (priority is still something you'd decide internally, not your customers directly), but hope that clarifies things.
The priority database field is how you communicate this factor, but there is a point where reasonable people can disagree and the organization needs a way to make decisions clear.
To draw out the example even more: you could have a $600K invoice riding on customer acceptance, when the person at the customer site who signs off won't do so until the customer's logo color is correct. While that crasher when you enter a non-numeric character in a numeric field of a new feature? "We accept the functionality meets the milestone so will sign off but we don't plan to roll it out until next quarter after we have begun training personnel on the new feature."
Sure, every good organization should want everybody, not just developers, to understand the customer's business and such but sometimes you just need to get it shippable.
Why? What is the business value of having severity as essentially a protest field logged by people who don't understand business impact? You are basically in all your points explaining the business value of priority which nobody ever disagreed with and then going "and that's why we need two fields".
I don't understand where you could possibly get the "protest field" idea. Severity is an objective statement regarding the known impact of a bug as verified by a developer. It summarizes the technical impact of a software defect. Stating that bug X is high-severity because it crashes is not a protest, and just because the PM decides to give priority to other more pressing issues it doesn't mean you should throw a tantrum.
Crash bugs aren't bad because crashes are inherently bad, they're bad because they have negative user impact - if the program crashes and loses user context, or data, or takes time to restart... those are bad things. If it crashes a little untidily when it receives a shutdown event from the operating system... maybe not so much.
Same goes for performance issues, scalability problems, security flaws, even badly structured code - they don't have technical impact unconnected to their user (or business, at least) impact.
TFA provides a concrete definition and also the method to classify bugs based on severity.
Severity does not divorce a bug from "the user impact". There is, however, also the problem of low-severity bugs or even tasks having low user impact but high business impact.
But that's a contradiction. Unless the users aren't important (and the business is another entity, e.g, a CxO that has clout and demand a fix for a thing that users don't care about).
I do get that people have all sorts of adaptations to dysfunctional working conditions. So if a severity field is one of them, fine. But I don't want people to mistake that for healthy collaboration.
Are they? That's how majority (all?) systems that are asynchronous work. The data to be communicated has to be persisted. I think asynchronous is a good communication method.
For example in the teams I lead I make sure developers participate on PO + stakeholder meetings as observers.
This way when devs fix something or develop a new feature they know first-hand what the business expects.
A nice bonus is that the team often gets personal praises from our clients.
If the color is indeed the one set by the dev/designer/... and it's juste not the right one, that's a controlled thing, it's not severe. The app is doing what you want it to, you just gave it the wrong instructions.
The crash on the other hand is uncontrolled. That's severe, because it's a case where the app is not doing what you want it to.
Severity is something for the dev team internally, priority is something for the product manager and business people that the dev team follows.
When you start mixing the two in a single thing, the dev specific needs are always what goes out the window first.
That's a bad example, something not severe is not severe... can't argue with that. A crash nobody cares about can be quite severe though, there's quite a bit of crash that has been used to make some of the most important security vulnerabilities. When I tried to reproduce the Dirty CoW vulnerability over a VM in a school setting made the teacher VM crash multiple time, he didn't care about the crash either ;), yet that single vulnerability allowed me to skip 90% of the vulnerabilities he wanted us to try to find.
I think your comment point something important that you miss too, different client may have different priority, and you too may. I think severity is closer to YOUR own priority. The severity is the impact this MAY have on your business if it wasn't fixed. Like you said, a small "cosmetic" slip up may be important for our brand, thus severe for us. That "cosmetic" slip up could still not be important for all of our client, like over a one page made to sell, yet still be on top in both priority and severity, because it matters for the next client or the sale team, which does matter to you.
> So I agree with the GP here; why do we need to have a definition of severity that does not align with the things that we care about when it comes to actually fixing things?
You could say that about many information in a ticket. It is still informative though, maybe it's no longer needed, but at one point it did was relevant and in many case, still is.
A ticket with an higher severity is something that you need to put more time on to decide whether it deserve an higher priority. I'll go back to your crash example, why would you works on a crash that nobody care about, why would you see "crash" in the ticket with so many steps to reproduce and believe that this deserve an high priority when it never going to happen? In an ideal world sure you would know 100% of the system and know 100% of the tickets and can understands 100% of the impact of this kind of crash and be able to fit it well in the priority, but let be honest, that ideal world doesn't exist. You have a limited time that you can put on each ticket to decide their priority, you have a limited capacity to understands the impact and everything, you have hundreds of tickets to go through, but something that you can do is see that the high severity set by the developer means that you may need to put more time on that ticket to choose its right priority.
Sure once the priority has been set, does the severity matter? That's a much bigger question that I won't try to answer, probably not, but did it matter at one point? It did. Would you remove any information from a ticket because it no longer matter? I sure hope not.
When your product is used by many clients you can’t rely on what one client says
If you tell the client that they're wrong and that you should fix the critical bug first how they react is a really good test of whether or not they're going to be a good client. If they insist on fixing the lower priority bug first then you should try to stop working with them, because at some point in the future they're going to demand more unreasonable and stupid things that blow up in their face, and you'll lose that client anyway. Save yourself the stress of working with bad clients and get rid of them at the earliest possible opportunity.
If they accept that you know what you're talking about and give you agency to fix things in the order that you think they should be fixed, do your absolute best to keep that client happy because they'll be a joy to collaborate with for years.
I'm all for firing bad clients. Fire the crazy ones. Fire the ones that don't pay. Fire the ones who make ridiculous demands at the last moment.
But the client wants you to deploy a fix to the logo ASAP, you do it. That's the client's priority. As long as they accept the disruption to the rest of the work, it's no problem.
Basically, if what you are trying to do is too complex/nuanced for everybody to understand - don't do it.
A developer or tester might be good at specifying how severe a bug is: impact (does everyone get it, or just some users) multiplied by frequency (does it happen in commonly used functionality?) multiplied by a category factor (is it a crash, cosmetic, dataloss...) will give you the severity.
Some completely different person can come along and give this a high priority because of business reasons, and to prioritize fixing the bug compared to other functionality. Two bugs with the same assigned severity might impact two different markets where the business knows one is critical for the survival of the business, and one is already a safe market. Then it's a priority to fix it.
A second reason might be due to ROI (easier severe bugs are better to prioritize than difficult bugs of the same severity).
Ultimately you could have a low-severity bug, but it impacts a really important customer who cares about it, so it's high priority. This is the most common case I've seen.
Severity also can impact whether something gets prioritized at all. If it's low-severity and the business side isn't there either, maybe it just doesn't get done. Whereas something with high severity but low business impact still probably needs to be done regardless or it will lead to bigger problems.
We have applied the same metric to feature request prioritization. Severity can be thought of as necessity for an individual customer (must have, should have, nice to have, could have, won’t have) and frequency represents an estimation of demand in the market.
We even created our own Jira plugin to calculate the product of these and use that number as a self prioritizing index.
Working directly with PO's both "Technical POs" and "Product POs" I've found that creating a strong distinction between the two as well as what the two dimensions define has a significant impact on the quality of the product over time.
In the original post we see that both Priority and Severity are two different dimensions each representing a scale. This is what I think is causing a lot of the discussion in this thread.
Because both of these values are a scale we are creating a foundation over which we can get confused and we find ourselves needing to differentiate these two in some way. This comment is a good example of the confused conversations I would often see. All of variations on the same flavour:
> If the goal is to better communicate what "type" of bug it is, using an enum classification (eg, cosmetic vs crash vs data-corruption) would be more appropriate than a numeric scale.
To get around this in Agile contracts I've always advocated the following:
"Severity" scale. Severity of the issue on the user from 1-5. (For Jira users imagine this as your 'Priority' field renamed to severity.
1: The user is unable to complete the journey or part of the journey
2: The user is unable to perform an action, but can complete the journey
3: The user is able to perform an action but...
4: The user is able to perform an action but...
5: The user is able to perform an action but...
Of course, you can put in your own definitions. The important thing being that this is defined from the 'users' point of view rather than attempting to define the nature of the issue (crash, freeze, etc...). The nature of the issue can be discussed within the ticket itself.
The second dimension 'Priority' instead being thought of as 'Ranking' and is not a field which is added to the ticket. This defines the order in which the issues must be dealt with. Thus allowing a PO to appreciate the severity of an issue and also negotiate with the team it's relative important through ranking.
Buying clothes detergent because you ran out and have no clean clothes is high priority, low severity. Booking cancer screening sometime this year because you're reaching that age and there's a history of cancer in your family is low priority, high severity.
But maybe this bug occurs very rarely and in very specific circumstances.
On the other hand you have also found another bug that makes your app use bright green Comic Sans. It is not severe as it is only cosmetic.
But for some reason, on your last release this happens every time the app is started.
It seems likely that the second issue will have a much higher priority and will have to be fixed immediately, while the null pointer can wait.
Severity is purely technical. Priority is a project/commercial decision (which is informed by severity but also other considerations). They generally do not align.
Of course, scheduling is often dominated by other factors, but it gives you a default at least.
I could see where a severity dimension could be useful retrospectively if paired with classification. For example, you could look back and say, why did we have a big spike in severity 1 security bugs in Q4? However, I think I'd rather not have the dimension available than be forced to set it on every single bug on the off chance it will help with analysis later. "I bet we can do cool stuff with this someday" is never a good reason to add process overhead. If somebody retrospectively categorizes and summarizes tickets by some dimension and proves that it yields useful information, that could be grounds for asking engineers to track it in the future.
I've never seen impact/urgency/severity as being anything other than different ways to say priority or overcomplicated ways to derive priority. Then again I've never wrangled an absolutely massive bug/ticket queue.
Lets say you have 10 bugs and they are prioritized in order from 1 to 10. Bugs with the priority 2, 3, and 6 have been given a severity of P0 (a ship-stopper, as the original post described it). Bugs 1, 4, 5, and 7-10 are not considered P0 bugs.
Given those priorities and their relative severity, that means the team is committing to deliver, at least, bugs 1-6 prior to the next release. That makes severity a pretty useful dimension to consider on its own and not just as consideration when determining priority.
A bug might be Major or Minor but there is a high risk to fixing it. You might want to do it early in the development process, ahead of higher severity issues, so you get the most runtime on the fix before the next deployment.
Low severity and high priority can be:
- it breaks the software for just one person, but that person is on the critical path.
- if breaks users workflow. For example a keyboard shortcut that is used every second stops working.
- the bug is an easy, high value fix, so why wait?
- "severity" as level of importance the tester assigned it when they found the bug and - "priority" as level of importance a developer assigned to it after triage.
That first piece of information is important when you need to determine which bugs to look at first for triage, but how important is it after that point? Can it not just be replaced after the developer's judgement?
In other words, could you not have a single priority field? The tester uses a heuristic to assign an initial priority (e.g., crashes are P0, cosmetic are P4). The dev uses this to prioritize which bugs to triage first, and once they've determined a new priority based on customer experience combined with app behaviour, they replace the old one.
If you really need to go back and check what the tester assigned, then I assume you can just use the "history" or "revision" feature in your bug tracking app.
Additionally, as suggested in a different comment, you can add a label for the bug's type if you feel that's important (crashing, lagging, cosmetic, etc.).
Perhaps the message here is that the app's behaviour in a vacuum is not the sole determinant of its priority. But then that should be the message, rather than claiming there is another metric which needs to be separately tracked when evaluating bugs.
But I think it should be similar to risk calculation, multiply by the "impact" or "damage". A typo might offend a small number of people but have a big publicity impact
That's why the split between priority and severity exists: The tester is telling you the subjective view of the bug as if a user encountered it; and the triage process is all about judging the priority of fixing the bug in this particular release cycle.
If you can't at least do a rough triage of all the new bugs in one sitting, you're either not allocating enough time for triage, or letting yourself get way too bogged down with ancillary conversation during the triage meeting, or it's just time to scrap the product and go home.
If you can stay on top of your triage, then there's not really any need to worry about what order you do them in.
Priority is how the developer (or, more likely, the project management team) sees the bug. Highest priority bugs are the ones in the project manager's spreadsheet and ended up there because of the most vocal complainers (paying customers, an inconvenienced CEO, etc). The rest will never get fixed.
It's impractical to use the severity of the bug to rank the priority, because they're not strongly correlated.
I think they are highly correlated, but they occasionally diverge.
If there are people on the team who insist on objectively classifying bugs with no reference to customer impact, you don’t need to indulge them by giving them a meaningless drop down to play with in Jira, you need to get them up to speed with understanding your product and release priorities.
Anyway, the developer got this stop ship the day before Thanksgiving, so he worked all weekend and Monday morning turned in a fix. At which time the factory manager told him "thanks, but one of the parts to make this machine is on back order for 4 months so we weren't planing to make that product anyway".
Tags/label have been working quite well for me. Priority can be replaced with milestones, but you have to create them for each releases to ensure P0 are quickly enough.
If anyone has a workflow where "severity" is important for your decision-making, I'm curious to learn about it.
This has led me to think that a hierarchical priority structure would be useful. So we could say that these 100 Level 3 issues are generally of the same priority, but there's a ranking within that. But even the highest among them is still less than the lowest Level 2 issue.
That's not to say that it shouldn't also be 2-dimensional though.
Some would factor severity into priority (This is a crashing-type bug, so we should prioritize it), or priority into severity (This bug only causes minor harm, so it's not a high priority for us). But this was done silently and unconsciously, and different people had different directions for how they would do it. Due to this experience, I think one metric is best.
The problem is that "Severity" is being used for something that doesn't really match its intuitive meaning.
Maybe call the field "kind of impact" (or something similar) and have enumerated values or tags like "crash", "hang", "typo", etc. Now the field is intuitive and you don't need to retrain (or write blog posts) to get people to understand your new meaning for the word "severity".