Ask HN: Do you have a dedicated QA team?

We have a dedicated QA team. Recently they've been "rolled into" the dev team, but they're still a group of people dedicated to testing. The actual developers should be writing their unit tests and functionally validating before opening a PR, then the other devs can validate before approving that PR, but the full E2E, automation, etc. tests should have a dedicated team, because there is a lot involved in doing that properly, and developers need time to... ya know, develop.

By putting all those responsibilities on the devs, they have essentially turned a parallel development system into a serial one, which is inefficient and, frankly, dumb.

This seems to be a pretty common setup. I’ve heard opinions on both sides about whether it is a good idea.

One of the biggest challenges with it for me is that a dev who is a dev is a very different personality who a dev who manages the entire SDLC.

The former placed in the latter environment is going to be lost and annoyed at all the moving parts. They signed up to write code and to trust that the process would deliver good requirements.

The latter in the former environment is going to be frustrated by all the process. They signed up to really own the product and realize that they really just own a block of a process.

I suspect either can work if you have people suited for each. Switching people between them sounds like a recipe for attrition though.

In my company we also have dedicated QA teams. I agree with all IcePunkMetal wrote above, getting rid of them is kind of dumb. Like he wrote, they are now rolled up into the dev team instead of being part of the separate "QA division", which is better IMO. But they're still not developers.

In my team, they are fully integrated into the decision making process and roadmaps, they are CCd to almost all the team's internal emails, and a format version isn't released without their authorization.

Now, there are some problems with being a QA, personal stuff like having a low self-esteem since "you're not smart enough to be a dev", and enmity between devs, PMs and QA due to feelings that QA hold up the teams. As a former QA myself I'm fully aware of them. In our particular team these issues are negligible (well, at least I hope they are). I personally fully respect the QA team and stop whatever I'm doing to help them with whatever they need or ask. I see them as my first line of defense against bugs, especially the kind of UX bugs that are difficult to test for.

We didn't use to, but now we do. I think we did a better job when we didn't have the team TBH.

Engineers here used to be the ones testing out the code, verifying their changes on pre-production environments, and keep an eye on the deployed code in production to make sure it's working well.

Now I get a sense that engineers do the work up to the point of where QA steps in, and throwing it over to them and moving on. The QA team is new, and so not very well versed in the product, and rely on the engineers a great deal on how to test things, which I get the feeling also ends with the QA team not going out of their way to test things the engineers didn't explicitly say to test. I think it may be playing out in a way where our quality assurance could get worse, and our engineers will end up being less well-versed in the product, and feel less responsibility for delivering quality, while also requiring another team to manage, and spreading work over different people. As a disclaimer I think we may just have a not great setup more-so than saying all QA is bad.

My feel is that this pattern leads to quantitatively better software that is qualitatively worse. There are two reasons for this:

1. QA teams come up with new ways to break software in the way humans use it. Automated tests etc confirm the software works as designed.

2. Because the org making this choice isn't stupid, they measure bug frequency and impact and find that "only X users have Y impact where Y is ranked as low" per each bug. No one bug is ever worth fixing. But, user X aren't seeing that one bug, they get hit by tens or hundreds of these on a regular basis. As a result, their perceived experience with a given product is poor and it feels buggy.

This idea that you can measure bugs in isolation and deliver perceived product quality is faulty. A user experiences a product as a whole, and if you can't evaluate that as well, you will be seen as delivering a poor product.

Devs can add automation tests to a suite just fine. What you lose when you lose dedicated QA is: who improves the automation suite? Who checks out new testing tools and POCs them? Who reads QA blogs and goes to QA conferences and generally considers it their profession to find ways to test more effectively?

Good QAs are so important. There are so many bugs that would have never been found by us. In the past 3 weeks, both our QAs have been off, and the inbox list of problems have grounded to almost zero! Not because we are writing more-than-usual-amazing code, but because no one managed to find them.

We do have a dedicated QA team and it's absolutely essential because developers (like me) aren't necessarily domain experts.

The QA team brings something significant that developers (myself) will never have: their machines closely represent the customers machines. They use a well-polished methodology to push the models we ship to their limits and provide relatively detailed information about how to reproduce failures.

I can't count the number of times I've tested something 5 times, convinced myself that the code handled tons of error cases, shipped it, and in 10 minutes a customer types "five" into a field expecting an integer which causes the next step to use "0" for processing.

We don't have a QA team; we have an internal team that uses our product to do their work. It's part QA, part product development, part marketing; they help us prioritize feature work, they write blog posts about the cool stuff they can do with our product, and when bugs surface, they find them, because it's all part of their workflow.

This doesn't work for all companies; if you're building a system for children to learn how to type, you can't really hire a bunch of children to learn how to type... but I think for most companies, this can and does work.

Having been through the decision process in the past for whether to go when centralized QA, dedicated QA on teams, or automated testing as an engineering responsibility, I came away with the personal opinion that the quality of automated test frameworks has improved so drastically in the past five years or so that we really are in a different place than we were when concept of dedicated QA teams was invented long ago. Large scale software engineering is a human coordination problem (see Conway's Law and the Mythical Man Month), and the time required for engineers to write tests after already understanding the requirements is small compared to the time required for the organization to have someone else try to separately understand the same information and separately write the test. That didn't use to be the case, but today it is. I get not wanting to take time to write tests, just as I get not wanting to take time to document your code, but that's not a good enough reason for introducing the inefficiencies of having someone else try to do something that is best described as your own job (again, the economics used to be different because writing tests used to be slow and inefficient, but that is changing/has changed).

I will say however that I think you do generally still need a team (or individual at smaller shops) dedicated to tooling for automated testing to make the process of writing tests efficient for all those who are writing tests.

I always setup a dedicated QA team that has right of veto for releases. Developers indeed can do some testing, but quality assurance is more than that. It starts at the problem definition stage, when QA engineers ensure there’s no misunderstanding of the requirements, they make sense and they answer all important questions. It ends long after release, because production support is also part of QA. QA engineers do their part of testing, just like developers do, but they also coordinate delivery and support from quality perspective. Their feedback is often valuable to product managers and designers, they can help a lot your customer support team.

We have a QA team of 4 that supports x4 as many engineers.

If they weren’t there - engineers would spend at least 1/4th of their time performing QA duties with likely worse results and at significantly higher cost to the company, since their compensation is higher.

So it makes sense to do that only as a cost saving measure in hard times - you are picking losing which employees will be less detrimental in the long term, even if you expect that those that remain will be less productive.

I don't at my current job, but my last job did. The benefit of having them is that they maintain domain knowledge about how something works, will ask pertinent questions about new functionality, and are able to dedicate a lot of time to trying to break a component. The drawback of having them is that if they are mismanaged or get backed up, it can take a long time before a feature even gets looked at (my last job was so bad with this that I would often get asked about features that I submitted 4-6 weeks later).

The good thing about not having a QA team is that QA becomes a much more active part of the process where something must be tested by the developers before it's completed. You also don't have a single bottleneck if the QA team is overwhelmed. The drawbacks are that developers aren't always very thorough and that their testing time takes away from their development time.

I've always had dedicated QA teams when working with physical devices (android smartphones and VR headsets, specifically). I think a dedicated QA team is absolutely non-negotiable for hardware development, because there is no test harness that can accurately simulate the range of stupid shit that happens with hardware:

"smartphone doesn't boot reliably when the ambient temperature is below 40 degrees fahrenheit" (temperature lowers the battery voltage + high power usage during startup = voltage drops below FCC-mandated minimums for the antennas -> automated SoC shutdown)

"bluetooth controllers won't pair reliably in building 17" (too much interference to pair if there are more than ~100 other bluetooth devices nearby)

The combinatorial explosion of external factors in this kind of bug is insane - you just won't have a good time without dedicated folks whose focus is chasing these down (and I'll take a good QA over a good engineer for this sort of investigation, any day).

QA teams primary benefit shouldn't be running automated tests that the developers could do. It should be to do exploratory testing and trying to break things.

There should be a adversarial, yet respectful, relationship between dev and QA where dev tries to get their software released and QA is finding reasons why it shouldn't be released.

If there's only developers who test their own code then they're not incentivized for finding the ways their code breaks. They'll downplay the faults or even be blind to them because they're too close to the implementation and aren't "thinking like a user".

It seems odd to me when those in charge ignore the problems of disbanding QA. But I assume it's because the management may not be incentivized to promote quality software. Instead the incentives are to crank out features.

That said, maybe the market for the software doesn't punish the software makers for having buggy software. In such a case, ignoring QA work is a rational decision.

  1. Consolidation: There's usually 1 big player that everyone uses. When that's the case there is nowhere else to go.

  2. Lock in: Even if there was somewhere to go, once you've committed your business to a software vendor, it's really hard (time + expense) to switch.

  3. Executives get bonuses based on income: The easiest way to juice the numbers is to cut employees and departments. The results of cutting QA won't get noticed by the market for a few years.

  4. Over reliance on automated unit testing by management: I think unit testing is great, but it's really hard to utilize it on user XP. It was never meant to be a replacement to manual QA testing, but an augmentation. Of course refer to #3.

  5. Reduction in autonomous middle management: Executives say cut this and cut that, middle management who actually know the process have little to no say, or won't say because they don't want to lose their jobs.

We have a dedicated QA team. They work side-by-side with devs, and we rely on them. They are absolute lifesavers and let us focus on the dev side of things. Their knowledge of the products, all the various ways it interacts, and their dedication to testing are wonderful. The quality of the product would suffer if they didn't exist, hands down.

We do have QA team(2 members, 30+ engineers) but they are not the gate keepers for releases.

Developers write all sorts of tests(TDD + pair programming helps), and when we need an extra pair of eyes, we ask the product team to have a look at it.

But that's about it. QA's are involved more in the creative side of things post releases or have an independent path to see where the things could go wrong, more like doing monkey testing.

In my team's current state, there is no point in a QA team as we're already aware things are broken.

yes, the customer.

Problem I faced in past with a dedicated QA team is, the development process is often faster than QA. Which puts QA in the backseat with each new release. Also, human coordination becomes a problem as there will be a small configuration of which QAs team might not be aware of e.g. A/B configurations, hotfixes, etc.

On the other hand, with devs writing tests is there is learning curve and most devs have not written tests or think of it as a chore. They also don't have end-user perspectives and have biases towards things working.

Ideally in my opinion testing should be shared responsibility, where devs test feature they implemented and the QA team add checks from an end-user perspective.

P.S.- I'm co-founder of startup https://crusher.dev making QA easy by using a low-code approach.

In my experience, devs make terrible QAs, since they know what the code should do and never think of things like "pasting a novel into a textfield" and the like. They also tend to be much more expensive as well. Combine the two, and it's just not at all an efficient use of a company's budget.

As apps become easier to maintain and keep online, the extreme separation in roles is only important at companies with sufficient scale in some metric to need the specialization.

I can deploy a prod EKS/CI/CD setup with Terraform scripts, and be deploying code to the public in a morning now.

Like other service jobs, IT is going to contract sooner than later. NoCode infrastructure is where k8s was in 2014.

IT workers thinking in IT patterns as usual is a huge blocker to progress. Elder workers domain knowledge is stored in DBs and source control everywhere. It’s trivial to parse and transform into another syntax.

Congrats; you trained your Copilot replacements without even knowing it.

QA Engineer here. We don't have a centralized QA department, but we do have a dedicated engineer who is responsible for QA that is integrated within the development team.

Before I joined my team, it didn't have a dedicated QA member. The quality of the software was fine, but there were other compromises. The team didn't have a good test strategy - every developer made adhoc decisions on how their code was tested. Our E2E tests ran slowly and had tonnes of duplicates, since nobody had gone through the entire list and cleaned it up. A dedicated QA member has the time and the responsibility to solve problems with quality that most devs are likely to treat as secondary to developing features. And the improvements are substantial - our E2E tests now run 18x (!) faster.

For the most part, I see my role as being an enabler - I help devs build quality into their work and hold them accountable to it. I don't see it as an adverserial division of responsibility, but more as a collaborative effort, with the dev and QA coming from a different focal point.

The problem with strict division of labour and adverserial QA is that it leads to

1) Organizational / team silos

2) Local optimization (software development and QA is intricately interwoven, so it doesn't lend itself well to this strict division)

3) Information overhead and loss

A good QA engineer, IMO, requires better coordination and communication skills than a developer because they liaise with multiple developers and inspire them to make quality a priority in their work. Further, QA is involved from the definition of a feature (in sprint refinements) right up to when code is deployed and runs well in production. Therefore, at certain points, the boundaries between QA and the Dev team or QA and the PO blur and disappear entirely.

I found this to be a great read that summarizes my thoughts on the topic: https://www.thoughtworks.com/en-de/insights/blog/qa-dead

At this point I only see value in manual QA testing for front-end apps.

It's now possible to test pretty much anything via automation, although front-end testing is still harder than most and there are some intangible UX issues that can only be experienced by human users.

But for testing APIs? It is normally possible to cover 100% of what is needed via a combination of unit tests and integration tests. This is especially true now that tooling exists that can bring up ephemeral test environments with dependencies such as databases etc.