HN Explorer

Ask HN: Is there a React test framework that actually works?

9 points by PaulHoule ↗ HN
For very simple cases, such as a Component that takes props but doesn't use any hooks, I am happy with the react-test-renderer package that comes with React but is now deprecated, although outputting JSON instead of HTML is a definite case of "test framework introduced another abstraction I don't need that makes me feel less in control".

Enzyme and react-test-library on the other hand have the potential to write a test like:

   1. Mount component
   2. Do something to the component (say click on a button)
   3. Wait for React to settle down
   4. Check that the component looks OK
Both Enzyme and RTL have many minor problems in 1, 2 and 4. For instance some CSS selectors that should work in Enzyme don't seem to. The html() method crashes sometimes in Enzyme. RTL's dependence on the accessibility tree answers some of the problems of "test automation" but it also is a statement that "three.js developers don't deserve to have a test framework". Also I have enough things to worry about with my code and getting it to actually work inside the test framework (like discovering whatever shibboleth it takes to get my useEffect callbacks actually called) that I don't have any mental capacity to fight with the accessibility tree and the Rube Goldberg machine that extracts it from HTML. Thus instead of seeing the HTML which the end user "sees" I have a bunch of abstractions that introduce more uncertainty in a situation where I'm already uncertain of just about everything.

And it is (3) that is the elephant in the room because what you have to do for (3) is dependent on your application and it can get pretty complicated. The root problem is that your code might set some value by calling setSomeValue but even in a rerender, value doesn't change until React does some work that is scheduled in a promise. And that promise could cause another promise to get queued and so forth so all you can do is set a long timeout and keep polling to check that your component has really settled down which is dependent on your component, the implementation of your component, etc. (Hint: if your "unit test" has a 1000ms timeout it's not really a unit test... For that matter, RTL is always warning you how slow it is to look up items by role)

Given that React is so prominent in the industry and that people seem to think testing is so important you'd think somebody would patch React or the node.js runtime if necessary to keep track to see if there are still promises so (3) can be as simple as

   await done()
Looking at discussions in the ticket systems for the frameworks the developers seem to be as confused as everyone else.

As it was, if an evil billionaire had paid these framework developers to convince developers that testing is a total waste of time he'd be tickled pink by the results. Is anyone trying a different path? Is there a React testing framework developed by someone who actually tried writing tests for React?

9 comments

[ 5.5 ms ] story [ 33.5 ms ] thread
solardev
In the small-med biz teams I worked in, we didn't try to write component-level tests. We had a bunch of unit tests for basic utility functions, API calls, etc., but then to test the overall functionality of the frontend we just wrote a lot of frontend & E2E tests with Playwright/Puppeteer (or a cloud version, like https://docs.datadoghq.com/synthetics/ or https://www.browserstack.com/percy).

Those are just real browsers running headlessly, performing UI actions, and then verifying the results (either looking for something in a real DOM and/or comparing screenshots). If something went wrong, the devs can then follow the exact steps to replicate it and debug it.

It was a lot quicker (and arguably more useful, I think, since it emulates real users and mostly real systems -- if your staging setup is similar to prod) than trying to shim some intermediary component testing. (You can also fake certain inputs in lieu of waiting for a backend, if you prefer.) Is it perfect? No, but it's a quick and cheap stopgap.

Edit: A further argument is that this can also help catch unexpected DOM bugs (like z-indexes or viewport width weirdnesses or plain browser incompatibilities) that pure code bugs won't catch. At the end of the day it's what the user sees that matters, so IMO if you have limited resources and can't write tests at every level, it's better to have more E2E coverage that realistically emulates what a real user would experience, since that would typically cover logic bugs too (at least happy path ones).

PaulHoule
I've worked in teams that had testers who used commercial tools to run regression tests against real UIs and found that to be helpful.

As you point out there are certain things that are easy to test. The other day I wrote a paginator using a TDD approach because I knew roughly what behavior I wanted but wasn't sure how to implement it. I wrote a simple function that took a few arguments and returned an array like

   [0, 1, 2, -1, 10, 11, 12, 13, 14, -1, 26, 27, 28]
which made it really easy to test the inputs and outputs. I didn't think the code that generated the HTML from that was really worth testing because it was so superficial.
solardev
> [0, 1, 2, -1, 10, 11, 12, 13, 14, -1, 26, 27, 28]

What is that supposed to represent in terms of pagination? Are they cursors...?

> I didn't think the code that generated the HTML from that was really worth testing because it was so superficial.

For what it's worth (as a frontend person), pagination is traditionally one of the trickier things for us to make sure is working correctly. Off-by-one mistakes can happen for example (especially with zero-indexed items). Or sometimes the prev/next buttons don't correctly use the same math as the page 1, 2, 3, etc. buttons. It gets even harder if you don't know the total length in the beginning, or if you allow multiple page sizes or sorts/filters, or use lazy loading, etc.

For those reasons I try to use a ready-built lib like MUI where all that is already tested internally (like https://github.com/mui/material-ui/blob/next/packages/mui-ma... or https://github.com/mui/material-ui/blob/next/packages/mui-ba...), but we still add our own automated and manual tests in our own usages.

But then again I'm bad at math and division, lol, so maybe it's just my own weakness.

PaulHoule
The paginator was one of the sources of stress in that project, it actually had me walk away from the keyboard and think about it a lot before deciding how to do it.

Non-negative numbers are the index of the page starting at 0 and -1 is an ellipsis. This particular system in written in Python with Flask, I am using this CSS

https://getbootstrap.com/docs/5.3/components/pagination/

so 12 gets rewritten to

   <li class="page-item"><a class="page-link" href="?skip=12">12</a></li>
As you point out this is the kind of code that has devilish complexity in the corner cases (benefits from having tests) but is also easy to test (no need for mocks, callback engineering, etc.) Some other examples like this are almost any kind of string parsing where off-by-one and other silly errors are so easy to make and where tests are so easy to write.

Now you could also test the function that writes the HTML for "12" and it is pretty easy to do but the gain is small and it is also un-DRY because if you want to change the appearance of your HTML you have to change the test. (Granted, a good set of unit tests could prove that it's hard to sneak an HTML/JS injection in.)

In the early 2000s there was a huge controversy over Mock Objects in Java testing. Back then you had to write your own Mocks so it was a lot of work which could be just as error prone as your own code. At the time it was also easy to get into test induced design deformation sneaking your mocks into code.

10 years or so later there was Mockito and also advances in dependency injection that made mocks ergonomic in Java in most cases.

morbicer
Exactly this. Unless your product is components (e.g. you are in a team building UI kit) it's better to have unit tests for util functions and integration/e2e tests in Playwright or Cypress.

We try to avoid logic directly in components so apart from few exceptions (like complex tree pickers) it's moot to test them. For these few exceptions RTL serves us pretty well.

Testing redux reducers was cleaner back in time but hooks can be tested with renderHook/act and that's where your logic should be.

lmiller1990
You could try Cypress, it has a component testing solution. It's not for everyone, but the way Cypress works, is it will auto-wait for your assertion to run, so you don't need the whole async/await.

No solution is perfect! I see this as more integration / component testing than a true unit test.

JasonSage
When possible, if I can implement something without tying it to React I will do that first, test that, and then make a React integration for it.

User flows can be written as sagas, or generators, or state machines—these get tested. Reifying a data object into some model that has a lot of derived getters or actions the presentation layer needs—this can be implemented and tested, then made available with a hook.

To me this is always the logical starting point because I can unit test or TDD, evaluate my assumptions about problems and solutions, and present conflicts to stakeholders, all without having to have the project up in the browser or writing problematic React tests.

When you have a component library that needs testing, or specific things need interaction testing, Storybook’s visual tests have been the easiest way to go. And if you’ve separated the logic out from the presentation as I’ve laid out, stories are easy to write because you’ve decoupled the presentation from the complicated bits from the start.