I like test recorders (Selenium IDE). They aren't brittle if you manually enter your selectors. Plus you can hook a test recorder up to your own code to fill in any gaps. I do wish they showed mouse movement, though.
One I'd like to add is to keep it simple. When I hear the word pipeline I immediately think "oh no". I'm telling you man, some shops add a mind blowing amount of unneeded complexity with their pipelines.
The objects under test are brittle and bound to change dramatically. Selenium can't offer you insurance against this.
I now remember a test suite we had on my project (automotive). We tested against the code structure... that is, we would follow a variable's lifetime and check that its value is correct throughout execution. Two releases later, guess which function got slightly refactored and threw off some 20 testcases?
'Test Pyramid' considered harmful IMO. It discourages people from writing tests against the public interface of their system , and instead encourages testing against internals.
This leads to a test pack that prevents refactoring, and can't be trusted to catch regression errors.
This is partly down to the widely confused definition of a 'unit test' (which a lot of people interpret as 'one test per class').
TBH I feel like integration and UI tests can be combined at this point. Writing a bunch of tests to only check UI elements feels like a waste to me. I don't mean to go full on BDD where every step is spec'd out, but I've done it by interacting with elements and validating data with the API.
5 comments
[ 3.4 ms ] story [ 18.9 ms ] threadOne I'd like to add is to keep it simple. When I hear the word pipeline I immediately think "oh no". I'm telling you man, some shops add a mind blowing amount of unneeded complexity with their pipelines.
There's definitely some good with selenium, but I and others find it is often brittle and doesn't catch errors as reliably as would be ideal.
I now remember a test suite we had on my project (automotive). We tested against the code structure... that is, we would follow a variable's lifetime and check that its value is correct throughout execution. Two releases later, guess which function got slightly refactored and threw off some 20 testcases?
This leads to a test pack that prevents refactoring, and can't be trusted to catch regression errors.
This is partly down to the widely confused definition of a 'unit test' (which a lot of people interpret as 'one test per class').
If you haven't seen it, please watch Ian Cooper's "TDD, Where Did It All Go Wrong" https://www.youtube.com/watch?v=EZ05e7EMOLM
Also, I thought continuous testing was having your test pack run all the time while coding, but hey ho.