And the sunset calculator doesn't need a location provider. All it needs are location coordinates which can be given to the calculate function.
In fact the whole thing then becomes a standalone function which takes Location and other relevant parameters and returns the computer sunrise/sunset values. Pure function and super easy to unit test.
If one needs to do a lot of "mockups" for your unit tests then maybe one needs to consider the API and class design. Removing needless coupling helps testability by removing the need to use mocks in the first place.
In design terms, I would claim that Sunset Calculator is a "component" (an integration of units), with a "required interface" (i.e. abstracted) that provides the coordinates.
A role of one unit inside this component is to use the provided coordinates and only perform the calculation.
Unit testing is not overrated, if you feel this way you likely just suck at optimising your suite for high RoI tests or chase some stupid metric like code coverage getting mad when you find out you wasted a bunch of time writing worthless tests.
There's nothing that cements the value of units tests more, for me, than surfacing bugs in new code almost immediately that would otherwise need debugging "in situ" in the application.
Figuring out where your bug is is so much harder when you have to do it though the lens of other code, whereas in your unit test, you can just see "oh, I'm returning foo.X, not foo.Y".
It also makes sure you actually can construct your object at all without dragging in a dependency on the entire system. Code without tests tends to accrete things that can only be set up in a very long and complex process. This is both hard to reason about (because your system can only be thought about as an ensemble) and fragile when the system changes: you now have to unpick the dependencies to allow your "SendEmail" function to not somehow depend on an interface to some SPI hardware!
But there's certainly value in not spending hours testing obvious code to death: a getter is almost certainly fine, and even if it isn't, the first "real" test that uses it will fail if you're returning the wrong thing. But if, down the line, you do find a bug in it, then something was probably not that obvious!
Nothing cements the uselessness of unit tests more, for me, than not surfacing bugs in new code and having wasted all that time preemptively debugging code that works.
Then don't write tests for that code. Writing tests for trivial stuff is pointless. Write tests for code that could actually be wrong, or where you need to prove to yourself or others that the invariant is maintained. Critically, where it's important that the invariant stays true even after refactoring.
It's also valuable to have some testing framework available, just so that it's then easy to write tests when they're needed (which comes back around to "make sure that your objects can ever be constructed" thing). Not all unit tests have to be preemptive, even if the presence of the ability to quickly write them is.
My reaction to this is... if you're writing tests for pointless things to test.. it means that your testing framework/approach may not be giving you more of what you need... or it could be a defect in the language.
In Unit testing, you should only be testing the code you wrote. (Contract testing is more of an assurance test) In Java you have to write unit tests for your getters and setters because they might do more than what you expected (as that's within the realm of possiblity) If you find yourself writting getters or setter tests .. you should be looking into a better language like Groovy, Scala that do it for you, or something like Lombok or Records in later JDKs.
> Write tests for code that could actually be wrong, or where you need to prove to yourself or others that the invariant is maintained.
Yes, what you need to do in the end, is to use your judgement. But you will get a surprising amount of pushback against this obvious common sense solution. From management who thinks it's too risky, and from developers who have chosen a technical career just because they want to rely on their intelligence, and black and white thinking, and now you're asking them to make trade offs and decisions which is management territory. The way out in my opinion is to make technical leaders that have actual authority, down to the details.
Testing isn't only about bugs. Suggesting that's the only benefit is a strawman. Testing forces you to design better code. Sure you can mock the crap out of things and still create trash but if you think critically it usually affords making more robust code.
So, they suggest that you spin up the dependent services as part of the test runner and run tests for your service against those? I've worked on a codebase that did this extensively. Over time it lead to a test suite that takes a long time to run, slowing down the release process considerably, and inevitably requiring further engineering effort to make the tests run in a reasonable amount of time. I can't say I would recommend it as an approach based on my experience with it, but YMMV.
Can you (or someone else) tell me more about how to run database tests efficiently?
I'm working on an app that has 200+ tables and requires a lot of data to be in the database before anything even works at all. Of course good tests interacting with the database need a fresh database every time so even a test suite of about 10 tests can take 10-20 minutes already.
>Over time it lead to a test suite that takes a long time to run, slowing down the release process considerably
Slower, but the test suite is far more effective at catching bugs.
These tests do take more up front engineering but since they arent pale imitations of the code they also dont need to be totally rewritten every time the code is refactored. Higher capex, lower opex.
Didn't even knew that Spotify wrote an article about this "methodology" thanks for sharing.
No, all you have to do is spin up your service under test, its database and the service bus/event hub/whatever you use to communicate between services.
Nowadays with Docker it's really easy and fast to spin up test instances during your test run. (4-5 LoC with a good library in C#).
Since 1 or 2 years I'm doing the exactly same thing on my projects and it works great to speed-up development process and if a bug pops in production it's really easy to add it as a future test case.
You're only attached to the input request (endpoint, query params, request body) and the response body. But that's ok because it's already a part of the contract between the API and its consumers. That's it.
"Considering the factors mentioned above, we can reason that unit tests are only useful to verify pure business logic inside of a given function."
That just isn't true and it makes the rest of the blogpost also not true.
A unit test should test "a unit of functionality" not just a method or a class. Your unit tests also shouldn't be coupled to the implementation of your unit of functionality. If you are making classes or methods public because you want to unit test them, you're doing it wrong.
The exception is maybe those tests you are writing while you're doing the coding. But you don't have to keep them around as they are.
Wild speculation with a sprinkle of logic (garbage assumptions lead to garbage conclusions)
HOWEVER, I have skimmed through it by now and the last paragraph is actually quite good. It takes a while to get there and the chosen examples aren't great, though.
Me too. The author shows a deep misunderstanding and even obliviousness to testing, particularly from a practical point of view, that takes any credibility from any argument listed in the article.
It's even perplexing how the best example the author could come up with was this absurd chain of strawmen that a) it's impossible to test code that uses instances of HttpClient, b) dependencies used in dependency injection "serve no practical purpose other than making unit testing possible."
There are plenty of people talking about unit tests. This article is not one of which justifies any click.
That person never wrote portable code. I wouldn't envy anyone maintaining or refactoring it to support more platforms.
Main reasons to have dependency injection is to have properly runtime switchable code with less shared state.
That it makes testing easier is a side effect.
If you are making classes or methods public because you want to unit test them, you're doing it wrong.
I've heard this a lot, but how do I test private methods right? If my public method just calls 8 private methods (which each call out to a bunch of other private methods), how to I test to make sure all those private methods do what I want them to do so that I know which one is broken when my public method breaks.
That your public function calls private functions is an implementation detail. It does not matter, test the public function. Subsequently that tests the private functions for all relevant input.
There was recently a submission here on HN that talked about this and different perspectives on that topic. It's not a universally shared opinion, surprisingly.
This seems obviously correct to me and it's still not appreciated by many developer teams.
Then there's the world of behavior / property / invariant -based testing which slims down your tests to essentially data generation and testing observable behavior which seems like magic to people still.
I just try to be practical and make them protected so they can be called from the test class which lives in the same package. Perhaps adding a @VisibleOnlyForTesting annotation.
The reality is that these private functions are building blocks that can be easier to test. If you only test the public methods, testing gets a lot more challenging and the methods are more difficult to test and the tests classes get a lot of more complicated. You end up creating more mocks or other doubles and bending backwards.
> how to I test to make sure all those private methods do what I want them to do so that I know which one is broken when my public method breaks.
Others have pointed out that you test only public methods. The thing I'd add to that: Having large classes is a code smell. If your class is big, it is probably several classes in one. Break it up accordingly. Some of your private methods in the big class will become public methods in the small class.
IMHO the purpose of unit tests is to ensure the code is correct and allow you to safely refactor and introduce new functionality without breaking existing behavior. The purpose is not to pinpoint the exact line where the bug is, if a test fails.
What you describe is white-box testing, where the test is coupled to implementation details of a class. This gives you more fine-grained reporting in case of an error but it makes it impossible to refactor the code safely. So I don't think that is a worthy trade-off.
If you often have hard-to-locate bugs in particular large component, the solution is probably to refactor into smaller more loosely-coupled components with well-defined interfaces.
That's a sign that you're burrying too much functionality to make testing reasonable. It's a sign that you should start splitting that out to more generic functions.
Scala is great about making these functions generic. Java has an issue with this where lots of things are easy to get burried but hard to reason about pulling them out.
It reaks of the same arguement that DHH made about "i hate unit tests because I'm testing getters and setters" (From DHH's point.. yes it's tedious and unit tests on getters/setters are low signal.. but that's a defect in the langauge not with testing)
Some time ago I stopped debating the definition of "unit" in testing, and instead started focusing on whether my tests were fast, reliable, and provided a useful signal about the health of the system when they failed. I've been much happier since then.
I think the properties you list are good ones, but if you drop the word unit from your post the advice is just as good and you can't get mired in ontological discussions
Unit Tests are still underrated in my experience. It pains me when I see developers testing their functionality end to end, manually. Discover a formatting error in their input, then do this again. When I was working in the office I could see that easily 90% of a lot of developer's time is spent like this. Then I showed that a unit test can discover this, people are still in denial, that it was just for this small error. But software development is a lot about nitty gritty details. And unit testing fixes this.
Many developers seem to lack a focus on automation in the most basic sense of the word as well. While consulting, I see lots of backend developers constantly making a change, waiting for the server to reload with the new changes, switch to Postman/cURL/whatever, fire off their request and see if it's accurate. Rather than just having the request handler as a function that is under test with assertions, and have the test re-run on each change. So much time during a developers day is spent on just repetition, and they don't seem to care.
Once I wrote my third game in Unity, with unit tests coverage of about 80%. It was least buggy game in my entire career. It was also the hardest time as I had to start thinking more about creating testable components rather than messy spaghetti code I used to write before.
Since testing in game development is very much underrated, it would be interesting to hear more about this. Was this just as a hobby or is the game available somewhere? And have you ever done a demo/talk about how you've approached unit testing with Unity?
Games are likely to benefit significantly from unit tests: architecture is complex and likely to be improved by adding clean boundaries and layers for the sake of testability; supporting automation is likely to enable useful features (e.g. deterministic simulations); tests are likely to be meaningful and important.
As a gamedev who writes unit tests, I agree with GP. Games benefit from unit tests. That, however, doesn't change the systemic incentives that prevent tests from being written. The ways that gamedev is budgeted/funded is a more important factor than more tests = a more stable/less buggy game.
I’ve found that while I did spend a lot of time at first figuring out how to write testable code, I now am so much better at writing in this way that I am much more productive since everything has great test coverage. Saves me going back to modules I wrote 2 years ago to add features.
It pains me when I see developers assiduously write unit tests for every line of their code and every single bug in the code ends up encoded in those unit tests so that they fail when the bug is fixed.
Integration tests need to be the default and people need to learn when to use one or the other.
Are these bugs in the sense that the code's plain wrong or that the developer misunderstood the spec? Good unit tests (when written alongside the code they're testing) should go a long way towards preventing the former.
The latter. Also misunderstandings about APIs. IME these combined are usually something like 70% of all bugs so the ROI on unit tests is not particularly high.
I find integration tests paired with BDD do a good job of catching "code plain wrong" bugs as well as misused APIs and misunderstood specs. They're harder to build and run slower but the ROI is still higher.
Unit tests are extremely boring for people who don't particularly like programming but do it anyway as means to an end. It's just another piece of code you are writing instead of releasing the dopamine of the finished product. It's anticlimactic.
If you first write the test and then try to satisfy them(test driven development) can work to an extent as it can divide the large task into small pieces where you get a prize each time but it also alienates the developer from the larger picture, diminishing their value to the project since they no longer can put their intellectual output into the project.
But there's nothing long term about unit tests, they only verify a particular version of the code and must be tweaked whenever that code is changed to return different results. The "happy green checks" that really help in the long term are comprehensive type checks, reflecting a reasonably stable underlying design.
This reads to me as means/ends confusion. If you write your tests afterwards, to reify the implementation of the system, this is true. They may then require radical changes when the implementation changes. On the other hand, if you write your tests first to reflect business rules and requirements, then those tests will change only when the business rules and requirements change. (Which they will, and that's OK.)
I'm not an all-TDD-all-the-time type by any means. You kind of have to at least have something to start writing tests against or it'll just be compiler errors all day long. But for the projects where I've most consistently found success in building correct systems, writing tests as soon as the basic interfaces are in place has resulted in more flexible, more reliable tests that last over the long term.
As for type checks...isn't that what a compiler is for?
Don't you think that it also alienates you from the projects? Green checks are green checks anyway, you can no longer put your input into the project and the best strategy for you becomes the minimal viable effort to earn a green check.
Personally, I hate any system designed to give me lots of small bits of validation. I don't want to see green check marks, just show me any errors that I need to think about.
That's literally the same thing. I write a failing test, I fix it. Those are the errors to think about and the green check marks when I don't need to think about them.
It's the opposite. One is motivation from being completionist all-green checkmarks. Where checkmarks are good. I only want to see/think about errors. I don't care if it passed 100 tests or a 1000 tests successfully.
Edit: I get that logically they are the same, but that's true of a half-empty/half-full glass. Psychologically they are different.
In my experience, any complex and widely used library is dead in the water without unit tests - if you make a change, you WILL break some use case, ending up in an endless cycle of fixing your changes based on user feedback, and souring your users on your library, who will refuse to update.
If you don't believe this, just write some (good) unit tests, then make the change - it's very very likely they will catch some weird corner case you've forgotten even existed.
> Then I showed that a unit test can discover this
Yeah but the catch is that in order for your code to be "testable" you have to rewrite it completely in a way that comes with large sacrifices, making it much more confusing, abstract, larger and much harder to work with, in every way except testing it.
I think the focus that everything should be testable is a wrong one, but almost every codebase has functionality that can be easily unit tested without major refactors. For everything else, integration tests can already help a lot.
Ideally, everything should be testable or replaceable. The non testable parts of your code have network effects on the testability of other parts of the code that are dependent on those parts. This is where dependency injection (e.g via a constructor or function parameters) really shines.
I totally agree that if you have to choose (and usually you do, unless you have unlimited time and funding), choose to implement end to end integration tests first.
Imho one of the benefits of unit testing is precisely that you have to write/refactor your code to make it more testable and in the process you uncover and fix flaws in lacking abstraction and isolation.
My experience is the exact opposite of this. Modular code that has well defined boundaries and responsibilities can be mocked and tested in isolation. It can also be reasoned about in isolation, reducing complexity. New functionality can be assigned to the proper module because each module has a well defined purpose.
The simple verification aspect of unit tests makes writing code easier and more purposeful in my opinion. At a most basic level it's kind of like a small informal proof. When the code base get's complex and larger in size this is invaluable if unit tests are written correctly. Tests have to make sense and be written well just like any other piece of code.
It also helps form thinking about test cases and logic before writing code. Before I used exhaustive unit tests I would just start writing code right away and often times this would result in constant refactoring as the code evolved or during debugging. But unit tests encouraged me to scaffold out abstract functions and write tests before implementing the code. It got me thinking about what the logic and workflows then implementing the functions meant having the tests pass as verification.
So if you an existing, functional, code base, I totally agree. Rewrites in general are rarely worth doing. If you have a new code base, it can be incredibly valuable to design the code such that externalities can be injected.
Usually this means being able to swap out file system, RNG, system time, and networking layers. If your code uses a particular hardware platform, it can be useful to put the platform specific code behind an abstraction layer.
Not only does it make your code more unit testable, it makes your code much more ad hoc testable. Again, I would not refactor a large, non-disposable application just to achieve this.
If you write code that is not testable, how do you test it? What you propose is false dichotomy really. Code that isn't testable is almost always confusing, large, hard to work with etc.
> It pains me when I see developers testing their functionality end to end, manually
On the other hand I know developers that only unit test. Hundreds of lines of mock code, dozens of tests. No documentation, no sample configuration for the final application and dozens of data races because the unit tests don't cover multi threading. But hey an unusable buggy mess is a net positive as long as it makes the test coverage statistic happy and that is one of those things reported to management.
While a bit exaggerated one of the downsides of unit tests is that they quickly become part of a metric without regards for when they ad value.
I agree that automated integration tests typically offer higher ROI than equivalent unit tests written with mocks, because mock code is so difficult to write and fragile.
I disagree with your assertion that integration tests offer higher value than unit tests in general. Don't judge all unit tests by mock code.
I will counter that integration tests tend to have a much bigger footprint: one test covers a lot more ground and so per line of code will catch more things.
But, the failures from integration tests are often a lot harder to read. Even half-good unit tests tend to show exactly where the error is, and make the fix cycle much shorter. It is also much harder to write exhaustive integration tests for all of the inputs that might happen.
I, as a long-term qa-ish person (I usually write testing systems) see real value in having a mixture of both system, integration (multiple systems), and unit tests, as well as end-to-end manual tests. Anything big enough needs all of these.
If errors from your integration tests are hard to understand, that just means you need to improve the debugging information emitted by your software such that your developer can debug it properly using it, no?
> and that is one of those things reported to management.
Yeah unit testing as I see it is just optimising for the success of middle management, at the expense of both the organisation's goals, and the developers.
You are making predictability the only focus, because that's the metric which middle management is evaluated by. At the expense of performance, end product quality, and work satisfaction. "It was done in exactly the excruciatingly slow and mind numbing boring way, and was exactly as shitty as designed" = success for management.
If you would try to increase performance, end product usability, and/or work satisfaction, you would have to put the predictability at risk, which means that the success of the middle manager will not be the highest priority at all cost anymore.
Your comment is downvoted perhaps because it sounds so cynical, but after thinking about it I agree to a certain extent. Throughout my career I've noticed the biggest pushers for more unit testing are usually middle managers who love to see that coverage number goes up since it's an observable metric, but nuanced opinions like "more unit tests are not always good and can be detrimental" are generally not welcome.
yep. I've worked at places with unit tests/e2e/and static typing. And I've worked at places with zero unit tests/no e2e/dynamic typing.
The push for more testing and static typing (to degree) was always managers. They trade predictability for momentum.
The reality in both situations is that bugs happen. I've not noticed a difference in frequency of "critical" bugs in either scenario. The only difference is that at the no unit test/no e2e places, getting your code into production was trivial. Which always meant when bugs happen, they are fixed much faster than the unit test-everything place. We're talking minutes vs. days here. Because at a risk-adverse business (that is, any place more than a few hundred people), you have multiple stakeholders that have to sign off if you want to go take a piss in the bathroom.
While I totally agree that process can go nuts and get in the way, I will disagree that testing always makes things slower. Once you are past the prototyping phase, and get into actual production phase (some projects never make this turn) some testing infrastructure can absolutely make thing faster. Rather than having to deal with all of the bugs that your fellow developers are just dumping into the main branch, they can help weed out some of the bugs before others have to deal with them.
That is one of the big benefits I have found from writing code while planning to write unit tests for it (even if you don't write all of them). You will generally avoid patterns that will require a lot of mocks to test, which generally make that code easier to refactor and reason about. If you are able to make big chunks of your code into "pure" functions, they are now really easy to test.
A classic approach is that if you have to make some complicated API call with a bunch of arguments, you can put all the logic wrangling the arguments into a pure helper function and unit test the helper.
I use strategy pattern with Java functional interfaces to avoid using magic mocking frameworks. That way, my mocks are just trivial impls of the plugin parts.
Often times, you will have chunks of code that are either creating input for what would go into an interface/mock, or handling the output of it. You could pull those bits, even if they are relatively simple (maybe only a few lines, and only used in that one spot), and extract those out to a pure function to sidestep the mock.
Sometimes you can't really avoid it. In those cases, I would judge how much of the effort is testing your code, and how much is testing your ability to write mocks. Unless it is a really important functionality, or something complex enough that it would be easy to get wrong, I wouldn't bother with most tests that need extensive mocking. Instead handle those later with integration / e2e tests.
Also can be pretty handy when there's hardware that needs something very specific to happen to work properly. Mocks can give you (some) confidence that when you poke the I2C bus, you won't hang it up.
But it's very easy to overdo it and end up testing irrelevant implementation details instead.
I don't think it's ever a good idea to mock a database. Invest in a test framework that can performantly spin up a fresh database instance and throw it away at the end of the test run instead.
Mocking outbound HTTP calls is worthwhile, but thankfully most testing frameworks I've used (at least in Python world) make that pretty easy.
I mock database by using sqlite in-memory database. It is not 100% compatible with my production database but still god enough for most of my tests. This way I have fresh database for each unit test and the tests are still fast.
Property based tests have the benefit of being fun to actually think about (as far as testing goes) and also often exposing more bugs than unit tests (which are usually more like sanity checks than exploring the boundary of what will break or turn weird your functions).
Property based tests have caught 10-100x the number of buggy edge-cases than the handwritten unit tests I could have come up with. The material gains of getting a combinatorial explosion of test cases from property composition is extremely underrated.
However, there seems to be a general skepticism around property based tests in my experience. Any idea where it stems from?
IMO, you have to code in a certain way to make them ergonomic or even worthwhile - which is also true for example tests. They have some overlap with example tests but have a different advantage. Example tests document function/interface usage and typical or historical errors/exceptions. Property based tests are more about data integrity and invariants.
one example: sqlite3 has more testing code than its own code, so it can swap the whole design at will once a while, it is working there. unit testing is essential to me.
>Tests, whose scope lies higher on Mike Cohn’s pyramid are either written as part of a wider suite (often by completely different people) or even disregarded entirely.
My experience is the exact opposite. Actually my experiences are the opposite of many things in this article.
I'm wondering if that's part of the problem, the fact that unit testing started out in Smalltalk. It does make a lot more sense in a dynamic language, because any line that's not touched is a potential typo.
Tests are an extension of documentation, in my opinion.
I go on Github and I read the readme then I go to the test folder and try to see some valid usage examples.
This is what writing useful tests mean, from a client perspective I just want to understand what's valid and what's invalid.
Also useful to make sure we fixed specific scenario bugs.
I've seen a lot of people doing things wrong but we can summarise in two cases:
- "We've got 100% coverage, it must mean we've got no bugs, right?"
- We've got tests covering happy path, everything else is tested manually. It takes forever and it's not shared anywhere, but "trust me" I've tested this.
I don't like:
- manual testing
- fixing bugs found in production under extreme time pressure
- people finding embarrassing bugs in my code
- spending lots of time explaining what code should be doing to other developers
Unit testing massively reduces all of these. Therefore I assume people that don't like unit tests either enjoy doing these things or are masochistic.
As a predominantly fullstack web developer, I will only ever voluntarily write unit tests and end to end tests, but avoid in-code integration tests in most circumstances.
This is frankly the easiest and best bang for buck.
Unit tests force you to make your code more flexible, they're fast to write, fast to run. Maintenance depends on how overboard you go with your verifications - try to verify the most important parts.
End to end tests require no elaborate scaffolding of internals and allow you to test it as a real user interacts with it. Generally fast to make, slow to run, and maintenance depends on how disciplined you are with stable identifiers/interaction points.
As for in-code integration tests - I love the idea of them, but they're absolutely miserable 9 times out of 10 due to extremely convoluted processes to "bring up" parts of your application. If you use DI it shouldn't be as bad, but almost nobody does and it becomes a total clusterfuck not worth the maintenance burden.
As an idealist, I want all 3. But most codebases frankly cannot support all 3, so the most value will come from whatever is easiest. I've found that's most often unit and e2e.
The article actually argues the opposite. Developers should move their focus to integration / "real-world" tests. The major summary bullet point being:
"Aim for the highest level of integration while maintaining reasonable speed and cost"
My experience mirrors the author's. In any "real" business application, the unit tests end up mocking so many dependencies that changes become a chore, in many cases causing colleagues to skip certain obvious refactors because the thought of updating 300 unit tests is out of the question. I've found much better success testing at the integration level. And to be clear this means writing a tests inside the same project that run against a database. They should run as part of your build, both locally and in CI. The holy grail is probably writing all your business logic inside pure functions, and then unit testing those, while integration testing the outer layers for happy and error paths. But good luck trying to get your coworkers to think in pure functions.
> in many cases causing colleagues to skip certain obvious refactors because the thought of updating 300 unit tests is out of the question.
Good! They shouldn't do the refactor.
Because "obvious" refactors often introduce bugs (e.g. copy/paste errors), and if developers can't be bothered to write tests to catch them, they're going to screw over the other team members and users who will be forced to deal with their bugs in production.
> The holy grail is probably writing all your business logic inside pure functions, and then unit testing those, while integration testing the outer layers for happy and error paths.
So settle for half a loaf.
Write all the easy unit tests first. The coverage will be very incomplete, but something is better than nothing.
> Because "obvious" refactors often introduce bugs (e.g. copy/paste errors), and if developers can't be bothered to write tests to catch them, they're going to screw over the other team members and users who will be forced to deal with their bugs in production.
In my opinion, useful tests should be able to survive a refactor. That is the only sane way I've ever done refactoring.
If I'm doing a large refactor on a project and there are no tests, or if the tests will not pass after the refactor, the first thing I do is write tests at a level that will pass both before and after refactoring.
Rewriting tests during refactoring doesn't protect from regression on my experience.
Unit tests which can survive a refactor are a nice-to-have.
I would not rule out a refactor merely because I'd have to refactor some unit tests too. That's just part of the cost benefit analysis.
> Rewriting tests during refactoring doesn't protect from regression on my experience.
Your experience is completely at odds with mine. Every time I change code, there is the possibility for simple errors such as copy/paste mistakes. Trivial, cheap-to-write unit tests have saved me time and again from having to debug something down the line.
Overconfident devs who act as though they're above making such simple mistakes make for bad team members.
Tests that will survive a refactor are the most important tests to have.
The other tests are, at best, a false sense of security and often an active detriment that slows down future development. They might sometimes catch actual mistakes, but just as often they fail when nothing is broken, leading to the tests not being trusted and broken tests being updated even when something was actually broken.
I know where you're coming from. This is the classic argument for limiting unit tests to black-box testing of public APIs exclusively, avoiding clear-box testing altogether.
I agree that it's possible to write absolutely wretched fragile clear-box tests. And I agree that if you have a black-box test and a clear-box test which provide equal validation of functionality, the black-box test is superior because it will survive a refactor.
I generally dislike absolutist rules of any kind when it comes to unit testing and prefer to think of things in terms of ROI. Sometimes you can add a lot of value with a clear-box test because the functionality is impossible to write a black-box test for without a ton of extra work and time.
But sometimes you may be in an environment where absolutist rules are the only way to go.
I've had the same experience. Suboptimal code isn't refactored because of the test code overhead, or, much worse, the tests on that same subpar code somehow morph into a perceived "gold standard" for how that code should work.
I avoid tests (aside from hands-on end user testing) as much as possible, actually, since they rarely seem to tell you anything you'd didn't already know.
I understand what the article is arguing. I agree with it, but think it's idealistic. If swaths of your code are a mess, integration testing is super painful. You can't easily add it until you clean up the mess, so the other forms of testing are more practical more often in my experience. If you get to a point where your code isn't a mess, I'd agree that you should start introducing meaningful integration tests.
I think this is just one of those cases where there is a context-sensitive strategy to testing. It depends completely on the cleanliness of your code and experience working with it.
> The holy grail is probably writing all your business logic inside pure functions, and then unit testing those, while integration testing the outer layers for happy and error paths. But good luck trying to get your coworkers to think in pure functions.
I've come to a similar conclusion. Functions don't necessarily have to be pure in the academical sense, though - but I feel like the more the business logic is decoupled from dependency injection and the less it is relying on some framework, the better.
It makes testing a lot easier, but also code reuse. I've just been writing a one-off migration script where I could simply plug in parts of the core business logic. It would have been very annoying if that was relying on Angular, NestJS or whatever.
"Overrated" is the right word here. It's not like unit tests are useless, instead, the problem is that everybody uses them much more than it makes sense. And yes, I'm a critic of them since the fashion started at the turn of the century and I also use them more than they make sense.
Unit tests are very cheap, and people love cheap things. The problem is that they also provide almost no value, so their cost and benefit are on similar levels and they can easily get into negative net value.
> As for in-code integration tests - I love the idea of them, but they're absolutely miserable 9 times out of 10 due to extremely convoluted processes to "bring up" parts of your application.
I've definitely worked on projects like this (actually, I'm working on one now at $dayjob), but I found that with a little bit of effort you can get this right, and usually isn't that much effort.
What I often see people do is "oh I need a thingy for this test, and a thaty for this other test" and create it ad-hoc when needed, instead of creating a good convenient API to do all of this, which is then also used in the application itself.
The big advantage of these tests is that they tend to be a lot faster than E2E tests, at least in my experience, and often also easier to reason about once you get it right.
I disagree. I also feel UTs are a waste of time. I think in some cases UTs can work such as helper classes, but for business logic they are pretty useless and a waste of time. I guess it gets at that verification level DEPENDS ON what is being verified.
Unit tests are essential. I've developed some devops code around self-healing where the degrading signals were not reproducible. We knew they occurred because we lost a non-trivial number of nodes a day. A weird set of bugs between Linux, Kubernetes & Docker. The problem was we could recognize the signals and take action. The entire daemon I created to trap and execute on these was built and deployed on unit tests. In fact, I had more lines of unit test code than actual functional code because I had to mock the hell out of what the actual system would look like.
Another situation - where my wife used to work in medical devices. Feature development velocity was a problem because there was a queue on the very limited set of devices (CT scanners, PET scanners, etc.) that the team could use for testing. Debugging was very hard. Fixing bugs was hard because to debug you needed a device. With unit testing and mocks they made a lot of contention on the devices go away.
Write your unit tests. It will help you and the people coming in after you to maintain the code.
unit testing is essential, TDD might be a stretch in practice.
instead of having separate test/ directory with multiple unit test files, I put unit test code right in the source code(at the end of the file), so when I modify the source I can easily update the unit test in the same place, and they never get out of sync. It worked for me very well. They can be turned on/off easily too. Sometimes it's easier to understand the code by reading the unit tests than reading the whole lengthy code, and it's helpful when they're in the same place. For me, unit test code is part of the real code, not something 'additional'.
I was building an API and started adding some tests, then added coverage and It got me to think about the complex-ness of the codebase, how many branches / statements I had per file to test etc, I think it helps greatly to reason about modular and good software. But it takes time too.
YMMV
Unit testing is a tool, you don't need a 100% test coverage. But sometimes it's the right tool for the job. Sometimes it's not. If I have a piece of code... usually an algorithm that handles a handful of different use cases, and making a change might unexpectedly cause an issue to another use case, i'll unit test the heck out of it. If I have some code where it has a simple input, and output but it takes some time to test it, i'll write a unit test to make development faster.
But i'm not going to add an interface to every concrete class in my project, and design literally every component so I can mock it.
Writing software is a business, i can sprinkle TDD around for a high ROI. If I use it EVERYWHERE the ROI is very little if not negative.
Very much agree with "sprinkle TDD around for a high ROI"
This requires both developer expertise and trust from stakeholders.
But, VeryBigHugeBankCo requires me to have 70% unit-test coverage for 'new' code, as measured by Sonar or whatever, or else I am simply not able to deploy my change to PROD.
So here I am refactoring a DAO, using strategy pattern and functional interfaces, so that I can cover my DB interactions adequately for that 70% metric.
No, I can't just use H2 DB, as the SQL is Oracle-specific.
Yes, there is very little actual value in the new refactor, except to check the box.
Oh and don't get me started about how, especially when using mocking frameworks, the test code is sometimes wayyyy harder to understand than the actual code under test. NOT A WIN.
I have told mgmt that test code is equally likely to have bugs as production code, but, again, this is just box-checking, not actual attention to detail.
It gets worse than that. Often, code is pushed into an absurdly wrong level of the development stack, just because it makes testing it easier. For example, something should be a database trigger, but in-memory testing database gets in the way. Other times it strikes all the way to architectural topics - message queues replaced by REST calls, because the former is difficult to mock
Used to feel this way and largely agree when there are less contributors on the project or the tests are so trivial like things doing what they obviously do. However, unit tests can describe behavior and written succinctly can be beneficial for other people’s eyeballs. I try not to get into discussions about “clean” or “elegant” code for the most part, as most people prefer the smell of their own farts.
232 comments
[ 3.2 ms ] story [ 239 ms ] threadUnit Testing Is Overrated - https://news.ycombinator.com/item?id=23778878 - July 2020 (387 comments)
In fact the whole thing then becomes a standalone function which takes Location and other relevant parameters and returns the computer sunrise/sunset values. Pure function and super easy to unit test.
If one needs to do a lot of "mockups" for your unit tests then maybe one needs to consider the API and class design. Removing needless coupling helps testability by removing the need to use mocks in the first place.
A role of one unit inside this component is to use the provided coordinates and only perform the calculation.
Unit testing is not overrated, if you feel this way you likely just suck at optimising your suite for high RoI tests or chase some stupid metric like code coverage getting mad when you find out you wasted a bunch of time writing worthless tests.
https://youtu.be/z9quxZsLcfo
Figuring out where your bug is is so much harder when you have to do it though the lens of other code, whereas in your unit test, you can just see "oh, I'm returning foo.X, not foo.Y".
It also makes sure you actually can construct your object at all without dragging in a dependency on the entire system. Code without tests tends to accrete things that can only be set up in a very long and complex process. This is both hard to reason about (because your system can only be thought about as an ensemble) and fragile when the system changes: you now have to unpick the dependencies to allow your "SendEmail" function to not somehow depend on an interface to some SPI hardware!
But there's certainly value in not spending hours testing obvious code to death: a getter is almost certainly fine, and even if it isn't, the first "real" test that uses it will fail if you're returning the wrong thing. But if, down the line, you do find a bug in it, then something was probably not that obvious!
It's also valuable to have some testing framework available, just so that it's then easy to write tests when they're needed (which comes back around to "make sure that your objects can ever be constructed" thing). Not all unit tests have to be preemptive, even if the presence of the ability to quickly write them is.
In Unit testing, you should only be testing the code you wrote. (Contract testing is more of an assurance test) In Java you have to write unit tests for your getters and setters because they might do more than what you expected (as that's within the realm of possiblity) If you find yourself writting getters or setter tests .. you should be looking into a better language like Groovy, Scala that do it for you, or something like Lombok or Records in later JDKs.
Yes, what you need to do in the end, is to use your judgement. But you will get a surprising amount of pushback against this obvious common sense solution. From management who thinks it's too risky, and from developers who have chosen a technical career just because they want to rely on their intelligence, and black and white thinking, and now you're asking them to make trade offs and decisions which is management territory. The way out in my opinion is to make technical leaders that have actual authority, down to the details.
I'm working on an app that has 200+ tables and requires a lot of data to be in the database before anything even works at all. Of course good tests interacting with the database need a fresh database every time so even a test suite of about 10 tests can take 10-20 minutes already.
Slower, but the test suite is far more effective at catching bugs.
These tests do take more up front engineering but since they arent pale imitations of the code they also dont need to be totally rewritten every time the code is refactored. Higher capex, lower opex.
No, all you have to do is spin up your service under test, its database and the service bus/event hub/whatever you use to communicate between services.
Nowadays with Docker it's really easy and fast to spin up test instances during your test run. (4-5 LoC with a good library in C#).
Since 1 or 2 years I'm doing the exactly same thing on my projects and it works great to speed-up development process and if a bug pops in production it's really easy to add it as a future test case.
You're only attached to the input request (endpoint, query params, request body) and the response body. But that's ok because it's already a part of the contract between the API and its consumers. That's it.
"Considering the factors mentioned above, we can reason that unit tests are only useful to verify pure business logic inside of a given function."
That just isn't true and it makes the rest of the blogpost also not true.
A unit test should test "a unit of functionality" not just a method or a class. Your unit tests also shouldn't be coupled to the implementation of your unit of functionality. If you are making classes or methods public because you want to unit test them, you're doing it wrong.
The exception is maybe those tests you are writing while you're doing the coding. But you don't have to keep them around as they are.
> it makes the rest of the blogpost also not true
what do you know of the rest of the blogpost if you stopped reading?
HOWEVER, I have skimmed through it by now and the last paragraph is actually quite good. It takes a while to get there and the chosen examples aren't great, though.
> That just isn't true and it makes the rest of the blogpost also not true.
It's not? Writing unit test is all about minimizing side effects and interaction with other systems.
Not really. Writing unit tests is all about verifying behavior from combinations of inputs, and side effects are inputs as well.
You can write unit tests that inject delays and timeouts and retries and throw exceptions under specific circumstances.
Well, except it's not. If you use a side effect, you have to account for it in some way via mocks or whatever.
Me too. The author shows a deep misunderstanding and even obliviousness to testing, particularly from a practical point of view, that takes any credibility from any argument listed in the article.
It's even perplexing how the best example the author could come up with was this absurd chain of strawmen that a) it's impossible to test code that uses instances of HttpClient, b) dependencies used in dependency injection "serve no practical purpose other than making unit testing possible."
There are plenty of people talking about unit tests. This article is not one of which justifies any click.
Main reasons to have dependency injection is to have properly runtime switchable code with less shared state. That it makes testing easier is a side effect.
I've heard this a lot, but how do I test private methods right? If my public method just calls 8 private methods (which each call out to a bunch of other private methods), how to I test to make sure all those private methods do what I want them to do so that I know which one is broken when my public method breaks.
There was recently a submission here on HN that talked about this and different perspectives on that topic. It's not a universally shared opinion, surprisingly.
Then there's the world of behavior / property / invariant -based testing which slims down your tests to essentially data generation and testing observable behavior which seems like magic to people still.
You don’t. You test only the public API. This way, you can refactor the public method to your heart’s content without any tests breaking.
The reality is that these private functions are building blocks that can be easier to test. If you only test the public methods, testing gets a lot more challenging and the methods are more difficult to test and the tests classes get a lot of more complicated. You end up creating more mocks or other doubles and bending backwards.
So how do you test private methods? You don't, with the exception of testing during development, but those aren supposed to be kept around.
Others have pointed out that you test only public methods. The thing I'd add to that: Having large classes is a code smell. If your class is big, it is probably several classes in one. Break it up accordingly. Some of your private methods in the big class will become public methods in the small class.
What you describe is white-box testing, where the test is coupled to implementation details of a class. This gives you more fine-grained reporting in case of an error but it makes it impossible to refactor the code safely. So I don't think that is a worthy trade-off.
If you often have hard-to-locate bugs in particular large component, the solution is probably to refactor into smaller more loosely-coupled components with well-defined interfaces.
Scala is great about making these functions generic. Java has an issue with this where lots of things are easy to get burried but hard to reason about pulling them out.
Here are qualities of good unit tests:
A unit test that takes 1/10th of a second to run is a slow unit test.The test that is too short is most likely not being thorough enough, or the functionality it tests is so simplistic the test likely brings no value.
https://www.artima.com/weblogs/viewpost.jsp?thread=126923
I think the properties you list are good ones, but if you drop the word unit from your post the advice is just as good and you can't get mired in ontological discussions
https://m.youtube.com/watch?v=m5zrfTFKf_E
Integration tests need to be the default and people need to learn when to use one or the other.
I find integration tests paired with BDD do a good job of catching "code plain wrong" bugs as well as misused APIs and misunderstood specs. They're harder to build and run slower but the ROI is still higher.
You should know what result you want before you even start writing the code.
In theory, but IME about 50% of bugs end up being specification bugs at the high level anyway.
Once you drill down and start writing unit tests on lower levels to imitate higher level APIs that % only goes up.
If you first write the test and then try to satisfy them(test driven development) can work to an extent as it can divide the large task into small pieces where you get a prize each time but it also alienates the developer from the larger picture, diminishing their value to the project since they no longer can put their intellectual output into the project.
More green checks = more dopamine Higher quality code = long term satisfaction.
Code does things, tests verify that it does the things I thought it did. Getting a bit of validation makes me feel good while working.
If it now needs to do completely different things then it needs different tests.
Also, if you are making huge changes to tests whenever you change a single detail, that's an issue with your code, not testing in general.
Type checks are good too. Using a typed language is even better.
I'm not an all-TDD-all-the-time type by any means. You kind of have to at least have something to start writing tests against or it'll just be compiler errors all day long. But for the projects where I've most consistently found success in building correct systems, writing tests as soon as the basic interfaces are in place has resulted in more flexible, more reliable tests that last over the long term.
As for type checks...isn't that what a compiler is for?
That's literally the thesis of the Kent Beck book.
I'm not some idiot copy and pasting `expect(true).toEqual(true)`
I add functionality one test at a time. I get satisfaction of knowing it works as specified in the form of an increasing count of green checks.
I don't understand why it would alienate me.
Edit: I get that logically they are the same, but that's true of a half-empty/half-full glass. Psychologically they are different.
As you say, you don't care that there are 20k passing tests. You care just care that there is 1 failure.
If you don't believe this, just write some (good) unit tests, then make the change - it's very very likely they will catch some weird corner case you've forgotten even existed.
Yeah but the catch is that in order for your code to be "testable" you have to rewrite it completely in a way that comes with large sacrifices, making it much more confusing, abstract, larger and much harder to work with, in every way except testing it.
I totally agree that if you have to choose (and usually you do, unless you have unlimited time and funding), choose to implement end to end integration tests first.
It also helps form thinking about test cases and logic before writing code. Before I used exhaustive unit tests I would just start writing code right away and often times this would result in constant refactoring as the code evolved or during debugging. But unit tests encouraged me to scaffold out abstract functions and write tests before implementing the code. It got me thinking about what the logic and workflows then implementing the functions meant having the tests pass as verification.
Not only does it make your code more unit testable, it makes your code much more ad hoc testable. Again, I would not refactor a large, non-disposable application just to achieve this.
On the other hand I know developers that only unit test. Hundreds of lines of mock code, dozens of tests. No documentation, no sample configuration for the final application and dozens of data races because the unit tests don't cover multi threading. But hey an unusable buggy mess is a net positive as long as it makes the test coverage statistic happy and that is one of those things reported to management.
While a bit exaggerated one of the downsides of unit tests is that they quickly become part of a metric without regards for when they ad value.
They tend to have much higher value than pure unit tests.
I disagree with your assertion that integration tests offer higher value than unit tests in general. Don't judge all unit tests by mock code.
But, the failures from integration tests are often a lot harder to read. Even half-good unit tests tend to show exactly where the error is, and make the fix cycle much shorter. It is also much harder to write exhaustive integration tests for all of the inputs that might happen.
I, as a long-term qa-ish person (I usually write testing systems) see real value in having a mixture of both system, integration (multiple systems), and unit tests, as well as end-to-end manual tests. Anything big enough needs all of these.
Yeah unit testing as I see it is just optimising for the success of middle management, at the expense of both the organisation's goals, and the developers.
You are making predictability the only focus, because that's the metric which middle management is evaluated by. At the expense of performance, end product quality, and work satisfaction. "It was done in exactly the excruciatingly slow and mind numbing boring way, and was exactly as shitty as designed" = success for management.
If you would try to increase performance, end product usability, and/or work satisfaction, you would have to put the predictability at risk, which means that the success of the middle manager will not be the highest priority at all cost anymore.
The push for more testing and static typing (to degree) was always managers. They trade predictability for momentum.
The reality in both situations is that bugs happen. I've not noticed a difference in frequency of "critical" bugs in either scenario. The only difference is that at the no unit test/no e2e places, getting your code into production was trivial. Which always meant when bugs happen, they are fixed much faster than the unit test-everything place. We're talking minutes vs. days here. Because at a risk-adverse business (that is, any place more than a few hundred people), you have multiple stakeholders that have to sign off if you want to go take a piss in the bathroom.
I use strategy pattern with Java functional interfaces to avoid using magic mocking frameworks. That way, my mocks are just trivial impls of the plugin parts.
Sometimes you can't really avoid it. In those cases, I would judge how much of the effort is testing your code, and how much is testing your ability to write mocks. Unless it is a really important functionality, or something complex enough that it would be easy to get wrong, I wouldn't bother with most tests that need extensive mocking. Instead handle those later with integration / e2e tests.
I've seen code so mocked it was dubious that it could be testing anything, accurately at least.
Mocking should be kept to a minimum and ideally avoided.
Mocking is most useful when adding tests to legacy code, but new code should be designed so it can be tested without the need for mocking.
But it's very easy to overdo it and end up testing irrelevant implementation details instead.
Mocking outbound HTTP calls is worthwhile, but thankfully most testing frameworks I've used (at least in Python world) make that pretty easy.
No wonder they are not getting value out of unit tests.
https://blog.metaobject.com/2014/05/why-i-don-mock.html
However, there seems to be a general skepticism around property based tests in my experience. Any idea where it stems from?
This software is still buggy and broken.
Ergo, unit testing is not working.
My experience is the exact opposite. Actually my experiences are the opposite of many things in this article.
However you need integration tests and end-to-end tests to improve software correctness.
This is what writing useful tests mean, from a client perspective I just want to understand what's valid and what's invalid.
Also useful to make sure we fixed specific scenario bugs.
I've seen a lot of people doing things wrong but we can summarise in two cases: - "We've got 100% coverage, it must mean we've got no bugs, right?" - We've got tests covering happy path, everything else is tested manually. It takes forever and it's not shared anywhere, but "trust me" I've tested this.
Unit testing massively reduces all of these. Therefore I assume people that don't like unit tests either enjoy doing these things or are masochistic.
Consider trying to write tests in a non-trivial Spring Boot app without mocks
https://quii.gitbook.io/learn-go-with-tests/meta/why
The book is Go specific but there's a lot of wisdom in this particular chapter that applies to any language.
This is frankly the easiest and best bang for buck.
Unit tests force you to make your code more flexible, they're fast to write, fast to run. Maintenance depends on how overboard you go with your verifications - try to verify the most important parts.
End to end tests require no elaborate scaffolding of internals and allow you to test it as a real user interacts with it. Generally fast to make, slow to run, and maintenance depends on how disciplined you are with stable identifiers/interaction points.
As for in-code integration tests - I love the idea of them, but they're absolutely miserable 9 times out of 10 due to extremely convoluted processes to "bring up" parts of your application. If you use DI it shouldn't be as bad, but almost nobody does and it becomes a total clusterfuck not worth the maintenance burden.
As an idealist, I want all 3. But most codebases frankly cannot support all 3, so the most value will come from whatever is easiest. I've found that's most often unit and e2e.
"Aim for the highest level of integration while maintaining reasonable speed and cost"
My experience mirrors the author's. In any "real" business application, the unit tests end up mocking so many dependencies that changes become a chore, in many cases causing colleagues to skip certain obvious refactors because the thought of updating 300 unit tests is out of the question. I've found much better success testing at the integration level. And to be clear this means writing a tests inside the same project that run against a database. They should run as part of your build, both locally and in CI. The holy grail is probably writing all your business logic inside pure functions, and then unit testing those, while integration testing the outer layers for happy and error paths. But good luck trying to get your coworkers to think in pure functions.
Good! They shouldn't do the refactor.
Because "obvious" refactors often introduce bugs (e.g. copy/paste errors), and if developers can't be bothered to write tests to catch them, they're going to screw over the other team members and users who will be forced to deal with their bugs in production.
> The holy grail is probably writing all your business logic inside pure functions, and then unit testing those, while integration testing the outer layers for happy and error paths.
So settle for half a loaf.
Write all the easy unit tests first. The coverage will be very incomplete, but something is better than nothing.
Write all the easy integration tests next.
Never write the hard tests if you can help it.
> Because "obvious" refactors often introduce bugs (e.g. copy/paste errors), and if developers can't be bothered to write tests to catch them, they're going to screw over the other team members and users who will be forced to deal with their bugs in production.
In my opinion, useful tests should be able to survive a refactor. That is the only sane way I've ever done refactoring.
If I'm doing a large refactor on a project and there are no tests, or if the tests will not pass after the refactor, the first thing I do is write tests at a level that will pass both before and after refactoring.
Rewriting tests during refactoring doesn't protect from regression on my experience.
I would not rule out a refactor merely because I'd have to refactor some unit tests too. That's just part of the cost benefit analysis.
> Rewriting tests during refactoring doesn't protect from regression on my experience.
Your experience is completely at odds with mine. Every time I change code, there is the possibility for simple errors such as copy/paste mistakes. Trivial, cheap-to-write unit tests have saved me time and again from having to debug something down the line.
Overconfident devs who act as though they're above making such simple mistakes make for bad team members.
Tests that will survive a refactor are the most important tests to have.
The other tests are, at best, a false sense of security and often an active detriment that slows down future development. They might sometimes catch actual mistakes, but just as often they fail when nothing is broken, leading to the tests not being trusted and broken tests being updated even when something was actually broken.
*sigh*
I know where you're coming from. This is the classic argument for limiting unit tests to black-box testing of public APIs exclusively, avoiding clear-box testing altogether.
I agree that it's possible to write absolutely wretched fragile clear-box tests. And I agree that if you have a black-box test and a clear-box test which provide equal validation of functionality, the black-box test is superior because it will survive a refactor.
I generally dislike absolutist rules of any kind when it comes to unit testing and prefer to think of things in terms of ROI. Sometimes you can add a lot of value with a clear-box test because the functionality is impossible to write a black-box test for without a ton of extra work and time.
But sometimes you may be in an environment where absolutist rules are the only way to go.
& if your tests arent catching those bugs and require extra maintenance to go green again you are doing them wrong.
I avoid tests (aside from hands-on end user testing) as much as possible, actually, since they rarely seem to tell you anything you'd didn't already know.
I think this is just one of those cases where there is a context-sensitive strategy to testing. It depends completely on the cleanliness of your code and experience working with it.
I've come to a similar conclusion. Functions don't necessarily have to be pure in the academical sense, though - but I feel like the more the business logic is decoupled from dependency injection and the less it is relying on some framework, the better.
It makes testing a lot easier, but also code reuse. I've just been writing a one-off migration script where I could simply plug in parts of the core business logic. It would have been very annoying if that was relying on Angular, NestJS or whatever.
Then you get to use fuzzer & Arbitrary for basically what is a coverage guided property based test.
But it's hard to maintain that idea at all times when you are writing.
Unit tests are very cheap, and people love cheap things. The problem is that they also provide almost no value, so their cost and benefit are on similar levels and they can easily get into negative net value.
I've definitely worked on projects like this (actually, I'm working on one now at $dayjob), but I found that with a little bit of effort you can get this right, and usually isn't that much effort.
What I often see people do is "oh I need a thingy for this test, and a thaty for this other test" and create it ad-hoc when needed, instead of creating a good convenient API to do all of this, which is then also used in the application itself.
The big advantage of these tests is that they tend to be a lot faster than E2E tests, at least in my experience, and often also easier to reason about once you get it right.
Another situation - where my wife used to work in medical devices. Feature development velocity was a problem because there was a queue on the very limited set of devices (CT scanners, PET scanners, etc.) that the team could use for testing. Debugging was very hard. Fixing bugs was hard because to debug you needed a device. With unit testing and mocks they made a lot of contention on the devices go away.
Write your unit tests. It will help you and the people coming in after you to maintain the code.
instead of having separate test/ directory with multiple unit test files, I put unit test code right in the source code(at the end of the file), so when I modify the source I can easily update the unit test in the same place, and they never get out of sync. It worked for me very well. They can be turned on/off easily too. Sometimes it's easier to understand the code by reading the unit tests than reading the whole lengthy code, and it's helpful when they're in the same place. For me, unit test code is part of the real code, not something 'additional'.
But i'm not going to add an interface to every concrete class in my project, and design literally every component so I can mock it.
Writing software is a business, i can sprinkle TDD around for a high ROI. If I use it EVERYWHERE the ROI is very little if not negative.
But, VeryBigHugeBankCo requires me to have 70% unit-test coverage for 'new' code, as measured by Sonar or whatever, or else I am simply not able to deploy my change to PROD.
So here I am refactoring a DAO, using strategy pattern and functional interfaces, so that I can cover my DB interactions adequately for that 70% metric.
No, I can't just use H2 DB, as the SQL is Oracle-specific.
Yes, there is very little actual value in the new refactor, except to check the box.
This is how we do...
I have told mgmt that test code is equally likely to have bugs as production code, but, again, this is just box-checking, not actual attention to detail.
Why would using an in-memory database prevent use of triggers?