I genuinely wanted a good critique (It's nice to get dissenting viewpoints) but reading text so charged with anger and so scant in information is stressing me out.
agreed - there are certainly some disadvantages to TDD, and I was hoping for an discussion raising some ideas I'm yet to formulate fully.
Instead there were lots of parallels that I don't recognise.
It was disappointing the author didn't seem to try and find the advantages in TDD and discuss a counter argument or alternative, such as modifying a foreign code base.
OK, I'll give it a shot. This isn't a criticism of TDD per se, but of over-reliance on code coverage tools.
Because it's easy to measure whether you're exercising every code path, software engineers think in terms of how close they are to "100% coverage" of tests. What you really should be thinking about is whether you're exercising every possible input, not every possible code path.
I think you're missing the most important one: ensuring you don't unexpectedly break your own code in the future (when you come back in 6 months and forget why exactly it's arr[1:n-1] not arr[0:n-1] or arr[1:n]).
A test won't say why code is the way it is; it will simply provide some sample input and expected output to that code. This may be enough, but not always.
They are also a word of encouragement to whoever sees your code for the first time because he needs to make a change in it, five years from now.
"Look, this code looks strange to you, you would have done it differently, and you are afraid you might break things you don't even know exist. But fear not, there are tests, they run, and they will guide you."
It's always relieve to find that some random old crap is actually well tested.
Not just to someone else. Nothing is more pleasant than going in to refactor code you wrote years ago, and find that you wrote comprehensive unit tests. Thank you past-me!
> I am against writing unit tests in general, since experience shows that they actually prevent high-quality code from emerging from development
And he's lost me in his first sentence.
Unit testing, good unit testing at least, is not just about developing as it is about preventing regressions. A unit test that runs on every build ensure that something is true and that it stays true forever.
This. I agree TDD can cause more harm than good in software design, but my test suite saves me from shipping several regressions per day that I would have otherwise not noticed.
Obviously if you are against testing in general you are against TDD (it's just testing taken to the extreme).
I thought he at least was going to say that he preferred higher level testing, to low level unit tests (which can be a fair point, you can find 90% of the issues with 10% of the test code if you accept that you can't always isolate the failure based on the test report). But no. He is against testing.
You quickly get to the situation where the code is clean, concise modular and all those good words, but requires one more tweak/indirection only for the unit testing. the result is popularly called "Test induced design damage".
On the teams I've work on, pushing code that doesn't build and pass all unit tests is grounds for mocking. I have an alias in my zshrc file called "safepush" that runs
[clean command] && [build command] && [test command] && git push
If any piece doesn't succeed, the command stops. No push without unit tests passing.
And, as the article points out nicely, this basically prevents people from refactoring their code. In any nontrivial codebase, when you have a lot of tests, then what would have been 15 minutes of work rearranging class structure turns into a whole day of rearranging test cases to fit the architecture again.
It can be, I won't entirely disagree. But it depends how highly coupled your unit tests are to the design.
Unit tests are a safety mechanism- you can easily bog yourself down with too many, tying things up in bad ways, or presuming a specific design. But do you really want to live with no safety at all?
The art, as I see it, lies in writing tests in such a way that you get the guarantees you need, the safety of not letting the next guy (or yourself) screw it all up during later changes while simultaneously not preventing you from doing needed refactoring.
In any halfway decent build your tests will run whenever you build the project, to prevent tests from ever getting outdated.
If any test fails, it is because you broke it just now, so you go in and see if your test is broken, or, as is often the case, your refactored code is breaking the expected behaviour of the tested code.
Indeed. I was ready to hear a rant about TDD with an open mind since I have some issues with it myself, but the opening line completely ruined it. He might as well rally against refactoring because unit tests are essential part of that too.
> Unit testing ... is not just about developing as it is about preventing regressions.
Absolutely this! Unit testing helps you during the dark years of maintenance phase, when you must fix a bug or add a new feature and you don't have the knowledge, nor the time, to build a full representation of the application in your head: you just touch as little code as possible to get the work done, and unit tests help to ensure that all the rest stays equal.
I think I see part of his point, though. By its nature, a unit test is (often, albeit not necessarily) tightly coupled with the thing it is testing--which means that anyone who changes the implementation must change the test, which increases the complexity of code changes.
I know that in my own work, the single most important thing for me is to be able to massively refactor, restructure and redesign my and others' code, as I'm working with it (this is probably why I like dynamic languages and macros so much, and probably why I benefit from static typing so much); anything which gets in the way of deleting, reorganising, clarifying, duplicating, altering, reducing and shifting code is going to slow me down and make the resulting code worse.
Perhaps, though, there is a middle ground: while the internals of a library must be free to mutate, the external interface ought not to change nearly as much. Perhaps all unit tests should be written to test the external interface, not the internal details. That might also help avoid the tests-which-test-that-the-code-does-what-it-does disease.
I do really appreciate tests, and they do help detect and prevent certain types of regressions.
Regression tests are absolutely awesome, especially if written for nontrivial components which will later be modified or expanded.
Having worked with dynamic languages quite a bit, I found that iff your language has a decent REPL, so you can code iteratively, you don't need unit tests to drive your design. You perform those tests in REPL yourself, which helps you flesh out the design, but you don't have layers of code that'll make changing the design harder.
The way I personally work with Lisp is a mix of writing code in a file, compiling particular parts of it and playing around with it in REPL until I've figured out the right design - at which point I may as well start adding some tests around tricky places. The idea is to not burden yourself with permanent tests until you have your design figured out.
Which is why testing behaviour not implementation is so important. Admittedly this sounds easier than it is in practice. Personally I've found that in-memory coarse-grained tests more akin to integration tests, that test from the outside in are helpful and easier to maintain because they tend to be less concerned with the how. I can then choose to write finer-grained unit tests where appropraite
Agreed! A code base I recently inherited (and exorcised) contained unit tests with roughly 90% coverage... And not a single assert. They didn't understand the idea of testing behavior, they just copied the implementation as tests.
I reduced 20,000 lines of tests to 5,000, and caught a dozen bugs.
> all unit tests should be written to test the external interface, not the internal details
Yes, this is the most useful approach to unit testing. If you have a functional interface which doesn't cause side effects, (i.e. state changes of any kind, external actions that can't really be effectively verified by test code, etc) then unit testing is very valuable and useful given you just test the interface using the simplest possible approach: a table of some inputs which should map to some outputs.
I'd go further and say you should always write the tests only after this interface has solidified. Test first makes sense only under either the assumption that your first design is always the best design, or that you are happy to spend the time to morph the tests, as well as the code when you realise your first design wasn't the best and have to iterate and tweak.
We all know from experience that the first assumption is never true. As for the second assumption, it may be true, but then the question becomes, well, why waste the time? You know those 'test-first' tests will just be rewritten anyway. Why not just write the tests last and save yourself a lot of wasted effort?
I work somewhere without any unit tests at all, and he still lost me at that sentence.
If he thinks test driven development can produce horrible implementations then he's right, they can. But he's horribly wrong that the solution is to abandon unit testing.
Without unit tests developers begin to live in fear of parts of the code, and refuse to clean it up "in case it breaks something", and there's no way to verify those fears aren't founded.
Any refactoring job starts with annoying a lot of people by breaking things that previous worked and questions start to appear from people that "it used to work before" when some of those breakages slip through to the customer.
Refactoring becomes something to fear because "we'll have to retest everything" which is a large expense that cannot be afforded.
I'd be a lot less gung-ho about unit tests if all the tests I wrote always passed the first time I expected them to pass, and always stayed passing regardless of what changes people make to the code (including people who are not me). As neither of these things happens, I'm pretty gung-ho.
I love the first quote for how (ironically) true it is:
"Trying to improve software quality by increasing the amount of testing is like try[ing] to lose weight by weighing yourself more often."
From someone who has lost over 15kg in the past in few months, one of the things that helped most was starting weighing myself as I didn't do that previously. It kept reminding me that I wasn't still there and gave me more motivation
And it's actually been confirmed in a recent study:
"Frequent Self-Weighing and Visual Feedback for Weight Loss in Overweight Adults"
"The major finding of this study is that the use of frequent weighing accompanied by visual feedback of weight, without a prescribed diet or exercise plan, was effective in producing a small but sustainable weight loss in overweight males."
What is "more often" ? Weighing yourself daily is useless, since the weight fluctuates too much. Not to mention that "weight" is useless as well, while you're actually trying to loose FAT. Fat percentage and resulting lean body mass (body weight minus body fat) are the metrics to measure. But still a waste of time doing it daily, twice per week to gain insight on the delta and where it's trending is enough.
Weighing yourself daily is not useless. The idea of doing it weekly is being spread explicitly because GenPop doesn't understand the concepts of a moving average and a low-pass filter - instead, they freak out over those daily fluctuations. If you weigh yourself daily and remember to always look at the average of the last few samples, you get a more useful indicator of your current weight trend.
It's not useless, it taught me that weight fluctuates a lot by the hour and some by the week (;
I took the average of the 3-4 last days at the same time (about waking up) is a good enough approximation. Once I learned this, "more often" means once daily, as opposed to "once monthly" (if any). It wasn't only about the hard number, it was about the motivation.
Ha that is hilarious. I am actually trying to gain some weight and i find i weigh myself too often. I think i can could come up with the study that shows the opposite effect. After all it doesn't have to be true, just significant.
And yes, it may be stupid to test before a design emerges - but only if you start with a very fine-grained test. Usually when I'm coding from scratch I write a very, very, VERY coarse-grained test that "tests something", and when I reach the point of passing it (which may involved creating and designing multiple classes) I probably have a working design and I may begin creating other, smaller unit tests for individual components. The initial test may disappear or become an integration or acceptance test.
By the way there's little content in the article. It's just a rant. And the article about not writing test cases at all is simply ridiculous - since an error can exist in a test, we shouldn't write test cases?!?
>Usually when I'm coding from scratch I write a very, very, VERY coarse-grained test that "tests something", and when I reach the point of passing it (which may involved creating and designing multiple classes) I probably have a working design and I may begin creating other, smaller unit tests for individual components. The initial test may disappear or become an integration or acceptance test.
This reads pretty much how I work too.I don't think it is unusual. Start coarse, outside in. Write the finer grained 'unit' tests where appropriate. Focus on where you need the crutch (design and confidence) not on clambering to test all the things.
> Usually when I'm coding from scratch I write a very, very, VERY coarse-grained test that "tests something", and when I reach the point of passing it (which may involved creating and designing multiple classes) I probably have a working design and I may begin creating other, smaller unit tests for individual components.
That is exactly how I start coding something from scratch, except I don't have that initial test. I don't think such a vague starting test adds anything real.
TDD assumes you know what all the interfaces are going to be beforehand. If you discover that an interface needs to change you have to refactor all of your tests. Unit tests are momentum against change.
It also assumes a bug free app with 100% code coverage is the objective, regardless of the cost it takes to write all that test code. An app that has a few bugs but takes half the resources to develop can make better business sense.
> It also assumes a bug free app with 100% code coverage is the objective, regardless of the cost it takes to write all that test code.
I disagree on this. 100% coverage is actually a bad thing, in my opinion, because there's always a lot of code you do not need to test (simple constructors, accessors/mutators, etc).
A unit test should test things that you are relying on being correct. You write a unit test as a way to protect future developers (including yourself) from totally screwing things up by accidentally changing something important.
An application with a few bugs that took half the resources to develop is nice- until the customer needs a few extra changes 18 months later, and you turn 'a few' bugs into 'a whole lot of' bugs.
I get the point, but feel different. When a project becomes large enough, changes can result in breakage in the weirdest places. Proper testing can help discovering this breakage before shipping. For me, tests are a safety net for changes in the tested code.
Large projects without any tests are effectively unmaintainable, often times even for the original author, and most certainly for others.
I do agree with this, and have felt it from both ends. If you need to change a core class or interface which cascades throughout the system, those unit tests are sure nice to have. I'm not sure I'd be confident enough to change the core class without the unit tests. In that sense, lack of unit tests is momentum against change.
There is no such thing as bug-free app, and I do not agree for 100% code coverage (rather write test for what's important to you, i.e. prioritise). However, tests gives you a higher level of confidence when you perform refactoring.
PS. Regression is far worse than momentum against change, when your app is `broken` you can't deliver it.
Just wait until he has to write a system with several hundred web services that have been documented to perform in a very precise manner given particular data sets, and have hundreds of customers who have integrated to that API, and absolutely required that there be no variance in the output in the API, or there (extraordinarily expensive) system integration will fail, at great costs to their business systems.
I'm not saying TDD makes sense everywhere, but being able to confirm that your 3000+ continuous integration tests all passed green before shipping a new version to all of your customer is a huge way to avoid embarrassingly predictable bugs. (Leaving, of course, the opportunity to ship the embarrassingly non-obvious bugs)
And, honestly, it's not clear that he was opposed to TDD, as he was opposed to being constrained to having to write huge test frameworks during the exploratory phase of code development, in which you may want to have a bit more freedom to write/discard while you feel out requirements/solutions.
What you described having tests, not doing TDD. That is, it's important to have those test cases verifying that your APIs still work, but that doesn't mean you have to design the APIs by writing tests, as opposed to designing by thinking about what you actually want to achieve.
Fair point. I was more referring to his comment here: I am against writing unit tests in general, since experience shows that they actually prevent high-quality code from emerging from development--a better strategy is to focus on competence and good design.
Unit tests are useless for ensuring that a web service API works, they are too low level. You're supposed to be testing only the "units", aka classes or groups of methods.
I stand corrected twice. Probably just goes to show that a non-(current)developer should spend 30 minutes of research on a topic before commenting on a thread.
Way stronger than I'd write, and I don't agree with it entirely, but the author has a point.
Personally, what annoys me the most about TDD I've seen in the wild are two things: designing for tests instead of actual problems, and tests affecting the structure of "real" code.
Designing for tests - the standard TDD approach, first we write tests, then we write code to pass the tests. Quite often the consideration of the problem being solved disappears. It's a fine approach when your task is to write a small black box that takes some data in and transforms it into something else. But I've never seen a case where someone made it work for complex tasks. It always ends up the same - your tests become more complicated than the tested code. It happens with any non-trivial problem, because the thinking you have to do to write tests that make sense is the same as the thinking you need to do to solve the problem in the first place. So you're basically writing the program twice, only in a convoluted way, and without considerations for global design.
Tests affecting the structure - this is IMO a strong code smell. If you're modifying your design to accomodate for tests, by e.g. adding superfluous dependencies, hooks or injection points, you've screwed up. It only makes the code more complicated and less reliable.
The only tests I've found valuable so far (in terms of effect for effort spent) are regression tests - the ones you write to catch bugs in order to make sure they won't happen again. Everything else in TDD seems to be easily replaceable by proper iterative programming.
Maybe that's why TDD is popular in the languages without a sane REPL.
> Tests affecting the structure […] is […] a strong code smell
On the other hand, code being hard or impossible to test is often thought of as a code smell as well. Code that is easy to test is easier to understand — not in the least because there are tests demonstrating its use. Techniques such as dependency injection help a lot here.
The effect that I'm complaining here is visible in "weaker" languages, like Java. People are too afraid to use reflection to instrument the code for testing, so e.g. in a codebase I'm working on right now at $JOB I get to see classes in which you can't really tell what code is there for business purposes and what code was added so that the business code could be tested.
I agree with you that a code that's hard or impossible to test is a code smell. I argue that having to modify a good design to accomodate for more testing is also a code smell. I think the two heuristics narrow down the design space nicely, pointing one towards designs that are easy to test because of their natural boundaries, and not because of additional testing cruft being added.
Techniques like dependency injection can be really useful -- see Angular -- but too often I see a perfectly understandable piece of code expand into a mess of multiple constructors (only one ever called in production) and helper methods all so that strict unit testing can be done. The post's commit story was unsurprising. If TDD is being done this mangling often just happens upfront. DI is great in the same way interfaces are great, but if your code actually just cares about a particular implemention you instantiate wherever, or even better is built-in to the language, it's easiest to reason with that implementation. Taken to the extreme, you get Enterprise FizzBuzz.
> Personally, what annoys me the most about TDD I've seen in the wild are two things
It's like saying that one is annoyed by programming in general because he's seen too many horrible things done with it! Anything can be abused, TDD is not an exception.
> your tests become more complicated than the tested code
Well, then don't tdd that code on unit level. Keep some high-level(smoke, integration etc) tests that executes it , relax and write/design it without TDD the best way you can.
> Tests affecting the structure - this is IMO a strong code smell.
Yes, and this smell (by definition) shows you a flaw in the code design. TDD helped to identify this. Apparently)
Good observations! At that company, sure, TDD is ridiculous.
But other companies don't do it that way, and require TDD, so the article took a narrow, biased view without considering pros and cons and different scenarios.
His experience might be a lot different from mine, but I've found TDD enormously helpful when tackling legacy systems. Maybe that's not "test-driven development" and more like "test-driven refactoring", but working on complex legacy systems where the original developers are basically all gone is scary, and TDD has helped me make some sense of it and feel a lot better about making changes.
Does anyone have a good counter-point worth reading? I understand that it's more of a anecdotal rant, but it's not the first time I've read something like this about TDD.
I've tried TDD, but I've found that it just stifles my productivity too much.
I don't necessarily use TDD all the time, but I think it provides significant value. A key one is to help guide developers break down complexity.
This is quite apparent especially when I conduct pair programming interviews. Developers who were exposed to TDD (or to projects with significant test code) approach the problem in a far more structured manner, and their code is much more pleasant to look at :)
> Let me emphasize: you write the test cases for your program, and then you write your program. You are writing code to test something that doesn't even exist yet. I am not rightly able to apprehend the kind of confusion of ideas that could provoke such a method.
I don't understand this. What is so absurd about specifying facts about the program you will write? When we have tools that can prove facts, we will be doing formal specifications instead of random sampling. But still, testing is a way of statistically specifying facts about your program, for which it seems sensible to be written before the program.
Because like the author says, there is no substitute for competence. If you can't write good code, then your assumptions about the future design will be wrong and/or bad, and your tests will be as poorly written. In other words, TDD (anecdotally) doesn't improve your code but only introduces extra levels of complexity. Which in the hands of an incompetent developer becomes an even bigger problem than if there was no TDD.
If you can't write good code, then you will not write good code.
But, there is another common case where unit tests (written before or after) also come in tremendously handy: if you can't write good mistake-free code 100% of the time.
Which describes all programmers who have ever existed.
TDD is truly cult like in it's extreme. Usually it's managers and slightly weaker technical folk with a scrum certification that spew the absolutist TDD path.
I'm a contract java developer and I know how to 'play the game'. The hypocrisy in the amount of fizbuz clones I've written in a TDD fashion on interviews, to then see the production code has no little to no unit tests.
I once got negative feedback on an interview because I wasn't TDD enough after expressing the opinion that TDD works great for most use cases but there are limits, e.g. positioning of a front end element isn't always best done TDD. The guy interviewing was non technical and of course any dissent to TDD meant I was a bad fit.
Anyways - end rant. Like I said I just learnt to play the game.
If you're doing TDD (or software development) like this, you're doing it wrong. Yes, yes, I know. No True Scotsman [1]. But when many - I mean LOTS OF - people say "I get benefits form [Technique]", you can't just say: "It cannot work. I tried it, it sucked.".
I mean, you can say that. But doing so makes you look... ignorant - at best. You know, there is a possibility that you just got it wrong.
So, many great programmers say that they get benefits from TDD. They get benefits. Not the suits. Not their co-workers. They.
TDD is hard. I had to learn it and practice it. And I'm still learning and practicing it, even though I'm now also teaching it to others and helping teams implement it. But I get benefits from it. Writing tests helps me to think about problems and to actually improve my designs [2]. And writing tests helps me to know when to stop - To not gold-plate my designs.
Sorry, but this "article" is just an angry rant, with no real arguments. Please, don't give up on TDD early because of rants like this. If you have any questions about TDD or need help getting started, feel free to ask (here or in private - you'll find my email address in my profile)...
My point is: If TDD leads you to bad design you're doing it wrong. You are probably not listening to your tests and you are probably not taking care to refactor towards a better design. Maybe you are even writing bad unit tests [1].
BTW, if any technique in software development leads you to bad design, and you don't stop and try to improve something, you're doing software development wrong. If a technique does not help you, you have two possibilities: Trying to do it better (maybe with outside help), or trying something different.
>- You're trying to make a design before you learn anything about it.
>- People write something one way, but then are afraid to change it because they'll have to rewrite the testing code that goes along with it.
Both of these suggest tests of the wrong granularity; that people are, as he says, writing silly little fencepost error checks for every single function.
> "Trying to improve software quality by increasing the amount of testing is like try[ing] to lose weight by weighing yourself more often."
Giving the benefit of the doubt that this quote is in fact an accurate metaphor – to begin with this quote is to begin with the assumption that TDD's single motivation is improving software quality. TDD is a fantastic practice for ensuring conformity to a specification; which one might describe as software's correctness. A more objective metric than "quality".
It's not really a rant, it is a very strongly communicated opinion. And I agree with a lot of it.
I don't mind writing a few test cases during or after I am done with my work. This is mostly to protect someone (including me) to change my perfect design after I have found it, though ;-)
> How often have you seen a program crash? If it was developed by a large software company, chances are it was written using TDD. Clearly, TDD is not a magic bullet. So, TDD does not "prove your code works".
> Developing software is like a painting commission
Several reasons I gave up a third of the way in. I rarely write tests first, but I can appreciate that it works for plenty of people - my brain just doesn't work that way.
It's hard to take anything in this article seriously because of the nerdrage and the dismissal of anything he disagrees with as "stupid".
153 comments
[ 2.8 ms ] story [ 252 ms ] threadThis is just a rant. I was hoping for an actual study, but the title should have given it away.
Instead there were lots of parallels that I don't recognise.
It was disappointing the author didn't seem to try and find the advantages in TDD and discuss a counter argument or alternative, such as modifying a foreign code base.
Because it's easy to measure whether you're exercising every code path, software engineers think in terms of how close they are to "100% coverage" of tests. What you really should be thinking about is whether you're exercising every possible input, not every possible code path.
https://twitter.com/brlewis/status/553191394639360000
1. Well-tested parts
2. Decoupled design
3. Rapid feedback
4. Local context to test in.
At the same time, there are many cases where they don't help much. More here: http://henrikwarne.com/2014/09/04/a-response-to-why-most-uni...
"Look, this code looks strange to you, you would have done it differently, and you are afraid you might break things you don't even know exist. But fear not, there are tests, they run, and they will guide you."
It's always relieve to find that some random old crap is actually well tested.
And he's lost me in his first sentence.
Unit testing, good unit testing at least, is not just about developing as it is about preventing regressions. A unit test that runs on every build ensure that something is true and that it stays true forever.
Obviously if you are against testing in general you are against TDD (it's just testing taken to the extreme).
I thought he at least was going to say that he preferred higher level testing, to low level unit tests (which can be a fair point, you can find 90% of the issues with 10% of the test code if you accept that you can't always isolate the failure based on the test report). But no. He is against testing.
If you're developing something which must work 100% every time or else someone will die, then unit tests are a must.
If you're developing with people who break the build all the time, then unit tests are useful.
Unit tests that aren't 100% up to date with the code are worse than no unit tests.
A bit of a tautology... broken code is broken. Unmaintained code, even unit tests, should be deleted.
If any piece doesn't succeed, the command stops. No push without unit tests passing.
Unit tests are a safety mechanism- you can easily bog yourself down with too many, tying things up in bad ways, or presuming a specific design. But do you really want to live with no safety at all?
The art, as I see it, lies in writing tests in such a way that you get the guarantees you need, the safety of not letting the next guy (or yourself) screw it all up during later changes while simultaneously not preventing you from doing needed refactoring.
It's a fun challenge.
If any test fails, it is because you broke it just now, so you go in and see if your test is broken, or, as is often the case, your refactored code is breaking the expected behaviour of the tested code.
Absolutely this! Unit testing helps you during the dark years of maintenance phase, when you must fix a bug or add a new feature and you don't have the knowledge, nor the time, to build a full representation of the application in your head: you just touch as little code as possible to get the work done, and unit tests help to ensure that all the rest stays equal.
I know that in my own work, the single most important thing for me is to be able to massively refactor, restructure and redesign my and others' code, as I'm working with it (this is probably why I like dynamic languages and macros so much, and probably why I benefit from static typing so much); anything which gets in the way of deleting, reorganising, clarifying, duplicating, altering, reducing and shifting code is going to slow me down and make the resulting code worse.
Perhaps, though, there is a middle ground: while the internals of a library must be free to mutate, the external interface ought not to change nearly as much. Perhaps all unit tests should be written to test the external interface, not the internal details. That might also help avoid the tests-which-test-that-the-code-does-what-it-does disease.
I do really appreciate tests, and they do help detect and prevent certain types of regressions.
Having worked with dynamic languages quite a bit, I found that iff your language has a decent REPL, so you can code iteratively, you don't need unit tests to drive your design. You perform those tests in REPL yourself, which helps you flesh out the design, but you don't have layers of code that'll make changing the design harder.
The way I personally work with Lisp is a mix of writing code in a file, compiling particular parts of it and playing around with it in REPL until I've figured out the right design - at which point I may as well start adding some tests around tricky places. The idea is to not burden yourself with permanent tests until you have your design figured out.
I reduced 20,000 lines of tests to 5,000, and caught a dozen bugs.
Yes, this is the most useful approach to unit testing. If you have a functional interface which doesn't cause side effects, (i.e. state changes of any kind, external actions that can't really be effectively verified by test code, etc) then unit testing is very valuable and useful given you just test the interface using the simplest possible approach: a table of some inputs which should map to some outputs.
I'd go further and say you should always write the tests only after this interface has solidified. Test first makes sense only under either the assumption that your first design is always the best design, or that you are happy to spend the time to morph the tests, as well as the code when you realise your first design wasn't the best and have to iterate and tweak.
We all know from experience that the first assumption is never true. As for the second assumption, it may be true, but then the question becomes, well, why waste the time? You know those 'test-first' tests will just be rewritten anyway. Why not just write the tests last and save yourself a lot of wasted effort?
If he thinks test driven development can produce horrible implementations then he's right, they can. But he's horribly wrong that the solution is to abandon unit testing.
Without unit tests developers begin to live in fear of parts of the code, and refuse to clean it up "in case it breaks something", and there's no way to verify those fears aren't founded.
Any refactoring job starts with annoying a lot of people by breaking things that previous worked and questions start to appear from people that "it used to work before" when some of those breakages slip through to the customer.
Refactoring becomes something to fear because "we'll have to retest everything" which is a large expense that cannot be afforded.
"Trying to improve software quality by increasing the amount of testing is like try[ing] to lose weight by weighing yourself more often."
From someone who has lost over 15kg in the past in few months, one of the things that helped most was starting weighing myself as I didn't do that previously. It kept reminding me that I wasn't still there and gave me more motivation
"Frequent Self-Weighing and Visual Feedback for Weight Loss in Overweight Adults"
"The major finding of this study is that the use of frequent weighing accompanied by visual feedback of weight, without a prescribed diet or exercise plan, was effective in producing a small but sustainable weight loss in overweight males."
http://www.hindawi.com/journals/jobe/2015/763680/
I took the average of the 3-4 last days at the same time (about waking up) is a good enough approximation. Once I learned this, "more often" means once daily, as opposed to "once monthly" (if any). It wasn't only about the hard number, it was about the motivation.
http://www.hindawi.com/journals/jobe/2015/763680/
http://david.heinemeierhansson.com/2014/tdd-is-dead-long-liv...
And yes, it may be stupid to test before a design emerges - but only if you start with a very fine-grained test. Usually when I'm coding from scratch I write a very, very, VERY coarse-grained test that "tests something", and when I reach the point of passing it (which may involved creating and designing multiple classes) I probably have a working design and I may begin creating other, smaller unit tests for individual components. The initial test may disappear or become an integration or acceptance test.
By the way there's little content in the article. It's just a rant. And the article about not writing test cases at all is simply ridiculous - since an error can exist in a test, we shouldn't write test cases?!?
More generally, it's about not 'coding too far into the future', even (especially) with tests.
But I like to code a little ways into the future.
'README driven development' is also interesting.
This reads pretty much how I work too.I don't think it is unusual. Start coarse, outside in. Write the finer grained 'unit' tests where appropriate. Focus on where you need the crutch (design and confidence) not on clambering to test all the things.
That is exactly how I start coding something from scratch, except I don't have that initial test. I don't think such a vague starting test adds anything real.
It also assumes a bug free app with 100% code coverage is the objective, regardless of the cost it takes to write all that test code. An app that has a few bugs but takes half the resources to develop can make better business sense.
I disagree on this. 100% coverage is actually a bad thing, in my opinion, because there's always a lot of code you do not need to test (simple constructors, accessors/mutators, etc).
A unit test should test things that you are relying on being correct. You write a unit test as a way to protect future developers (including yourself) from totally screwing things up by accidentally changing something important.
An application with a few bugs that took half the resources to develop is nice- until the customer needs a few extra changes 18 months later, and you turn 'a few' bugs into 'a whole lot of' bugs.
I get the point, but feel different. When a project becomes large enough, changes can result in breakage in the weirdest places. Proper testing can help discovering this breakage before shipping. For me, tests are a safety net for changes in the tested code.
Large projects without any tests are effectively unmaintainable, often times even for the original author, and most certainly for others.
PS. Regression is far worse than momentum against change, when your app is `broken` you can't deliver it.
I'm not saying TDD makes sense everywhere, but being able to confirm that your 3000+ continuous integration tests all passed green before shipping a new version to all of your customer is a huge way to avoid embarrassingly predictable bugs. (Leaving, of course, the opportunity to ship the embarrassingly non-obvious bugs)
And, honestly, it's not clear that he was opposed to TDD, as he was opposed to being constrained to having to write huge test frameworks during the exploratory phase of code development, in which you may want to have a bit more freedom to write/discard while you feel out requirements/solutions.
Personally, what annoys me the most about TDD I've seen in the wild are two things: designing for tests instead of actual problems, and tests affecting the structure of "real" code.
Designing for tests - the standard TDD approach, first we write tests, then we write code to pass the tests. Quite often the consideration of the problem being solved disappears. It's a fine approach when your task is to write a small black box that takes some data in and transforms it into something else. But I've never seen a case where someone made it work for complex tasks. It always ends up the same - your tests become more complicated than the tested code. It happens with any non-trivial problem, because the thinking you have to do to write tests that make sense is the same as the thinking you need to do to solve the problem in the first place. So you're basically writing the program twice, only in a convoluted way, and without considerations for global design.
Tests affecting the structure - this is IMO a strong code smell. If you're modifying your design to accomodate for tests, by e.g. adding superfluous dependencies, hooks or injection points, you've screwed up. It only makes the code more complicated and less reliable.
The only tests I've found valuable so far (in terms of effect for effort spent) are regression tests - the ones you write to catch bugs in order to make sure they won't happen again. Everything else in TDD seems to be easily replaceable by proper iterative programming.
Maybe that's why TDD is popular in the languages without a sane REPL.
On the other hand, code being hard or impossible to test is often thought of as a code smell as well. Code that is easy to test is easier to understand — not in the least because there are tests demonstrating its use. Techniques such as dependency injection help a lot here.
I agree with you that a code that's hard or impossible to test is a code smell. I argue that having to modify a good design to accomodate for more testing is also a code smell. I think the two heuristics narrow down the design space nicely, pointing one towards designs that are easy to test because of their natural boundaries, and not because of additional testing cruft being added.
It's like saying that one is annoyed by programming in general because he's seen too many horrible things done with it! Anything can be abused, TDD is not an exception.
> your tests become more complicated than the tested code
Well, then don't tdd that code on unit level. Keep some high-level(smoke, integration etc) tests that executes it , relax and write/design it without TDD the best way you can.
> Tests affecting the structure - this is IMO a strong code smell.
Yes, and this smell (by definition) shows you a flaw in the code design. TDD helped to identify this. Apparently)
But other companies don't do it that way, and require TDD, so the article took a narrow, biased view without considering pros and cons and different scenarios.
I've tried TDD, but I've found that it just stifles my productivity too much.
This is quite apparent especially when I conduct pair programming interviews. Developers who were exposed to TDD (or to projects with significant test code) approach the problem in a far more structured manner, and their code is much more pleasant to look at :)
http://www.drdobbs.com/tdd-is-about-design-not-testing/22921...
I don't understand this. What is so absurd about specifying facts about the program you will write? When we have tools that can prove facts, we will be doing formal specifications instead of random sampling. But still, testing is a way of statistically specifying facts about your program, for which it seems sensible to be written before the program.
But, there is another common case where unit tests (written before or after) also come in tremendously handy: if you can't write good mistake-free code 100% of the time.
Which describes all programmers who have ever existed.
Not really absurd, just impractical. It's because all those facts are wrong, and you don't know yet why.
I'm a contract java developer and I know how to 'play the game'. The hypocrisy in the amount of fizbuz clones I've written in a TDD fashion on interviews, to then see the production code has no little to no unit tests.
I once got negative feedback on an interview because I wasn't TDD enough after expressing the opinion that TDD works great for most use cases but there are limits, e.g. positioning of a front end element isn't always best done TDD. The guy interviewing was non technical and of course any dissent to TDD meant I was a bad fit.
Anyways - end rant. Like I said I just learnt to play the game.
I mean, you can say that. But doing so makes you look... ignorant - at best. You know, there is a possibility that you just got it wrong.
So, many great programmers say that they get benefits from TDD. They get benefits. Not the suits. Not their co-workers. They.
TDD is hard. I had to learn it and practice it. And I'm still learning and practicing it, even though I'm now also teaching it to others and helping teams implement it. But I get benefits from it. Writing tests helps me to think about problems and to actually improve my designs [2]. And writing tests helps me to know when to stop - To not gold-plate my designs.
Sorry, but this "article" is just an angry rant, with no real arguments. Please, don't give up on TDD early because of rants like this. If you have any questions about TDD or need help getting started, feel free to ask (here or in private - you'll find my email address in my profile)...
[1] http://davidtanzer.net/no_true_scotsman_in_agile
[2] But you have to refactor ruthlessly. And learn and practice refactoring, which is hard.
What do you think is the right way to do it, then?
BTW, if any technique in software development leads you to bad design, and you don't stop and try to improve something, you're doing software development wrong. If a technique does not help you, you have two possibilities: Trying to do it better (maybe with outside help), or trying something different.
[1] http://www.makinggoodsoftware.com/2012/01/27/the-evil-unit-t...
Ultimately, we're too young a field to be able to replace common sense with a process.
http://www.drdobbs.com/tdd-is-about-design-not-testing/22921...
>- You're trying to make a design before you learn anything about it.
>- People write something one way, but then are afraid to change it because they'll have to rewrite the testing code that goes along with it.
Both of these suggest tests of the wrong granularity; that people are, as he says, writing silly little fencepost error checks for every single function.
Giving the benefit of the doubt that this quote is in fact an accurate metaphor – to begin with this quote is to begin with the assumption that TDD's single motivation is improving software quality. TDD is a fantastic practice for ensuring conformity to a specification; which one might describe as software's correctness. A more objective metric than "quality".
I don't mind writing a few test cases during or after I am done with my work. This is mostly to protect someone (including me) to change my perfect design after I have found it, though ;-)
> How often have you seen a program crash? If it was developed by a large software company, chances are it was written using TDD. Clearly, TDD is not a magic bullet. So, TDD does not "prove your code works".
> Developing software is like a painting commission
Several reasons I gave up a third of the way in. I rarely write tests first, but I can appreciate that it works for plenty of people - my brain just doesn't work that way.
It's hard to take anything in this article seriously because of the nerdrage and the dismissal of anything he disagrees with as "stupid".