DHH's keynote from the first day of RailsConf, in which he talks a bit about testing and the general perception of writing software as "science", is online here:
Perhaps design by contract is the way forwards? Explicit runtime validation of the flow of data through code at each step in the system.
As a frontend developer, I have been moving towards a greater emphasis on functional & integration testing. We're also using scripter here, a test tool that actually compares screen caps with the actual on-screen state of our applications. So a purely visual approach.
There's a wonderful world of testing tools out there. I can imagine it's incredibly daunting to a newcomer trying to decide which to pick up and run with, especially with so many job specs specifically asking for "TDD", and now with DHH saying this.
Job specs ask for a lot of things but in my experience it's often as much about what the company aspires to rather than what they're really doing. More often than not that will be dabbling more than fanaticism.
I fully agree that TDD has turned into a cargo cult obsession for many developer shops. The benefits of unit tests are usually for the developer, but in over abundance can lead to inflexibility and debt when your system needs to change.
TDD has turned into a way for devs to 1-up other devs. When testing becomes a discipline of its own with the knowledge of these testing frameworks, something has gone wrong. Tests shouldn't need much more than assert statements, and maybe a little extra.
Part of the problem is developers are often too hesitant to delete tests. Tests, like any code, should be deleted when their value is lower than their cost. Tests have maintenance cost like any other code. Plus they add to the time it takes to produce a build.
When making significant changes to implementation of something it's often worth just leaving acceptance tests in place for regression and deleting unit tests and TDDing your new implementation
> in over abundance can lead to inflexibility and debt when your system needs to change.
I hear this argument a lot, and I think it is absolutely incorrect. If tests are hindering your system then you're just doing it wrong. It's like saying "climbing rope is a hindrance for the rock climber, since I have tied gordian knots around my feet and now cannot move."
TLDR doing something badly doesn't mean the thing you have botched is itself bad.
Yes, this is really what I mean by "over abundance". I'm suggesting that excessive application of unit test "fundamentalism" can lead to a variety of problems. A particular manifestation of what you describe as "doing something badly".
This post is amazing. I can't stand the TDD zealots who I've worked with in the past. They act as if they aren't even programmers, they are test-framework gurus first and foremost. Product is #1 in my book. Tests are needed but these guys would cowboy around taking weeks to work on a task because they'd use the excuse 'gotta write tests...' This post is a breath of fresh air.
All this talk of gurus, senseis, ninjas, zealots, rockstars, and superstars and shit has to stop. It's seriously making me annoyed I work in this industry.
I agree with the sentiment but, despite the proviso that we should not go anti-TDD, I still think it's too strongly worded.
If you are rapidly prototyping something then writing tests first is a hinderance. But if you have well defined requirements then writing tests first will save you time.
What's best will differ depending on whether you're designing a web app or building an interface to an SAP system or an external gateway. (My guess is that DHH does very little of the well defined interface building type stuff)
I fully agree with the sentiment. I've always been against TDD being pushed. Obviously as developers we love the stress-free appeal of unlimited time, test first, achieve 100% confidence in code. Wow so glamour.
In practice this will always never work, since we are time and budget constrained. Halfway we find that we are dumping tests in favor of writing even more hacky-tacky code just to meet a deadline.
This code will never be refactored, because the client is satisfied with the results, and does not appreciate all the edge-cases because you said you would take care of them.
No, I'd rather produce code that is well written, can be deployed, and taken over by other devs if needed, not several levels of testing paradigm's that need to be satisfied before code can be migrated.
Yes I agree that there is obviously a place for tests, and IMHO that's when certain business logic is considered implemented, and tests will need to catch all the edge-cases to make sure it will deliver in the future and not break with modification.
>Test-first units leads to an overly complex web of intermediary objects and indirection in order to avoid doing anything that's "slow". Like hitting the database. Or file IO. Or going through the browser to test the whole system. It's given birth to some truly horrendous monstrosities of architecture.
Oh yes - I agree completely with this.
I'm not sure I agree with testing things with Capybara though. How are you sure your backend controller actions are doing what they're supposed to be doing? Sure your browser might render what you need but what if a model attribute is set to false instead of true? How do you account for that using a frontend test suite?
dhh has spent so much of his career actively leading Rails that it's tough to tell the difference between his personal evolution as a developer and general trends in the industry. I don't think he even attempts to make a distinction in this essay.
I worry about a push away from unit tests. I can't imagine having to refactor a large application with only system-level integration tests to work with, especially in a dynamically-typed language like Ruby. I haven't written a Rails app in a long time though, maybe these days it's easier to just start over than to refactor.
But any bad day for dogma is a good day for the rest of us. Write software using whatever methodology works for you and always question the value of so-called "best-practices".
It's not so much about moving away from unit tests are more to do with picking the best points at which to test your apps. For web apps the natural points at are the model level and at the client level. For the Django apps I write these tend to be the most stable areas since they are likely to be heavily influenced by user requirements. The result is tests that have a long lifetime and provide better system documentation since they generally exercise the business logic of an app. That neatly avoids one of the major problems I have found with TDD that unless you know in advance exactly what you are doing and going to do you will end up writing a lot of tests that get junked.
"I have found with TDD that unless you know in advance exactly what you are doing and going to do you will end up writing a lot of tests that get junked".
If you know all that up front, you might as well go with waterfall methodology.
TDD is but one example of the apparently inevitable fate of good ideas in software development: it becomes the One True Way, universally applicable, and the new litmus test for distinguishing Real Developers from troglodytes. Whenever a developer gets a new tool, she is supposed to throw out the old one (there will never, of course, be more than one in the toolbox at any given time.)
I dislike mocks. I've never seen the point in testing code against an entirely fictional representation of the most complicated and slow part of the system, just because it happens to be more convenient. Of course it's more convenient. The only compelling reason I can see for mocks is when you've got code that hits external live-APIs that don't give you any real option for automated testing (E.G., reading from and posting to the Twitter API).
If an app is worth writing, and worth writing tests for, do it justice and test the whole shooting match. Yeah it's hard, but that just makes it all the more worth doing. Automate your tests so that they cover everything that is important, from DOM elements on a dynamically built webpage to your model (and therefore the data that gets written to your SQL database), but don't pretend that stubs are substitutes for this. If your model relies on a database, let it rely on the database during the tests too, otherwise what exactly are you testing?
If you are testing a single function, why would you care whether the data comes from the database, the network or from a mock ? All you need is to verify that the given function produces the expected output given the right (and wrong) inputs. Being able to test this way also makes easy to keep the components in your software decoupled.
My philosophy is to use mocks for unit tests and the real thing for integration tests.
Well speed of testing is not the only consideration. Mocks are also (primarily?) intended to isolate your testing so you're not looking through an entire stack to find a bug. Often, especially in early phases of software they're over kill, and distract from implementation time however.
Mocks (or I guess fakes, really) can be useful for simulating certain scenarios that are difficult to produce in real life but your code needs to handle. For example if in an integration test you want to see how your application deals with high latency or errors in a service dependency, you can write a mock of that service and have it introduce arbitrary delays or errors.
Of course it helps to test against the real service as well, since it's often prohibitively time-consuming or impossible to model ever aspect of it, but I wouldn't discount mocks so easily.
That's the money link. People who save time by using mocks instead of stubs are deferring design problems to later stages, which is really bad. When I'm only responsible for my module and don't give a shit about the system as a whole, then mocking is fine. But in this case, the development process is toxic.
Mocks to me represent a different approach to testing - think London School of Testing vs Classic TDD. With Classic TDD, it can be difficult to test side effects in your code since its more focused on input and output.
I use mocks heavily and I also test the database heavily. I use mocks in unit tests which is part of our automated build process. We also have fixtures and actual db level testing in our second level testing. Why not test both? unit testing will often catch a whole class of errors that integration tests will not or at least not accurately catch. I agree that the TDD dogma is silly but mocks and unit testing have serious value that you seem to be discounting.
I dislike mocks too; it introduces a lot of duplication that I think is deeply problematic. Automated tests are for me a way to support changing the system easily and safely; lots of mocks often work against that.
But what you're getting is speed. Tests are at their most valuable when they give you quick feedback. As test suite times creep up, people stop running them as much, or at all. I've been on projects where all our tests run in seconds, and it is an enormously different development experience than when they run in minutes.
It seems to me that if you are writing a lot of mocks, you are probably retrofitting tests onto an existing code base. If your code is written to be testable you usually can avoid mocks.
That being said, if you are writing a test on an algorithm and the data comes from a database this means you are not just writing a test on the algorithm - you are writing a test on the database with all the cruft that comes along with that. This test can be hundreds of times slower due to the DB dependency.
So mocks do have their place. The test pyramid is the answer - you need units (sometimes with mocks, integration, system).
In my experience most companies are way too integration heavy and very light on the unit side of things. For example for my current project our dev tests take 12 hours to run. This is the result of going too far away from the unit level and being too integration heavy.
There is a balance! Sounds like the rails world the balance is too unit heavy. In the enterprise world I think it's too integration/system heavy.
> I dislike mocks. I've never seen the point in testing code against an entirely fictional representation of the most complicated and slow part of the system, just because it happens to be more convenient.
You aren't testing code against an entirely fictional representation of the system, you are testing one piece of code in isolation, and using a fictional representation of another piece of the system to eliminate variables.
Unit testing is about verifying that individual components do their job correctly -- when doing it, you literally don't care what other pieces of code would do, because that's out of scope of the unit test.
Testing that the whole system works together is system/integration testing, and is a different thing. You don't use mocks for system testing, only unit testing.
> If an app is worth writing, and worth writing tests for, do it justice and test the whole shooting match.
Yes, that's what system/integration testing is for.
That doesn't mean you shouldn't have code that is amenable to proper unit testing.
TDD on the unit level does seem to lead to high granularity, and complexity in the system as a whole in favor of simplicity in test construction. However, some of the problems described sound like they are coming from a premature optimization mindset rather than TDD itself.
Once I learned about Cucumber and the idea of stating a requirement/test that you can't even parse yet, much less have a test for, much less have the code for, I started liking the idea of TDD as a wish list pyramid.
This allows you to think in terms of the big picture requirements, and then drill down as required to fulfill those requirements. Because you've spec/documented your design on the way down to the unit tests you can always step up a few levels and reconsider, and rewrite architectural "tests".
For example my first test is "I have a software tool for editing photos". Now I implement this test by checking if there is an executable in a path. Fail. Now I make a hello world exe for that path. Pass. Now I write a new test: "It opens an OpenGL window.", and later "it uses a mvc pattern", "the edits are represented as a scene graph", etc. all the way down to specific logic.
You later realize that a scene graph is not the right way to model your process, so you change that test to a different requirement, the altered requirements now redirect the TDD flow of an entire section of the application instead of just unit by unit.
This is a stunning opinion considering the fact that there are many Rails shops that can't turn around a build in less than an hour because you can't test models independently of the database.
Yeah. The hand-wavey reference to advances in parallelisation and cloud runner infrastructure doesn't adequately address the drawbacks of this "post-TDD" approach.
I think that is part of it. When I tried to TDD existing PHP code, it was pretty awful and I hated testing.
At the same time, I think when you design away as much complexity as possible at the product requirements level, as 37Signals proudly does with their products, it is hard to appreciate the complexity inherent to the requirements in other codebases.
I've seen this with other developers coming into large codebases with lots of complexity and scale requirements wanting to use a simple/naive ORM solution with no caching or worry about query speed/quantity. That solution falls over and they quickly learn that the requirements are more complex than a simple app with a few dozen users.
Basecamp is obviously at a high level of traffic and scaling complexity, but they still work to reduce requirement complexity which isn't always an option if you aren't the product owner.
He did mention fixtures, which speed things up dramatically, so I don't think he shares the same experiences with the Rails shops you are talking about (Who I assume do hit the database since the industry is in favor of factories). Of course fixtures vs factories is a different topic, but you are wrong in your analogy.
Not that I agree with him, but if your project can get away with a local sqlite database for testing (that is, no stored procs or database specific queries) and you make sure you don't create when build will suffice, you can have models tested with a database and still be reasonably fast.
> Over the years, the test-first rhetoric got louder and angrier, though. More mean-spirited.
Where are those mean TDD zealots? Can you point me at blog posts or mailing list messages displaying such behavior? I've never seen it personally. On the other hand, now and again a blog post like this comes up that's full of disdain towards the practice of testing first. I do TDD because it helps me get my work done. I'm happy to help others write tests if they wish, but I'd never look down on another developer because he or she doesn't use this tool. Some people like drawing diagrams, some people like to use an IDE... Do what works for you.
If you were developing rails apps in the mid-late 2000s then you'd have known too many to count.
There was a period in time between 2005-2008 in Boston where if you didn't practice TDD you were persona non-grata.
I think interestingly enough, a good number of those people now have 10+ years experience programming and now understand what those of us who had 10 years experience at that point in time were trying to tell them.
The irony is that that attitude is something you see from rails people about anything that's currently fashionable in the rails world. And I think they picked it up from DHH himself. You didn't get the same kind of TDD zealotry among e.g. Django people.
Where are those mean TDD zealots? Can you point me at blog posts or mailing list messages displaying such behavior?
"My thesis is that it has become infeasible, in light of what's happened over the last 6 years, for a software developer to consider himself 'professional' if he does not practice test driven development."
I think a problem is that we tend to overgeneralise our own experience. When we try out ideas, such as TDD, we (hopefully) get a good feel for how they work (or don't work) for us. Reading articles or getting advice might help us to understand and use those ideas better, but the impression is still based on your personal experience. That's fine when you're talking about how you like to work, but I think it comes unstuck when you start trying to apply it to other people. Even if you read other people's perspectives, it's often tempting to consider those that disagree to be idiots (although you might not be so blunt), and those that agree with you to be sensible.
I think it's more productive to try out a technique, try out some variations based on others' advice, and then share the experience: I found this technique useful in these situations, but not in these. Here are some tips that I found effective, and here's what didn't work for me. Then, people can try out techniques for themselves, and learn from your experience.
If we want to generalise what's effective and what's not for software development overall, then I think we need a much rigorous approach then getting a vibe from the community. I've only ever seen a handful of studies that try to be unbiased and somewhat scientific in assessing techniques like TDD.
You also can't be human or probably any animal for that matter. Pattern matching and generalization is how we survive and function in a world that is bigger than us.
I've been feeling this way about testing for a while now and I'm glad that DHH wrote about it so I don't have to feel "wrong" or "embarrassed" by it anymore.
I personally have gone a bit further and only write tests after a successful incarnation of a project (that means customers) that I want to keep developing, or if writing and running the test itself costs me less time then F5'ing a page or mucking around in the rails console.
I have noticed that it's much easier to throw away code that I haven't invested a lot of time in and usually the second time I write it (with tests), I have a much clearer idea of how to implement and test something.
People like talking about the reality of implementing design patterns but leave out huge variables, like what is being built.
I think his article highlights something else about FAD design patterns, hype and pressures. The amount of GUI developers I met running around yelling TDD was amusing, and refreshing meeting people who do what is right instead of implementing hype-patterns... new word? :¬)
Interesting. I agree with everything you wrote until the last sentence. I think he made some great arguments against TDD, but none against unit testing.
Really? What did you think was an argument against TDD?
> Test-first units leads to an overly complex web of intermediary objects and indirection in order to avoid doing anything that's "slow".
This is an argument against unit testing (and is a good example of the conflation). Personally I do TDD with slow, system tests, that mainly are testing business requirements.
He makes a couple of points that some TDD people are rabid, but that is just an argument that some people are jerks. It's not an argument against TDD. (And exactly the same is true of Unit Testing).
I think a big part of the TDD phenomenon happened when the "big name" rails consulting shops insisted on using it.
In consulting, I think this makes all the sense in the world.
a) When you hand off code, you can demonstrably prove what you handed off works.
b) You can take the business objectives of whoever you are consulting for and translate those into high level tests, which helps you stay focused.
c) You get to charge double what you'd normally have charged the client.
When running a in-house team at a startup none of those bullet points translate, and while TDD does offer other benefits, really at the early stages of the game every line of code you write needs to solve the problem you're trying to validate.
I live out of consulting, and my biggest projects is writing experimental consumer behavior simulation engines.
Is not that TDD does not have a value, it is just impractical
For once, since they are simulations, they are trying to predict the future, hence there is no right answer, there is the "Acceptable" answer, or the "Defensible" answer; so having a little test on the side that tells me that I am still producing the predicted result does not help much because there is no such thing as the correct predicted result
Sure I can do test to make sure I match the excel spreadsheet that I receive for spec, they would tell me that I copy the formulas correctly; but then, for seconds, I bring the results back and they test it with more real data, and they realize that we need to tweak 100s of different little formulas (which invalidate all the unit tests), change the order of a few of them, and introduced a couple of new ones (as I was told once "because I don't know what I was smoking that Tuesday, so we should use this new one"), and of course "can we have these changes for the end of the day?"
So it all unit tests are voided, make all those changes which are simple enough to be done in a couple of hours. And I am not going to know what is the correct result because there is no valid spreadsheet now, so I don't know in advance what is expected, hence I cannot write the test first
And wait until I try to tell them, the modifications will take 2 hours, but I need another 2 days to write the new set of unit tests, which since I wrote with the result of the modifications instead of before the modifications, they will not prove anything
Rinse and repeat 10s times a month for the last 4 years...
where is this Rails train going? it used to be cool, but it looks like it's heading towards the chasm now.
posts like this just encourage my opinion about the Merb merge with Rails being a disaster which helped Rails transform from "PHP in Ruby" to something beautiful to work with, while leading to self-destruction of Merb.
Rails gained a lot of traction and is now abandoning any good decision Yehuda and other great developers made.
this post simply sums up to: "I don't know how to test-first Rails applications, and I've invented Rails. we're abandoning test-first." admitting you have a problem is a first step in fixing it, but this is not going to fix it in the long run.
I've felt the same way for a long time too. The Merb merge had more to do with soothing large egos and indulging some architecture astronauts than it did about making it a better framework for developers to work with. Now it feels a lot like the over-engineered Java frameworks it was designed to liberate us from.
Yes let's move towards more coarse grained tests so we can lets the fine grained subtle bugs slip through. You know stuff like heartbleed. Theres a good idea.
To be fair a lot of unit test regimes wouldn't have caught something like heartbleed. Specifically cause it's a security issue and not necessarily a functional issue. But the point is software is damn complex and easy to screw up.
Maybe rather than doing ultimately less test (system instead of unit). We should be slowing down doing even more test at ALL levels (unit, system, GUI, security, performance, whatever) and building something to the best of our abilities. Or does that even matter anymore in the MVP build and sell quick software world?
While the idea of testing is good, TDD is so full of stupid quirks and BS disguised as "best practices" it's not even funny
These often result in slow (and sometimes useless) tests. If you want to use TDD to have more time at the foosball table, great
Tests that only test one thing? So I have 30% tests/condition check (or less) and the rest as boilerplate?
"and I do not write software test-first"
Me neither. It's idiotic
Also, TDD fanatics have a tendency of building software that has great coverage and thousands of tests, but fail the simplest of smoke tests.
The world has not had TDD for a long time, and software got delivered. (And sometimes much more stable and durable than today's "one update per week" software)
164 comments
[ 4.7 ms ] story [ 232 ms ] threadPart 1: http://www.justin.tv/confreaks/b/522089408 (starts at 11:00)
Part 2: http://www.justin.tv/confreaks/b/522101045
As a frontend developer, I have been moving towards a greater emphasis on functional & integration testing. We're also using scripter here, a test tool that actually compares screen caps with the actual on-screen state of our applications. So a purely visual approach.
There's a wonderful world of testing tools out there. I can imagine it's incredibly daunting to a newcomer trying to decide which to pick up and run with, especially with so many job specs specifically asking for "TDD", and now with DHH saying this.
c.f. Agile, Kanban, BDD
When making significant changes to implementation of something it's often worth just leaving acceptance tests in place for regression and deleting unit tests and TDDing your new implementation
I hear this argument a lot, and I think it is absolutely incorrect. If tests are hindering your system then you're just doing it wrong. It's like saying "climbing rope is a hindrance for the rock climber, since I have tied gordian knots around my feet and now cannot move."
TLDR doing something badly doesn't mean the thing you have botched is itself bad.
If you are rapidly prototyping something then writing tests first is a hinderance. But if you have well defined requirements then writing tests first will save you time.
What's best will differ depending on whether you're designing a web app or building an interface to an SAP system or an external gateway. (My guess is that DHH does very little of the well defined interface building type stuff)
No, I'd rather produce code that is well written, can be deployed, and taken over by other devs if needed, not several levels of testing paradigm's that need to be satisfied before code can be migrated.
Yes I agree that there is obviously a place for tests, and IMHO that's when certain business logic is considered implemented, and tests will need to catch all the edge-cases to make sure it will deliver in the future and not break with modification.
Oh yes - I agree completely with this.
I'm not sure I agree with testing things with Capybara though. How are you sure your backend controller actions are doing what they're supposed to be doing? Sure your browser might render what you need but what if a model attribute is set to false instead of true? How do you account for that using a frontend test suite?
I worry about a push away from unit tests. I can't imagine having to refactor a large application with only system-level integration tests to work with, especially in a dynamically-typed language like Ruby. I haven't written a Rails app in a long time though, maybe these days it's easier to just start over than to refactor.
But any bad day for dogma is a good day for the rest of us. Write software using whatever methodology works for you and always question the value of so-called "best-practices".
If you know all that up front, you might as well go with waterfall methodology.
I think point of this is... TDD is still good. But it's not ALWAYS good. A great way to frame thoughts, but writing tests last isn't "wrong" either.
If an app is worth writing, and worth writing tests for, do it justice and test the whole shooting match. Yeah it's hard, but that just makes it all the more worth doing. Automate your tests so that they cover everything that is important, from DOM elements on a dynamically built webpage to your model (and therefore the data that gets written to your SQL database), but don't pretend that stubs are substitutes for this. If your model relies on a database, let it rely on the database during the tests too, otherwise what exactly are you testing?
it depends, what do you want to test ?
If you are testing a single function, why would you care whether the data comes from the database, the network or from a mock ? All you need is to verify that the given function produces the expected output given the right (and wrong) inputs. Being able to test this way also makes easy to keep the components in your software decoupled.
My philosophy is to use mocks for unit tests and the real thing for integration tests.
Of course it helps to test against the real service as well, since it's often prohibitively time-consuming or impossible to model ever aspect of it, but I wouldn't discount mocks so easily.
http://martinfowler.com/articles/mocksArentStubs.html
But what you're getting is speed. Tests are at their most valuable when they give you quick feedback. As test suite times creep up, people stop running them as much, or at all. I've been on projects where all our tests run in seconds, and it is an enormously different development experience than when they run in minutes.
That being said, if you are writing a test on an algorithm and the data comes from a database this means you are not just writing a test on the algorithm - you are writing a test on the database with all the cruft that comes along with that. This test can be hundreds of times slower due to the DB dependency.
So mocks do have their place. The test pyramid is the answer - you need units (sometimes with mocks, integration, system).
In my experience most companies are way too integration heavy and very light on the unit side of things. For example for my current project our dev tests take 12 hours to run. This is the result of going too far away from the unit level and being too integration heavy.
There is a balance! Sounds like the rails world the balance is too unit heavy. In the enterprise world I think it's too integration/system heavy.
You aren't testing code against an entirely fictional representation of the system, you are testing one piece of code in isolation, and using a fictional representation of another piece of the system to eliminate variables.
Unit testing is about verifying that individual components do their job correctly -- when doing it, you literally don't care what other pieces of code would do, because that's out of scope of the unit test.
Testing that the whole system works together is system/integration testing, and is a different thing. You don't use mocks for system testing, only unit testing.
> If an app is worth writing, and worth writing tests for, do it justice and test the whole shooting match.
Yes, that's what system/integration testing is for.
That doesn't mean you shouldn't have code that is amenable to proper unit testing.
Once I learned about Cucumber and the idea of stating a requirement/test that you can't even parse yet, much less have a test for, much less have the code for, I started liking the idea of TDD as a wish list pyramid.
This allows you to think in terms of the big picture requirements, and then drill down as required to fulfill those requirements. Because you've spec/documented your design on the way down to the unit tests you can always step up a few levels and reconsider, and rewrite architectural "tests".
For example my first test is "I have a software tool for editing photos". Now I implement this test by checking if there is an executable in a path. Fail. Now I make a hello world exe for that path. Pass. Now I write a new test: "It opens an OpenGL window.", and later "it uses a mvc pattern", "the edits are represented as a scene graph", etc. all the way down to specific logic.
You later realize that a scene graph is not the right way to model your process, so you change that test to a different requirement, the altered requirements now redirect the TDD flow of an entire section of the application instead of just unit by unit.
This is the engineering form of confirmation bias. If you make something hard to do, it's hard to like it.
At the same time, I think when you design away as much complexity as possible at the product requirements level, as 37Signals proudly does with their products, it is hard to appreciate the complexity inherent to the requirements in other codebases.
I've seen this with other developers coming into large codebases with lots of complexity and scale requirements wanting to use a simple/naive ORM solution with no caching or worry about query speed/quantity. That solution falls over and they quickly learn that the requirements are more complex than a simple app with a few dozen users.
Basecamp is obviously at a high level of traffic and scaling complexity, but they still work to reduce requirement complexity which isn't always an option if you aren't the product owner.
Where are those mean TDD zealots? Can you point me at blog posts or mailing list messages displaying such behavior? I've never seen it personally. On the other hand, now and again a blog post like this comes up that's full of disdain towards the practice of testing first. I do TDD because it helps me get my work done. I'm happy to help others write tests if they wish, but I'd never look down on another developer because he or she doesn't use this tool. Some people like drawing diagrams, some people like to use an IDE... Do what works for you.
There was a period in time between 2005-2008 in Boston where if you didn't practice TDD you were persona non-grata.
I think interestingly enough, a good number of those people now have 10+ years experience programming and now understand what those of us who had 10 years experience at that point in time were trying to tell them.
"My thesis is that it has become infeasible, in light of what's happened over the last 6 years, for a software developer to consider himself 'professional' if he does not practice test driven development."
Robert C. Martin, 2008
http://www.infoq.com/interviews/coplien-martin-tdd
I think it's more productive to try out a technique, try out some variations based on others' advice, and then share the experience: I found this technique useful in these situations, but not in these. Here are some tips that I found effective, and here's what didn't work for me. Then, people can try out techniques for themselves, and learn from your experience.
If we want to generalise what's effective and what's not for software development overall, then I think we need a much rigorous approach then getting a vibe from the community. I've only ever seen a handful of studies that try to be unbiased and somewhat scientific in assessing techniques like TDD.
Because the world doesn't have enough drama.
I personally have gone a bit further and only write tests after a successful incarnation of a project (that means customers) that I want to keep developing, or if writing and running the test itself costs me less time then F5'ing a page or mucking around in the rails console.
I have noticed that it's much easier to throw away code that I haven't invested a lot of time in and usually the second time I write it (with tests), I have a much clearer idea of how to implement and test something.
I think his article highlights something else about FAD design patterns, hype and pressures. The amount of GUI developers I met running around yelling TDD was amusing, and refreshing meeting people who do what is right instead of implementing hype-patterns... new word? :¬)
I agree that many other people also conflate them, and are zealots for (or against) them together, but they are still separate concepts.
He does make some good arguments against Unit Testing, but he makes none against TDD.
> Test-first units leads to an overly complex web of intermediary objects and indirection in order to avoid doing anything that's "slow".
This is an argument against unit testing (and is a good example of the conflation). Personally I do TDD with slow, system tests, that mainly are testing business requirements.
He makes a couple of points that some TDD people are rabid, but that is just an argument that some people are jerks. It's not an argument against TDD. (And exactly the same is true of Unit Testing).
In consulting, I think this makes all the sense in the world.
a) When you hand off code, you can demonstrably prove what you handed off works. b) You can take the business objectives of whoever you are consulting for and translate those into high level tests, which helps you stay focused. c) You get to charge double what you'd normally have charged the client.
When running a in-house team at a startup none of those bullet points translate, and while TDD does offer other benefits, really at the early stages of the game every line of code you write needs to solve the problem you're trying to validate.
I live out of consulting, and my biggest projects is writing experimental consumer behavior simulation engines.
Is not that TDD does not have a value, it is just impractical
For once, since they are simulations, they are trying to predict the future, hence there is no right answer, there is the "Acceptable" answer, or the "Defensible" answer; so having a little test on the side that tells me that I am still producing the predicted result does not help much because there is no such thing as the correct predicted result
Sure I can do test to make sure I match the excel spreadsheet that I receive for spec, they would tell me that I copy the formulas correctly; but then, for seconds, I bring the results back and they test it with more real data, and they realize that we need to tweak 100s of different little formulas (which invalidate all the unit tests), change the order of a few of them, and introduced a couple of new ones (as I was told once "because I don't know what I was smoking that Tuesday, so we should use this new one"), and of course "can we have these changes for the end of the day?"
So it all unit tests are voided, make all those changes which are simple enough to be done in a couple of hours. And I am not going to know what is the correct result because there is no valid spreadsheet now, so I don't know in advance what is expected, hence I cannot write the test first
And wait until I try to tell them, the modifications will take 2 hours, but I need another 2 days to write the new set of unit tests, which since I wrote with the result of the modifications instead of before the modifications, they will not prove anything
Rinse and repeat 10s times a month for the last 4 years...
posts like this just encourage my opinion about the Merb merge with Rails being a disaster which helped Rails transform from "PHP in Ruby" to something beautiful to work with, while leading to self-destruction of Merb. Rails gained a lot of traction and is now abandoning any good decision Yehuda and other great developers made.
this post simply sums up to: "I don't know how to test-first Rails applications, and I've invented Rails. we're abandoning test-first." admitting you have a problem is a first step in fixing it, but this is not going to fix it in the long run.
To be fair a lot of unit test regimes wouldn't have caught something like heartbleed. Specifically cause it's a security issue and not necessarily a functional issue. But the point is software is damn complex and easy to screw up.
Maybe rather than doing ultimately less test (system instead of unit). We should be slowing down doing even more test at ALL levels (unit, system, GUI, security, performance, whatever) and building something to the best of our abilities. Or does that even matter anymore in the MVP build and sell quick software world?
These often result in slow (and sometimes useless) tests. If you want to use TDD to have more time at the foosball table, great
Tests that only test one thing? So I have 30% tests/condition check (or less) and the rest as boilerplate?
"and I do not write software test-first"
Me neither. It's idiotic
Also, TDD fanatics have a tendency of building software that has great coverage and thousands of tests, but fail the simplest of smoke tests.
The world has not had TDD for a long time, and software got delivered. (And sometimes much more stable and durable than today's "one update per week" software)
That's an argument against any kind of improvement; you can replace "TDD" with anything introduced since people started writing software.
I'm not against improving things, I'm against saying it's "the only true way" and ignoring the shortcomings.
Sure, do your RoR project using TDD, now, doing TDD in C is a whole different problem.