This is pretty far down the page. Possibly only a discussion between me an you ;-) I enjoyed your article. Thank you for writing it. However, I have a different point of view which you might find interesting.
There is a difference between TDD and testing. Testing is something you do to see if what you've done is correct. Quite frequently I work the way you described and it's very nice to feel confident that I have implemented what I set out to implement.
At that point, I often just throw the test away. Sometimes if I have a pair programming with me, they'll be shocked. "Why did you throw away that perfectly good test", they will ask. "Because I wrote it to find out if the code works and now I know that it works".
TDD is about something else. With TDD you aren't so much worried about the correctness of the behaviour, you're worried if the behaviour has changed since the last time you touched the code. Ideally, if it has changed, it would be nice to know what changed. Even better, it would be nice to know what assumption was violated that caused the change in behaviour.
Imagine that you have an application that you have tested inside and out. It works perfectly. Next imagine that you have a special magical device that tells you if the behaviour changes every time you modify the code. It doesn't tell you if the behaviour is correct, it simply tells you that it changed, what the change was, and what caused it to change.
Now every time you add code, if the behaviour doesn't change, you know that it is still operating correctly -- because it was before and it hasn't changed. If the behaviour does change, you can observe the behaviour and decide if the change is good or bad. If the change is good, then the code is still operating correctly. If the change is bad, then the code is operating correctly except for the change. If your magic device also tells you where your assumptions are violated, then it becomes easy to decide how to make the behaviour good again.
There are a couple of really cool things about this magic device. It doesn't need to know if the system is operating correctly or not (which is difficult in most cases and impossible in the general case). It just needs to know if the behaviour is the same as before (which is a much simpler problem). The other really cool thing is that if you make a change to the behaviour of the system and the magic device doesn't detect it, then you know the magic device is broken. Since the magic device is very useful, it is probably a good idea to fix it right away.
Of course the magic device is a test suite. But it's important to understand that it's a very special kind of test suite. It measures behaviour and detects if the behaviour changes -- not if the behaviour is correct (you can test that separately, either in a manual or automated fashion). Often in legacy code I'll write a whole bunch of "tests" by running functions with various inputs and recording the results. That's all I need. I don't need to know if the results are correct or not. As I modify the code and watch the differences, I can determine if the differences are good or not, and I may find bugs. But bugs are not my main worry with this style of test suite. I'm using the test suite to inform (and later confirm) my assumptions about the behaviour.
Second, this kind of test suite needs to tell you what is actually wrong. For example, you might have a test that determines if a particular result was produced. If the test "fails", you might say "the test fails". This is pretty useless, though. Now I have to go and debug the code. Instead, I want to be told exactly what is different between what I expected and what I received. I also want to be told what context the program was in when I got the result. So I need to be able to see at a glance the input, output and processing (big hint: fixtures, as ...
2 comments
[ 3.0 ms ] story [ 18.7 ms ] threadThere is a difference between TDD and testing. Testing is something you do to see if what you've done is correct. Quite frequently I work the way you described and it's very nice to feel confident that I have implemented what I set out to implement.
At that point, I often just throw the test away. Sometimes if I have a pair programming with me, they'll be shocked. "Why did you throw away that perfectly good test", they will ask. "Because I wrote it to find out if the code works and now I know that it works".
TDD is about something else. With TDD you aren't so much worried about the correctness of the behaviour, you're worried if the behaviour has changed since the last time you touched the code. Ideally, if it has changed, it would be nice to know what changed. Even better, it would be nice to know what assumption was violated that caused the change in behaviour.
Imagine that you have an application that you have tested inside and out. It works perfectly. Next imagine that you have a special magical device that tells you if the behaviour changes every time you modify the code. It doesn't tell you if the behaviour is correct, it simply tells you that it changed, what the change was, and what caused it to change.
Now every time you add code, if the behaviour doesn't change, you know that it is still operating correctly -- because it was before and it hasn't changed. If the behaviour does change, you can observe the behaviour and decide if the change is good or bad. If the change is good, then the code is still operating correctly. If the change is bad, then the code is operating correctly except for the change. If your magic device also tells you where your assumptions are violated, then it becomes easy to decide how to make the behaviour good again.
There are a couple of really cool things about this magic device. It doesn't need to know if the system is operating correctly or not (which is difficult in most cases and impossible in the general case). It just needs to know if the behaviour is the same as before (which is a much simpler problem). The other really cool thing is that if you make a change to the behaviour of the system and the magic device doesn't detect it, then you know the magic device is broken. Since the magic device is very useful, it is probably a good idea to fix it right away.
Of course the magic device is a test suite. But it's important to understand that it's a very special kind of test suite. It measures behaviour and detects if the behaviour changes -- not if the behaviour is correct (you can test that separately, either in a manual or automated fashion). Often in legacy code I'll write a whole bunch of "tests" by running functions with various inputs and recording the results. That's all I need. I don't need to know if the results are correct or not. As I modify the code and watch the differences, I can determine if the differences are good or not, and I may find bugs. But bugs are not my main worry with this style of test suite. I'm using the test suite to inform (and later confirm) my assumptions about the behaviour.
Second, this kind of test suite needs to tell you what is actually wrong. For example, you might have a test that determines if a particular result was produced. If the test "fails", you might say "the test fails". This is pretty useless, though. Now I have to go and debug the code. Instead, I want to be told exactly what is different between what I expected and what I received. I also want to be told what context the program was in when I got the result. So I need to be able to see at a glance the input, output and processing (big hint: fixtures, as ...