I've seen these systems and I think they can be good for teaching programming.
Basically, they force the student to think about the whole problem at once, design a complete solution, and implement it without making mistakes. If you screw up, you'll have a very tough time finding the mistakes, since the system doesn't tell you where the mistake might be. It prevents you from repeatedly patching your system rather than redesigning it, which is a big problem I see in a lot of programmers.
It's not good for the absolute beginners, but for people who know a little bit how to think about algorithms and data structures, and are trying to improve their code quality, it seems like a good exercise. I know I've had some rewarding experiences debugging code by thinking about it; using a debugger, while easier, seems generally less satisfying.
I don't see how these systems limit your ability to debug your code. You need just write your own testcases, which is good exercise in itself, and debug against those. If it passes your own testcases and still fails in the grading system, it indicates that either your testcases are not good enough, or that there is a bug in the system.
It prevents you from repeatedly patching your system rather than redesigning it, which is a big problem I see in a lot of programmers.
Uh, this is the norm in the industry. There is rarely ever a chance to do a complete redesign. From the micro to the macro, there's always some hacky solution hanging around.
I had a course that employed the same system. The course wasn't about debuging or real world applications, it was about algorithms. The task was to implement an algorithm correctly. If you did not implement the algorithm correctly, the test cases would fail. If you did not use the correct data structure, the test would time out. That's what it's all about. No specific clues were given so you can't do workarounds taylored to the test cases, or god forbid, hardcode the expected results.
You could do as much testing on your machine as you'd like, with your debugger of choice. And when you are sure you understood the algorithm and implemented it correctly, you can submit.
You can only submit your code for a given assignment a maximum of N times, where N is appropriate to the assignment difficulty and the level of the class. Then you can't pursue a haphazard "try this? nope, didn't work" strategy, and you have to reason out your program correctly on your own without appealing to the black box.
Alternately, if you wish to be less draconian, simply deduct points for every incorrect submission.
The places I have seen this kind of system in use, they work just like that. Limited number of submissions and flexible pointing system. Those systems gave points according to how many tests you passed, and there was minimum score to pass. The other system I have seen, also scored by coding style, ie. commenting and correct indentation etc. You could see style points and functionality points separately and there was separate minimum points (usually quite high in style).
And IMHO these are good tools. But they don't teach you debugging (at least not directly). They teach that you need to test and debug your code before you submit it, which imho is good practice anywhere, ie you should test and debug your code before committing to SVN (or push your changes to main tree in DVCS-talk).
Of course that should be told to students, and they should be taught proper debugging/testing-methods. I mean these kind systems do not prohibit doing your own tests and debugging those before submitting, now do they?
The problem isn't just that the system doesn't give enough feedback. The problem is that it also gives it at the wrong time.
In the real world, if your own testing misses a bug, you don't find out until it's live and causes a problem. In most grading systems, if your own testing misses a bug, you don't find out until the grader's (hopefully more comprehensive) testing catches it and you lose points.
In this system, if your testing misses a bug, you're immediately told that a bug exists, and given no other information -- but there's no actual penalty for doing this. In the real world, you're told what the incorrect behavior is, but there's a penalty in that this incorrect behavior has at least inconvenienced someone.
This is how the Facebook puzzles work except that you only get a pass/fail response. To help people ensure that they aren't failing due to some sort of packaging problem, they have simple puzzles (one step up from hello world), and it's not frowned upon to help someone with those.
What Facebook puzzlers end up doing is creating their own testcases and validating that those testcases encompass all the interesting corner cases and are complex enough to prove the efficiency of the algorithm in use.
Perhaps the hidden bit of learning from "The System" is that it forces students to think through the entire scope of possible inputs for their program and ensure they test against a representative set of cases.
I have a similar experience from college. We'd get an empty Java project, a "runner" and a set of simple test cases in the form of input/expected-output files.
We were told that the teachers would run hundreds of test cases more on out code, so we'd better think about it.
A very important thing to remember is that this course was a programming course (concurrent, I think), NOT a software development course. Many of these "I had to do this stupid thing in school that I'll never need in the real world"-rants miss those differences.
9 comments
[ 3.7 ms ] story [ 29.4 ms ] threadBasically, they force the student to think about the whole problem at once, design a complete solution, and implement it without making mistakes. If you screw up, you'll have a very tough time finding the mistakes, since the system doesn't tell you where the mistake might be. It prevents you from repeatedly patching your system rather than redesigning it, which is a big problem I see in a lot of programmers.
It's not good for the absolute beginners, but for people who know a little bit how to think about algorithms and data structures, and are trying to improve their code quality, it seems like a good exercise. I know I've had some rewarding experiences debugging code by thinking about it; using a debugger, while easier, seems generally less satisfying.
Uh, this is the norm in the industry. There is rarely ever a chance to do a complete redesign. From the micro to the macro, there's always some hacky solution hanging around.
You could do as much testing on your machine as you'd like, with your debugger of choice. And when you are sure you understood the algorithm and implemented it correctly, you can submit.
You can only submit your code for a given assignment a maximum of N times, where N is appropriate to the assignment difficulty and the level of the class. Then you can't pursue a haphazard "try this? nope, didn't work" strategy, and you have to reason out your program correctly on your own without appealing to the black box.
Alternately, if you wish to be less draconian, simply deduct points for every incorrect submission.
And IMHO these are good tools. But they don't teach you debugging (at least not directly). They teach that you need to test and debug your code before you submit it, which imho is good practice anywhere, ie you should test and debug your code before committing to SVN (or push your changes to main tree in DVCS-talk).
Of course that should be told to students, and they should be taught proper debugging/testing-methods. I mean these kind systems do not prohibit doing your own tests and debugging those before submitting, now do they?
In the real world, if your own testing misses a bug, you don't find out until it's live and causes a problem. In most grading systems, if your own testing misses a bug, you don't find out until the grader's (hopefully more comprehensive) testing catches it and you lose points.
In this system, if your testing misses a bug, you're immediately told that a bug exists, and given no other information -- but there's no actual penalty for doing this. In the real world, you're told what the incorrect behavior is, but there's a penalty in that this incorrect behavior has at least inconvenienced someone.
What Facebook puzzlers end up doing is creating their own testcases and validating that those testcases encompass all the interesting corner cases and are complex enough to prove the efficiency of the algorithm in use.
Perhaps the hidden bit of learning from "The System" is that it forces students to think through the entire scope of possible inputs for their program and ensure they test against a representative set of cases.
We were told that the teachers would run hundreds of test cases more on out code, so we'd better think about it.
A very important thing to remember is that this course was a programming course (concurrent, I think), NOT a software development course. Many of these "I had to do this stupid thing in school that I'll never need in the real world"-rants miss those differences.