He mentions the small sample size (5 people) as a negative, but I wonder if this situation fits Jakob Nielsen's arguments about testing smaller groups.
I actually approve this argument. My personal experience also shows that usability testing can be done with very few people at a time, over many iterations.
Obviously, this is not a universal rule. It's useful for practical usability testing, but not for testing all cognitive processes.
Perhaps, if the variations are small enough, it may be sufficient.
Jakob Nielson argues that, specifically in usability testing, 5 people gives you a majority of your flaws but not the entire picture - where ~15 people would give you that. That is to say, more is better but there is diminishing returns. Given this, he argues that rather then expending resources to gather the entire picture for a single iteration, it's even better to test 3 sets of 5 people with design improvements between each set.
I think this argument is slightly different than just saying 5 people is enough for any sort of study. I still would have liked to seen more people. Or at least additional studies with modifications to inspect or confirm specific aspects of their findings.
I'm pretty sure he means languages like Lisp and (especially) Haskell which reduce the number of side-effects (and, not incidentally, the amount of boilerplate code) by abstracting away the loop and replacing it with some notion of mapping a function over some sequence of values (and/or key-value pairs, etc.).
The advantage is that off-by-one errors are largely eliminated, it's easier to see how to parallelize the code, and there is no visual noise associated with maintaining index variables.
The problem is that it's no longer as natural to bail on a loop early: You need some notion of exceptions or, in the most general case, first-class continuations, as in Scheme.
13 comments
[ 2.5 ms ] story [ 35.6 ms ] threadhttp://www.useit.com/alertbox/20000319.html
I think this argument is slightly different than just saying 5 people is enough for any sort of study. I still would have liked to seen more people. Or at least additional studies with modifications to inspect or confirm specific aspects of their findings.
http://debasishg.blogspot.com/2009/06/code-reading-for-fun-a...
http://www.oreillynet.com/ruby/blog/2006/05/post.html (section about "making Mongrels stable and secure"
http://news.ycombinator.com/item?id=483824
http://blog.rubybestpractices.com/posts/gregory/005-code-rea...
Reading code needs patience and getting used to the coder's style.
Guess loops aren't the source of ALL evil.
That's why I prefer languages without loops.
Do you mean like Lisp which favors recursive function (or lambda calculus which uses the Y-combinator :)?
The advantage is that off-by-one errors are largely eliminated, it's easier to see how to parallelize the code, and there is no visual noise associated with maintaining index variables.
The problem is that it's no longer as natural to bail on a loop early: You need some notion of exceptions or, in the most general case, first-class continuations, as in Scheme.
On the other hand: You can see loops as a special case of recursion. Though I like languages that don't treat this special case as a special case.