The Britney Spears Problem (americanscientist.org)

Ignore the name - this is a clearly-written introduction to stream processing algorithms.

I enjoyed this. Would most large scale applications of these ideas happen with Hadoop? Any other software I should be aware of for, e.g., estimating frequencies in a large text stream?