Google Caffeine Explained (theregister.co.uk)

TL;DR: Caffeine is apparently

SELECT * FROM BigTable WHERE `Keywords` LIKE '%$query%';

This is a tangent, but does anybody else simply not read articles like this because all the blinky flashy ads are painful to look at?

Finally some insight into google caffeine. Its remarkable what they can do with indexing in real time. MapReduce is great for number crunching when you have time to wait for the data but in google's case thats just not feasible. Could they have added further capacity to speed up the process? Yes but would that have been more efficient than what they've now produced? Unlikely. I cant wait to see the paper thats produced. Some of the smartest minds have worked on this. Working on huge data sets like this has got to be fun.

If you are looking to build something similar, check out the open source elasticsearch (http://www.elasticsearch.com). It was built with the mentioned architecture in mind (sorry for the "plug", I am the creator of elasticsearch).