HN Explorer

Ask HN: What does the Twitter/Datasift deal mean for developers?

9 points by rwwmike ↗ HN
So, I write for ReadWriteWeb and I've been paying pretty close attention lately to the situation with Twitter, API access, the API Terms of Service and the developer ecosystem. Earlier this week, Twitter announced a deal with DataSift (formerly TweetMeme) to sell access to Twitter info (including historical) at a price much lower than the full firehose access offered with Gnip. The access is based on search, comes with extra data from Klout, PeerIndex, Qwerly and Lexalytics.

(Here's our writeup of the announcement too: http://www.readwriteweb.com/archives/twitter_announces_fire_hose_marketplace_up_to_10k.php )

So, I'm looking for reactions and interpretation from developers on what this means for the Twitter developer ecosystem. Got one? Got some insight? Let's talk!

10 comments

[ 3.0 ms ] story [ 48.1 ms ] thread
rwwmike
Oh, and I'm Mike. My email is mike@readwriteweb.com if you don't want to just comment here. If you are willing to be quoted for a story, I'll need to know your full name and position/job title/company, what have you. :)
jrhchicago
My understanding is the primary difference is that data from Datasift is not for display (analysis only), which, if accurate, would only work for a small set of companies currently accessing the data.
rwwmike
I thought that I had asked them about that part. Hmmm.
sinak
I'm not sure if that's true. Many of their use case examples seem to display the data from user's streams, meaning that it's not just analysis.
doug_williams
Hi -- The license the consumer receives is strictly for non-display use cases, meaning you are not able to go to Gnip or DataSift to license Twitter content to display to end users. We (Twitter) still maintain a robust set of APIs for display use cases which we will license directly to partners for display products.

Thanks, Doug

jrhchicago
I'm basing that on this excerpt from the announcement on the Datasift website

"This means that developers interested in building tools to monitor and analyze Twitter data can now filter Tweets from the full Twitter Firehose, using DataSift for non-display analysis."

jvaleski
check your facts by reaching out to Twitter directly.

I'm Jud Valeski, CEO of Gnip (jud@gnip.com). So we're all running on factual data, apparently a problem these days,...

Twitter's value added resellers (e.g. Gnip and DataSift) must... - sell filtered data at no less than $0.10/1k Tweets consumed. Gnip makes this price explicit, whereas DS, apparently (I can't fully tell so check w/ them directly) rolls that number into their costs, making it opaque.

- sell only to firms who will not display the Tweets in "public".

- sell only to firms who will not programmatically re-syndicate the raw Tweets further downstream.

rwwmike
Thanks Jud. I'll email you now. :)
doug_williams
Mike -- Your statement is factually false. Both DataSift and Gnip have two components of the cost they charge consumers for their search-based products. The first is a data license cost (10 cents per thousand tweets sent to the consumer) and a processing cost. In the case of Gnip, the processing cost is essentially the cost of the machine you license, while for DataSift it is the AWS-style compute cost. How they show these costs to the consumer may vary, but they are each compensate Twitter for the cost of the data license.

To simplify:

<total cost> = <data cost> + <processing cost>

<total cost> = <$0.10 * #tweets % 1000> + <processing cost levied by Gnip or DataSift>

Thanks, Doug