So I made an overnight prototype of this about 2 years ago as a side project from my day job. It has since grown into a startup and I'm proud to be associated with a great team. We have been working very hard the last few months to build a beta search and analysis tool and to show off a good public sampling of the capabilities of the platform we have developed. We hope to gain some exposure to secure second round funding for upcoming offerings and some commentary we need to improve.
It still has some rough edges which we very much hope you all can help us work out. As we obviously want to monetize parts of our platform in time, we want to listen to the community to create the best tool possible. To that end, we have already open sourced some of the libraries behind Tawlk via http://github.com/Tawlk and we welcome any contributions or forks. We very much believe in the "Open source -Almost- Everything" model.
Also worth nothing about this demo, is that to make good use of our current limited server resources and to avoid hitting API and CPU limitations, we are utilizing many relatively new techniques including combinations of WebSockets, HTML5 localStorage, HTML5 App Caching, and direct browser-side data collection to make it work.
I hope you find the beta interesting and intuitive. I look forward to your comments or questions.
If you find bugs, please screenshot them with steps to reproduce so they can be taken care of quickly and please be nice to our servers :-)
EDIT: If you see connection errors you can dismiss it with the "x" in the top right. Search should still work, you just wont get any of the websocket supported sidebar stats (which i very much encourage you to try out on another network). We are working on some fallback connection methods to resolve this for firewalled and otherwise unable-to-connect users.
Hey Irvick, I think the timestamps on some content is coming out wrong. I saw a facebook update which had timestamp 42 years ago. I'm guessing Unix timestamp of 0 seconds. Also, I saw another tweet which showed 'a few seconds ago' but was actually posted a few days ago. Great product BTW.
That date conversion mainly happens here https://github.com/Tawlk/hyve/blob/master/hyve.js#L208-211 which is fairly straight forward. I will have to do some more poking around to see whats going on. If you find any specific items that trigger this, please let me know so I can debug them individually.
I had a chance to see the tech demos for Tawlk, as lrvick is a member of our local hackerspace, and I have to say I was blown away. This thing is amazing, and the way the infrastructure works is pretty sick too. Definitely worth saving the link and checking back when you have more time.
Thanks, I missed the X before I saw your comment, but still, nothing comes through for me when searching in Chromium 17.0.950.0. Firefox nightly works.
The connection errors happen when there is limited websocket support due to browser or network. We are working on a migration to sockjs so we can have some flexible fallbacks just in case. If you dismiss the error the main search should still work, you just sadly wont get any stats. Probably best to wait till your on a direct connection for now.
So this pretty much polls Twitter, Digg, Picasa, Youtube and Flickr for the search term you enter, then updates in real time?
I like the interface a lot, and it certainly seems well architected (in that I've had it open in three tabs simultaneously and not noticed any lag whatsoever).
However, to use an idea from the Intercom.io blog[1], what job would I hire this app to perform?
It searches others, as data is available. Most of the results your getting are from the open source hyve library we have been developing to help manage load and API limits. http://github.com/Tawlk/hyve . If a service you like is not included, please add it to hyve, and it may well also make its way into our back-end analysis platform.
Well currently it could be used by brand managers to see how popular or unpopular a given topic or person is, or to have an easy way to digest overall stats about the topics social presence.
You could also do things casual like just seeing how people feel about Obama compared to Justin Beiber or android phones or use it while watching a live event to see what people are saying about it. Live results are one of the ways this stack really shines.
As far as where we are going to take the platform, we have some strong ideas ready to implement provided we get the right investment group interested. Lots of things that would be more interesting to those wanting to hire this to do a "job" for them or their company.
Didn't understand how the sentiment analysis works, since results look quite not what I expected. I did a test search on 2012 presidential candidates:
Obama: 44% positive ("barack obama" is 62% though)
Mitt Romney: 67% positive
Michele Bachmann: 50% positive
Herman Cain: 71% positive (?)
Ron Paul: 71% positive
Rick Perry: 81% positive (??)
I am not sure the sentiment calculation is just based on Positive/Negative dictionary comparison or something else. Is there a logical explanation of above percentages? Do results vary for other people?
So currently this is based on taking a sizable rolling window of textual posts for each topic then running it through our synt library (http://github.com/tawlk/synt) which does sentiment classification with a NaiveBayes classifier trained against iterations of a couple million samples. This setup averages about 80% accuracy against new labeled sample sets.
We however only have the server resources to have a large rolling window for a limited number of topics (currently the ones in the side bar).
When you searched for "Barack Obama", it based the sentiment score on just what your browser collects live on the fly, whereas when you searched "obama" you got sentiment calculations based on our server collected rolling data set which is far more stable.
Both are fairly accurate, however they differ in how large of a window they are averaging against.
As topics become more popular and as server limitations allow we automatically migrate them server side for more aggressive collection to provide more reliable sentiment.
We also only provide scores and reach assessments for topics we posses enough server-side data for to justify it.
Some of the sites polled, like Reddit, usually have fairly active comment threads. But I can't click through to the comments, only directly to the thing that Reddit is linking to.
That is reasonable. I will look into finding a tidy way to resolve this. Perhaps with the number of comments next to the post which when clicked lets you skip directly there?
Connection Failure: This is probably due to the fact
your browser does not appear to support a technology
called "WebSockets" which Tawlk relies on. We are
working hard to bring full compatibilty with a wider
range of browsers soon. In the interim please consider
using the latest version of Chrome, Firefox or Safari
This also happens when chrome simply drops the connection even if it is "supported". Chrome 16 and 17 have switched to a newer websocket standard which apparently does not deal with packet loss as well? For now compare with Firefox as well as its Websocket implementation seems to handle a bit better than latest chrome builds in some situations. At the very least we will push out a messaging change for this soon.
Once the load spike dies down a bit we will be working to push out some protocol updates to help provide fallback support for browsers that for whatever reason can't maintain a stable websocket connection.
I'm interested to see how this product will grow and how you can eventually capitalize on the idea to generate revenue. Many companies today are already experimenting with the idea of content analysis to gain insight about what the social web is saying about their company and/or to profile their customers using lifestyle based analytics.
One question I have is do you plan on filtering out the noise? There seems to be some content that I would consider spam or highly unrelated to the topic that I choose to search.
Yes, our next phase in development and design will be focused on relevancy. We will work on filtering out unrelated results and show those that are the most popular first.
We have developed a strong revenue strategy around social data and sentiment analysis. Feel free to shoot us an email for more info. In the meantime we are just looking for people like you to enjoy the product and have fun with it!
the left to right scrolling content is really disorienting for me -- it forces my eyes to go right to left to find new content, which is the opposite of how english speakers read. This makes me feel as if i'm going backward rather than forward. i think if you were to flip the direction of the scroll it would be a far, far more pleasurable reading experience. Looks nice, regardless.
Never considered that. Perhaps there could be an option to reverse it, Or even an option for vertical scrolling in the future too so people can have easy to change sticky preferences. Thanks for the input
Interesting. Never seen this behavior. That error should really only happen if some reason the Jsonp APIs were not responding properly for some reason. Is it possible your network blocks some major social media APIs like Reddit, Twitter, Facebook etc?
Tawlk communicates with all the major APIs directly from your browser on your side of the network.
Will Talwk be restricted to social media? Latest news/articles/blogs can add a lot of value to the feed, especially if one wants to follow a live event. Also, if user can integrate their facebook/twitter account to Talwk, the site can then highlight and prioritize the stuff shared by people in the user's network.
Also, Just to check the buzz about Talwk, I visited tawlk.com/#!/tawlk and witnessed the oh-so-tragic condition of the enlish language on the interwebs.
Well we use social media to do harvesting, however we will be indirectly supporting a wide range of news and content. If 50 people all tweet the same article from techcrunch, that mentions the word android, then that is a pretty good indication we should call that out specifically.
We will be implementing a lot of features like this soon to give a far more relevant and diverse range of data based on its social media popularity for a given topic.
This looks super interesting and useful, really quick too. Most of the bugs I've spotted have already been mentioned.
I noticed you've open sourced a bunch of your modules in Javascript and Python. Out of interest what does your technology stack look like? (framework choice etc)
The only thing not really detailed here is that the whole stack runs on a set of redundant servers with Round Robin DNS at the helm to randomly distribute between them.
Also "caching proxy" should really just be called "routing proxy" as that is all that we are using it for. If websockets go to Tawlk app, if else go direct to nginx static serve.
51 comments
[ 3.1 ms ] story [ 104 ms ] threadIt still has some rough edges which we very much hope you all can help us work out. As we obviously want to monetize parts of our platform in time, we want to listen to the community to create the best tool possible. To that end, we have already open sourced some of the libraries behind Tawlk via http://github.com/Tawlk and we welcome any contributions or forks. We very much believe in the "Open source -Almost- Everything" model.
Also worth nothing about this demo, is that to make good use of our current limited server resources and to avoid hitting API and CPU limitations, we are utilizing many relatively new techniques including combinations of WebSockets, HTML5 localStorage, HTML5 App Caching, and direct browser-side data collection to make it work.
I hope you find the beta interesting and intuitive. I look forward to your comments or questions.
If you find bugs, please screenshot them with steps to reproduce so they can be taken care of quickly and please be nice to our servers :-)
EDIT: If you see connection errors you can dismiss it with the "x" in the top right. Search should still work, you just wont get any of the websocket supported sidebar stats (which i very much encourage you to try out on another network). We are working on some fallback connection methods to resolve this for firewalled and otherwise unable-to-connect users.
In the mean time, here is a reference rendering: http://dumpon.us/media/uploads/TAWLK_GUI_HORIZONTAL_V18_1.jp...
[Edit: Added a missed word]
I tried with Safari and it worked.
Opera - OSX 10.6
I will probably check it out when I get home from work but as of now I am a little annoyed that I cannot try it out.
The connection errors happen when there is limited websocket support due to browser or network. We are working on a migration to sockjs so we can have some flexible fallbacks just in case. If you dismiss the error the main search should still work, you just sadly wont get any stats. Probably best to wait till your on a direct connection for now.
I like the interface a lot, and it certainly seems well architected (in that I've had it open in three tabs simultaneously and not noticed any lag whatsoever).
However, to use an idea from the Intercom.io blog[1], what job would I hire this app to perform?
[1] http://blog.intercom.io/when-personas-fail-you/
Well currently it could be used by brand managers to see how popular or unpopular a given topic or person is, or to have an easy way to digest overall stats about the topics social presence.
You could also do things casual like just seeing how people feel about Obama compared to Justin Beiber or android phones or use it while watching a live event to see what people are saying about it. Live results are one of the ways this stack really shines.
As far as where we are going to take the platform, we have some strong ideas ready to implement provided we get the right investment group interested. Lots of things that would be more interesting to those wanting to hire this to do a "job" for them or their company.
Obama: 44% positive ("barack obama" is 62% though)
Mitt Romney: 67% positive
Michele Bachmann: 50% positive
Herman Cain: 71% positive (?)
Ron Paul: 71% positive
Rick Perry: 81% positive (??)
I am not sure the sentiment calculation is just based on Positive/Negative dictionary comparison or something else. Is there a logical explanation of above percentages? Do results vary for other people?
We however only have the server resources to have a large rolling window for a limited number of topics (currently the ones in the side bar).
When you searched for "Barack Obama", it based the sentiment score on just what your browser collects live on the fly, whereas when you searched "obama" you got sentiment calculations based on our server collected rolling data set which is far more stable.
Both are fairly accurate, however they differ in how large of a window they are averaging against.
As topics become more popular and as server limitations allow we automatically migrate them server side for more aggressive collection to provide more reliable sentiment.
We also only provide scores and reach assessments for topics we posses enough server-side data for to justify it.
Once the load spike dies down a bit we will be working to push out some protocol updates to help provide fallback support for browsers that for whatever reason can't maintain a stable websocket connection.
One question I have is do you plan on filtering out the noise? There seems to be some content that I would consider spam or highly unrelated to the topic that I choose to search.
We have developed a strong revenue strategy around social data and sentiment analysis. Feel free to shoot us an email for more info. In the meantime we are just looking for people like you to enjoy the product and have fun with it!
For Flickr that is a very fair point, we need a way to link to the original Flickr URL. That is a UX oversight we need to resolve.
Where are you seeing "Reset Tawlk?"
Console is full of errors: http://dl.dropbox.com/u/533753/Screen%20Shot%202011-12-15%20...
It also took me a long time to figure out how to close the about modal...
Tawlk communicates with all the major APIs directly from your browser on your side of the network.
Also, Just to check the buzz about Talwk, I visited tawlk.com/#!/tawlk and witnessed the oh-so-tragic condition of the enlish language on the interwebs.
We will be implementing a lot of features like this soon to give a far more relevant and diverse range of data based on its social media popularity for a given topic.
I noticed you've open sourced a bunch of your modules in Javascript and Python. Out of interest what does your technology stack look like? (framework choice etc)
The only thing not really detailed here is that the whole stack runs on a set of redundant servers with Round Robin DNS at the helm to randomly distribute between them.
Also "caching proxy" should really just be called "routing proxy" as that is all that we are using it for. If websockets go to Tawlk app, if else go direct to nginx static serve.