Here's a little background about gistpoint, I was an FPGA developer working on DSP systems at a big company in D.C. Last May, bored of my job and frustrated at my career progress, I took the paths most travelled and quit my job to wander around the world. Along the way, I picked up ruby/rails and javascript because I want to transition into web programming. The learning curve was steep coming from the hardware world. Plus running from city to city also made it tough to maintain focus. 2 months ago, after many false starts and months of distraction, I finally decided to push it through completion.
gistpoint is a web app that lets you search and submit summaries for online articles. It's unique in that it has a scorer that rates how relevant a summary is compared to its fulltext. This is done by analyzing several factors such as information coverage, sentence grammar, word stemming, etc. So the best summary will rise to the top and gets shown as the default. No signup/login is needed to submit summaries, and quality is controlled without voting. When no user-submitted summary is available, it will extract the most relevant passage from the fulltext. From my experience if I just naturally summarize an article I just read, the scorer rates it highly compared to the extracted one.
Please let me know what you think. If anyone wants to know more about the scoring algorithm, I'll be happy to write a post about it. I definitely open source the summary extraction and grammar gems on github. They're already on rubygems https://rubygems.org/profiles/50055 for you to download, I just need to create the documentations.
Very cool, nice work. It would be interesting if you could do this with legislation, both proposed and passed. You'd probably have to add/ modify relevancy factors but it could be helpful to not have to read legalese and still get the gist of laws.
In my personal opinion your use of Comic Sans is fine considering the context, but you could placate the font connoisseurs and Comic Sans haters by using a handwriting font face such as one of these:
User-generated Cliff's Notes for online articles? I can see some value there, but in as much as Cliff's Notes has value. I worry sometimes about the intellectual shortcuts people are taking in not reading articles completely or depending on other's feedback / scores / comments to determine the value of the content. Since when was reading for comprehension a chore? I know people are short on time, but is it wise to shortcut really understanding something?
I understand your point, the way I see people use this is to determine before committing their time if the article content fits their interests and worth their time. It's not to bypass reading articles altogether, it's to filter out what they want to read.
Agreed - but in that case TL;DR is a misnomer. "Too long, didn't read", means that the user DIDN'T read it! Or, at least is summarizing it in a way that the user should not have to read it.
It would be better to call these "abstracts" if they are used as you imply above -- a synopsis of the article for those interested in learning more.
I think TL;DR is a symptom of intellectual laziness, whereas abstracts have long been used for research papers and the like.
I like the concept, but I feel that your algorithm for generating an initial summary needs a lot of work. For example, I was experimenting with the API:
thanks for the kind words, it's probably a problem with the fulltext extraction. If the fulltext is extracted correctly, the summary extraction should be good. Btw, I use the ruby-readability gem for the fulltext extraction.
Very interesting concept. I think it's necessary to find a way to gauge the most popular articles rather than trying to tl;dr the entire internet.
For instance, you could look into some APIs for the most popular submissions on Hacker News and reddit and present them as articles that need summaries.
You could also create a bookmarklet that users or authors can use to recommend an article to be summarized, which will then appear in some kind of summary queue on the main page. (Predefined) tags could also be included, so people who are only interested in summarizing certain articles, can cherry-pick these.
Another thing is to suggestion is to reward productive users and upvoted summaries with a link to the writer's flattr profile. People can do this, even though the person has a flattr profile now[1][2], and I think this might increase the incentive even more.
If this takes off, you could create some browser extensions similar to the Hacker News Sidebar[3], which displays a sidepanel on websites that have been submitted to Hacker News.
Great feedbacks! Actually, it already measures article popularity using twitter count api. The articles shown on the frontpage are sorted by twitter link count.
If you install the bookmarklet and click on any page it will show you the machine extracted summary. You can click edit to submit your own summary.
Great idea to incentivize users to submit summaries by linking to their flattr profile.
24 comments
[ 4.2 ms ] story [ 51.1 ms ] threadHere's a little background about gistpoint, I was an FPGA developer working on DSP systems at a big company in D.C. Last May, bored of my job and frustrated at my career progress, I took the paths most travelled and quit my job to wander around the world. Along the way, I picked up ruby/rails and javascript because I want to transition into web programming. The learning curve was steep coming from the hardware world. Plus running from city to city also made it tough to maintain focus. 2 months ago, after many false starts and months of distraction, I finally decided to push it through completion.
gistpoint is a web app that lets you search and submit summaries for online articles. It's unique in that it has a scorer that rates how relevant a summary is compared to its fulltext. This is done by analyzing several factors such as information coverage, sentence grammar, word stemming, etc. So the best summary will rise to the top and gets shown as the default. No signup/login is needed to submit summaries, and quality is controlled without voting. When no user-submitted summary is available, it will extract the most relevant passage from the fulltext. From my experience if I just naturally summarize an article I just read, the scorer rates it highly compared to the extracted one.
Please let me know what you think. If anyone wants to know more about the scoring algorithm, I'll be happy to write a post about it. I definitely open source the summary extraction and grammar gems on github. They're already on rubygems https://rubygems.org/profiles/50055 for you to download, I just need to create the documentations.
Can we use words than the general HN audience would understand?
When it is used at the start of a sentence, it is introducing a short summary (ostensibly accurate, but sometimes a joke).
Someone on Quora says it actually dates back to Slashdot, not Reddit: http://www.quora.com/What-is-the-origin-of-TL-DR
http://www.google.com/webfonts?subset=latin&category=han...
It would be better to call these "abstracts" if they are used as you imply above -- a synopsis of the article for those interested in learning more.
I think TL;DR is a symptom of intellectual laziness, whereas abstracts have long been used for research papers and the like.
The input page:
http://bookflavor.com/dukan-diet-2-steps-lose-weight-2-steps...
The API results:
http://gistpoint.com/get?u=http://bookflavor.com/dukan-diet-...
Rather than choosing a snippet of the main text, it chose a few words from one of the menu items...
If you can fine tune the algorithmic generation of snippets in the API this could be very useful. Keep up the good work!
It would be nice if the API would take a parameter for the approximate number of words to return, so a 100 word vs a 500 word summary.
For instance, you could look into some APIs for the most popular submissions on Hacker News and reddit and present them as articles that need summaries.
You could also create a bookmarklet that users or authors can use to recommend an article to be summarized, which will then appear in some kind of summary queue on the main page. (Predefined) tags could also be included, so people who are only interested in summarizing certain articles, can cherry-pick these.
Another thing is to suggestion is to reward productive users and upvoted summaries with a link to the writer's flattr profile. People can do this, even though the person has a flattr profile now[1][2], and I think this might increase the incentive even more.
If this takes off, you could create some browser extensions similar to the Hacker News Sidebar[3], which displays a sidepanel on websites that have been submitted to Hacker News.
[1]: http://blog.flattr.net/2011/04/opening-the-floodgates
[2]: http://blog.flattr.net/2011/05/flattr-almost-anyone
[3]: https://chrome.google.com/webstore/detail/hhedbplnihmkekhgma....
If you install the bookmarklet and click on any page it will show you the machine extracted summary. You can click edit to submit your own summary.
Great idea to incentivize users to submit summaries by linking to their flattr profile.