Hey folks, I'm a HNer. I created NewsBlur with Hacker News in mind the entire time. In fact, I thought up the idea during a summer NYC Hackers and Founders last year. For the past 16 months, I kept trying to pick a date when NewsBlur would be ready to submit. Finally, embarrassed as I am by the bugs galore, today is that day and I am glad to say that NewsBlur is far enough along to show off.
NewsBlur is an RSS feed reader that tries to do two things very well:
1) Shows you the original site instead of a context-less feed. Read the original and NewsBlur marks the stories you've read.
2) Filter stories you either like or dislike. A three-stop slider goes between dislike, neutral, and like (red, yellow, and green). Training is super-easy and all click-based (as opposed to you having to writing out what you like in a site, NewsBlur asks you, semi-Hunch-style, your opinions on facets of the site).
I started working on NewsBlur to see if I could do it. Put the AI together with the back-end feed processing and fetching, along with the nifty front-end of the original site. This is one of those projects where I just kept pushing in all directions until I felt I had something good, not knowing if I could do it at all, but believing the entire time that I was able to complete the project.
If you have a project kicking around in the back of your mind, just dedicate enough weeks and months to get it done and done well. Provided you do that, you're almost guaranteed to succeed in some way. Those first few months of writing NewsBlur, I felt like a fraud, wishing I could just get threaded feed fetching to not break every few minutes. Even what I considered to be simple python gave me headaches. (Turns out threading is evil, use multiprocessing instead.)
NewsBlur is entirely open-source: http://github.com/samuelclay/NewsBlur/. The iPhone app, the front-end JavaScript, and the back-end Python. Pick it apart, but know that it is more a learning experience than a business. Turning it into something profitable may not be easy, but I only need 80 premium users to make that happen.
It's interesting to see somebody tackling this (these) problem(s) of managing feeds through AI. As I said before http://news.ycombinator.com/item?id=1362554 , kudos and keep up the good work!
Whether or not its ultimately a success, you really developed a kick-ass product! It looks good, seems to work pretty well. If you did this all on your own, thats a tremendous accomplishment.
While I may have written and designed NewsBlur, I am using a ton of other open-source technologies:
* Django: Web framework written in Python, used to serve all pages.
* Celery & RabbitMQ: Asynchronous queueing server, used to fetch and parse RSS feeds.
* MongoDB, Pymongo, & Mongoengine: Non-relational database, used to store stories, read stories, feed/page fetch histories, and proxied sites.
* PostgreSQL: Relational database, used to store feeds, subscriptions, and user accounts.
* jQuery: Cross-browser compliant JavaScript code. IE works without effort.
* Underscore.js: Functional programming for JavaScript. Indispensible.
* And many miscellaneous jQuery Plugins: Everything from resizable layouts, to progress bars, sortables, date handling, colors, corners, JSON, animations. See the complete list on NewsBlur's GitHub Javascript folder: http://github.com/samuelclay/NewsBlur/tree/master/media/js/
Nicely made... shall have to look into it further.
First bug I noticed, most easily visible with Daring Fireball: click on a story from the bottom list. If there's more than one story headline in view, it also marks the bottom one as read.
1. Can you rate individual posts (I'm at work so I can't spend too much time on it right now) instead of just tags/authors? It wouldn't feel like that much work to me as a user. You could then just read the tags/authors from the rated posts and sort them accordingly.
2. I was searching for a way to close the training frame, but was confused by the gray "Cancel"-button. Gray generally means disabled. I understand that you want to highlight the "Save and Next"-button. I would just change the "Cancel"-button into a link, to make it more obvious that you can click it.
That being said, you should fill in the email address when I need to create an account. Maybe make it greyed out since it is optional, but if I'm giving you access to do the import, I'm likely going to tie it to my email address.
Very nice app, and as with the parent, Google Reader import got me to try it.
However, I was put off by having to choose 64 feeds to "turn on" immediately after the import procedure. It meant my first experience of NewsBlur was the difficult and uninteresting bookkeeping task of figuring out which of my few hundred feeds I really, really cared about, and I pretty much disengaged from the app at that point. If I didn't have to make that choice right then, my first experience would instead have been simply my familiar feeds in a (rather nice) new UI - and at your current price point that would probably have been enough for me to buy a premium account.
(To give more context - I subscribe to lots of feeds, and rarely unsubscribe except from really noisy feeds, so I probably have a lot of dormant feeds sitting in Google Reader. I certainly don't recognise by name all of my feeds, or which ones I'd choose to keep if I had to choose 64. The autoselect by popularity is a good start, but for me isn't a good match with the feeds I actually get most value out of.)
I fully understand why you'd limit free accounts - particularly with your servers on fire :) - but I would suggest imposing the 64 feed limit after an initial trial period, rather than immediately, so people get that initial great experience.
Well, NewsBlur has auto-selected your 64 most popular feeds for you. All you have to do is hit the big green button that says "Turn Them On". This makes it obvious that the premium version is the way to go for your use case. It's very low-barrier and gives the user a conceptual model of what eventually has to happen for them to get more feeds.
Maybe I gave the wrong impression - I thought the UI for choosing feeds was well implemented, but having to choose is itself laborious, however pleasant the UI for doing it. (And as I mentioned above, "most popular" was not in my case a helpful heuristic.)
Put another way - if I had built this, the main reason I would have included Google Reader import in the free version would be so people could experience the value of my app without having to think - about what blogs to subscribe to, or what existing subscriptions to keep. Putting the paywall at this point in the first user experience removes that benefit in cases like mine. I might think about the benefits of your app and come back and upgrade - or I might forget - but you could have had me as an impulse buy.
I see your point crystal clear. But what happens when I don't make you choose a max of 64 feeds? I still have to fetch 100% of your other feeds, which could be in the hundreds. And I have to do this for every single user, which is currently in the thousands (ohh, my poor server). It just doesn't scale with my economics. If I were Google, sure, I could just throw money at more machines. As it is, I'm stuffing envelopes full of cash and sending it to Linode.
Ideally, I'd make all feeds free and updated every 5 minutes. Realistically, if I force you to choose, I can perform that much better for everybody else. It's a trade-off which I try to stack in my favor by giving the user an easy-to-use UI to activate different feeds as they choose under the standard account.
You could pay $12 a year and get unlimited feeds. That would make both of us very happy. :-)
First, just give them 64 feeds and tell them about it on some form of status bar. The "updating your feeds 63/64" progress bar, but maybe yellow with "We're only updating 64 of your feeds. [a href="why-premium/"]Why?[/a]"
Also, you can choose those 64 feeds to update as a union of what they have subscribed to and what you're currently already having to fetch, thereby making the necessary amount of work for each new user theoretically less in the best case, and no worse in the worst case.
I think you should "shoot first, ask forgiveness later". Select just the 64 most popular feeds for the user, and then put a big button on top of their feed list saying "Limited to 64 feeds... see everything for only $12", or some better copy along those lines.
That is, I should go "wow!" before "awww, free is only 64", instead of the other way around.
I agree, but I feel like there should be some fill in the middle of the page. When I first clicked on it, there was a big white space in the middle. Maybe a carousel of prefetched articles would work?
Interesting the fact that is definitely going again the trend of extracting content from websites to present it in a more readable format (instapaper, safari reader). I like the general interface, but I'm not a big fan of having to ready my rss crowded with ads and banners.
Nice thing is that you have the option to only read the Feed instead of the Original Site. By clicking on 'Feed', you get the same treatment as Google Reader. And NewsBlur persists your preference.
I am on a bit of an anti-Apple binge over the last week on HN, but I will point out that Safari Reader, like most Apple "innovations", occurred somewhere else first, in the Readability extension for Firefox and Chrome released almost a year before SR that thousands of people were using, which was being recommended non-stop on Reddit and HN. Can we please not let another mythology grow up around Safari Reader as with all the other technology which Apple appropriates.[1]
Very impressive, the UI stuff has clearly been thought about. At the same time, I found it a bit confusing. Also, landing straight in the app is cool, but it doesn't tell me how it's different, so I don't know why I would spend the time to figure it out.
It seems to have a lot of mysterious features like "instafetch" and "intelligence trainer" and stuff.
I guess I'm saying that my first impression is that it looks nice but complicated, like there's the promise of great stuff there but it seems like it would be hard to figure out. Compare with Google in the early days: great technology, crazy fast, but all that tech is hidden in the background and you don't see that, you just see the search box and the results.
Perhaps you've been too enamored with the cool/powerful features and haven't focused enough on keeping it simple?
Well, the site is very front-end JavaScript heavy, so most of my time is spent on JavaScript. For the most part, Django has been great. I grew out of many relational SQL-based models and moved most of the key-value type data to mongo. (Stories, classifiers, feed load histories are all in mongo. Feeds, subscriptions, user accounts are still in postgresql.) And the feed fetcher is all custom code to run distributed and consistent.
Django has stayed out of my way and that's been nice. I've used Django on other projects (specifically Storybird: http://storybird.com, an collaborative storytelling startup) and it's been the moon for us. Absolutely incredible. And I write RoR at DocumentCloud and it too has been phenomenal. They're both great. Choose the language, not the framework.
As a front-end engineer who used to work on Bloglines and helped create the Bloglines Beta, I say good luck! Your UI is very much like the Bloglines Beta and I realize this probably took a lot of work. Your challenges will probably be maintaining read state, security issues with trusting content in feeds, and keeping crawlers running so you don't miss items, searching content, handling archives, and more!
And of course, there's no money in this and the use of feeds are declining (so integrate Twitter and Facebook API).
I can tell you made some very conscious choices about how to position the premium version of this tool, but from my perspective I think you've erred too far on the side of caution. I would recommend 3 changes that would keep the spirit of "not in your face" premium pushing, and yet increase the take-rate on paid customers.
* Be up-front (on the landing page) about what you get for free and simply allude to the fact that a power-user option is available
* The dashboard needs to know if I'm using this enough to warrant a premium account. On a dashboard note: once I've clicked a feed, I need an easy path back to the dashboard.
* I need to be able to broadcast somehow that I'm using this. If I "like" a story, why can't I have a goo.gl shortened url to the story wrapped by NewsBlur post to my twitter/facebook??
That dashboard idea is priority #1 (next to finishing up the iphone app). But I want it to be a really swell dashboard. Graphs, relevant data, not just the server puking aggregation on you.
I'm hesitant to go overboard on branding and pushing premium. Notice the logos nowhere to be found when you use the site. It's like a desktop app for the web.
And I'm also hesitant to wrap content in URLs like that. I don't know, shouldn't the URL go directly to the original writer?
I don't use Google Reader, so I would love to have an option to import OPML files.
Edit: Scratch that. Found the option I was looking for. But, don't like that uploading an OPML resets existing feed list. It should ideally check for duplicates and add the new entries.
You can import OPML! When you login, just click on the green + button in the bottom-left. (There's a callout that says, "First things first..." right above it). There you can easily import OPML.
Yes, I know it should look for duplicates, with an option to erase. Unfortunately, this was too low on the priority list. But it's sitting there, waiting to be implemented sometime in the next few months.
Wow, awesome. This looks very very slick, and the Google Reader direct import is just...nice!
Bug report, though: A couple of my blogs that I chose to "turn on" only show up as their comments feed. If I go ahead and turn them on, and click on the feed itself in the left pane, I get the site (fair enough), but the list of items in the bottom pane then only shows up the comments feed items. And clicking on them takes me to the comments. weird.
1) An application announcement on HN that genuinely appealed to me.
2) A single guy beat Google at UX in one of their best web applications (exposing the original site design for the feed is a fantastic idea!)
3) I paid for an application before it's ready.
4) I felt this was all so important I logged in to post this comment. :)
Kudos, you've done a fantastic job here. I really hope your environs permit you to keep this project going - it's a great idea, and I'm in love with it already!
One of the major problems I have with Reader is that certain low output feeds (making up the majority of my subscriptions) just seem to blend together - I forget why I subscribed to them, and have to go back to the original web page to try and figure out who the person is. Exposing the original page by default solves this beautifully. :)
Great work!
Edit: something I forgot. Visual display of the site allows me to instantly demonstrate RSS to my mother. Google Reader confused the hell out of her. :)
You've done a really good job here. A news reader that lets me both rate news, theoretically learns my preferences over time, and lets me read the entire article without clicking more than once, is a serious package. (Why can't other readers get those things right?) My only problem is I'd prefer a native reader app to a web-based one, just because the major upside of using a reader is that it's constantly fetching and storing my feeds so there's no wait when I want to go read some news. I may check out your iPhone app though.
Handling iFrame busters? By using iFrame buster busters, of course!
When I detect an iframe buster, I prevent it from doing its job, but instead of continuing to show the original site, I switch over to the feed view. It complies with the wishes of the parent site, and I don't look like a bozo with such a glaring and awful bug. The feed view is still a great view. (It's the Google Reader view.)
That's awesome. That's perfect tradeoff between the wishes of the parent website and convenience for your users.
I'm assuming you fetch feeds using a queue.
* How do you add jobs to the queue ? Do you run a cron every n minutes to check which feeds need fetching and add them to queue ?
* When you add jobs do you check to see if a previous job for fetching the queue is still in the queue or do you use flags to mark feeds for which the jobs are in queue and eliminate duplicate jobs that way ?
Yes, I use celery/rabbitmq as a queue. But I don't bother checking for duplicates. If the feed is still in the queue by the time it comes back around for processing again, I let it run. That means that the queue is behind enough that it will even out in the wash.
For what it's worth, it's "diggbar"-style garbage that I added that to thwart, and as much as I like your interface, I'm going to be leaving the iframe-buster in place.
Edit: it looks like A List Apart's feed also breaks, likely for a similar reason.
And I'll have to admit, the more I'm using this, the more I'm liking it.
This UI looks very similar to MS outlook and I really it.However unlike outlook you cannot have all 3 panes vertically.On large screen ability display data into 3 vertical panes makes it more usable.
are you planning to add this feature sometime in future?
88 comments
[ 452 ms ] story [ 2762 ms ] threadNewsBlur is an RSS feed reader that tries to do two things very well:
1) Shows you the original site instead of a context-less feed. Read the original and NewsBlur marks the stories you've read.
2) Filter stories you either like or dislike. A three-stop slider goes between dislike, neutral, and like (red, yellow, and green). Training is super-easy and all click-based (as opposed to you having to writing out what you like in a site, NewsBlur asks you, semi-Hunch-style, your opinions on facets of the site).
I started working on NewsBlur to see if I could do it. Put the AI together with the back-end feed processing and fetching, along with the nifty front-end of the original site. This is one of those projects where I just kept pushing in all directions until I felt I had something good, not knowing if I could do it at all, but believing the entire time that I was able to complete the project.
If you have a project kicking around in the back of your mind, just dedicate enough weeks and months to get it done and done well. Provided you do that, you're almost guaranteed to succeed in some way. Those first few months of writing NewsBlur, I felt like a fraud, wishing I could just get threaded feed fetching to not break every few minutes. Even what I considered to be simple python gave me headaches. (Turns out threading is evil, use multiprocessing instead.)
NewsBlur is entirely open-source: http://github.com/samuelclay/NewsBlur/. The iPhone app, the front-end JavaScript, and the back-end Python. Pick it apart, but know that it is more a learning experience than a business. Turning it into something profitable may not be easy, but I only need 80 premium users to make that happen.
* Django: Web framework written in Python, used to serve all pages.
* Celery & RabbitMQ: Asynchronous queueing server, used to fetch and parse RSS feeds.
* MongoDB, Pymongo, & Mongoengine: Non-relational database, used to store stories, read stories, feed/page fetch histories, and proxied sites.
* PostgreSQL: Relational database, used to store feeds, subscriptions, and user accounts.
* jQuery: Cross-browser compliant JavaScript code. IE works without effort.
* Underscore.js: Functional programming for JavaScript. Indispensible.
* And many miscellaneous jQuery Plugins: Everything from resizable layouts, to progress bars, sortables, date handling, colors, corners, JSON, animations. See the complete list on NewsBlur's GitHub Javascript folder: http://github.com/samuelclay/NewsBlur/tree/master/media/js/
First bug I noticed, most easily visible with Daring Fireball: click on a story from the bottom list. If there's more than one story headline in view, it also marks the bottom one as read.
I noticed a small typo on http://samuelclay.com. I think you mean "Written with Django...".
1. Can you rate individual posts (I'm at work so I can't spend too much time on it right now) instead of just tags/authors? It wouldn't feel like that much work to me as a user. You could then just read the tags/authors from the rated posts and sort them accordingly.
2. I was searching for a way to close the training frame, but was confused by the gray "Cancel"-button. Gray generally means disabled. I understand that you want to highlight the "Save and Next"-button. I would just change the "Cancel"-button into a link, to make it more obvious that you can click it.
edit: I'm totally cloning this repo in case you change your mind :P
Well played sir, well played.
However, I was put off by having to choose 64 feeds to "turn on" immediately after the import procedure. It meant my first experience of NewsBlur was the difficult and uninteresting bookkeeping task of figuring out which of my few hundred feeds I really, really cared about, and I pretty much disengaged from the app at that point. If I didn't have to make that choice right then, my first experience would instead have been simply my familiar feeds in a (rather nice) new UI - and at your current price point that would probably have been enough for me to buy a premium account.
(To give more context - I subscribe to lots of feeds, and rarely unsubscribe except from really noisy feeds, so I probably have a lot of dormant feeds sitting in Google Reader. I certainly don't recognise by name all of my feeds, or which ones I'd choose to keep if I had to choose 64. The autoselect by popularity is a good start, but for me isn't a good match with the feeds I actually get most value out of.)
I fully understand why you'd limit free accounts - particularly with your servers on fire :) - but I would suggest imposing the 64 feed limit after an initial trial period, rather than immediately, so people get that initial great experience.
Put another way - if I had built this, the main reason I would have included Google Reader import in the free version would be so people could experience the value of my app without having to think - about what blogs to subscribe to, or what existing subscriptions to keep. Putting the paywall at this point in the first user experience removes that benefit in cases like mine. I might think about the benefits of your app and come back and upgrade - or I might forget - but you could have had me as an impulse buy.
Ideally, I'd make all feeds free and updated every 5 minutes. Realistically, if I force you to choose, I can perform that much better for everybody else. It's a trade-off which I try to stack in my favor by giving the user an easy-to-use UI to activate different feeds as they choose under the standard account.
You could pay $12 a year and get unlimited feeds. That would make both of us very happy. :-)
First, just give them 64 feeds and tell them about it on some form of status bar. The "updating your feeds 63/64" progress bar, but maybe yellow with "We're only updating 64 of your feeds. [a href="why-premium/"]Why?[/a]"
Also, you can choose those 64 feeds to update as a union of what they have subscribed to and what you're currently already having to fetch, thereby making the necessary amount of work for each new user theoretically less in the best case, and no worse in the worst case.
That is, I should go "wow!" before "awww, free is only 64", instead of the other way around.
Exactly what I was trying to say; you put it better than I did. (And your suggested fix would have worked for me too.)
Also, it uses Google Reader shortcuts + imports feeds. Making the transition easy.
Excellent way to present a product versus a strong competitor.
[1]http://news.ycombinator.com/item?id=1812967
It seems to have a lot of mysterious features like "instafetch" and "intelligence trainer" and stuff.
I guess I'm saying that my first impression is that it looks nice but complicated, like there's the promise of great stuff there but it seems like it would be hard to figure out. Compare with Google in the early days: great technology, crazy fast, but all that tech is hidden in the background and you don't see that, you just see the search box and the results.
Perhaps you've been too enamored with the cool/powerful features and haven't focused enough on keeping it simple?
But definitely awesome!
After the hailstorm of traffic dies down, I'll look into this error. Thanks for the report.
Django has stayed out of my way and that's been nice. I've used Django on other projects (specifically Storybird: http://storybird.com, an collaborative storytelling startup) and it's been the moon for us. Absolutely incredible. And I write RoR at DocumentCloud and it too has been phenomenal. They're both great. Choose the language, not the framework.
I love the reading experience you've designed, though.
And of course, there's no money in this and the use of feeds are declining (so integrate Twitter and Facebook API).
Anyways, very nice. :)
* Be up-front (on the landing page) about what you get for free and simply allude to the fact that a power-user option is available
* The dashboard needs to know if I'm using this enough to warrant a premium account. On a dashboard note: once I've clicked a feed, I need an easy path back to the dashboard.
* I need to be able to broadcast somehow that I'm using this. If I "like" a story, why can't I have a goo.gl shortened url to the story wrapped by NewsBlur post to my twitter/facebook??
Great work overall!!
I'm hesitant to go overboard on branding and pushing premium. Notice the logos nowhere to be found when you use the site. It's like a desktop app for the web.
And I'm also hesitant to wrap content in URLs like that. I don't know, shouldn't the URL go directly to the original writer?
- The whole point of GR is I can read the original site even when it's not available - works nicely with restrictive filters etc.
- It's slower than GR - for obvious reasons.
However, it's great to see somebody taking a stab at an alternative news UI. Good luck on your project.
Bug report, though: A couple of my blogs that I chose to "turn on" only show up as their comments feed. If I go ahead and turn them on, and click on the feed itself in the left pane, I get the site (fair enough), but the list of items in the bottom pane then only shows up the comments feed items. And clicking on them takes me to the comments. weird.
Haven't been able to figure out why. E.g. the XKCD blag (http://blag.xkcd.com) and Cheap Talk (http://cheeptalk.wordpress.com) feeds.
p.s: when this bug is fixed, and archiving support added, I'm totally premium-ing - hold me to it! Great work once again!
1) An application announcement on HN that genuinely appealed to me.
2) A single guy beat Google at UX in one of their best web applications (exposing the original site design for the feed is a fantastic idea!)
3) I paid for an application before it's ready.
4) I felt this was all so important I logged in to post this comment. :)
Kudos, you've done a fantastic job here. I really hope your environs permit you to keep this project going - it's a great idea, and I'm in love with it already!
One of the major problems I have with Reader is that certain low output feeds (making up the majority of my subscriptions) just seem to blend together - I forget why I subscribed to them, and have to go back to the original web page to try and figure out who the person is. Exposing the original page by default solves this beautifully. :)
Great work!
Edit: something I forgot. Visual display of the site allows me to instantly demonstrate RSS to my mother. Google Reader confused the hell out of her. :)
The iPhone app is not submitted yet. Feel free to play with it: http://github.com/samuelclay/NewsBlur/tree/master/media/ipho.... I still have a few screens to make, and a ton of bugs to sort through. Maybe January?
When I detect an iframe buster, I prevent it from doing its job, but instead of continuing to show the original site, I switch over to the feed view. It complies with the wishes of the parent site, and I don't look like a bozo with such a glaring and awful bug. The feed view is still a great view. (It's the Google Reader view.)
I'm assuming you fetch feeds using a queue.
* How do you add jobs to the queue ? Do you run a cron every n minutes to check which feeds need fetching and add them to queue ?
* When you add jobs do you check to see if a previous job for fetching the queue is still in the queue or do you use flags to mark feeds for which the jobs are in queue and eliminate duplicate jobs that way ?
In such cases, maybe you could doctor the links so that they'll open in a separate tab?
if(top.location!=self.location)top.location.replace(self.location);
For what it's worth, it's "diggbar"-style garbage that I added that to thwart, and as much as I like your interface, I'm going to be leaving the iframe-buster in place.
Edit: it looks like A List Apart's feed also breaks, likely for a similar reason.
And I'll have to admit, the more I'm using this, the more I'm liking it.