HN Explorer

Ask YC: Built a great social media site, but need a recommendation engine. Any ideas?

12 points by grag ↗ HN
I've developed a website / Firefox toolbar that allows people to surf the web by topic or within specific domains. I think it has huge potential and that I'm on to something big here, but its usefulness relies on having a powerful recommendation engine behind it, something I cannot build myself. It's similar in many ways to stumbleupon (at least in regards to how the rec engine needs to operate. new content is introduced by users who fave websites from their toolbar and should be introduced to more and more users as it gains popularity within a period of time).

I've talked to a few recommendation engine providers (loomia, strands, etc). Strands is the only company that may be able to provide a back-end recommendation service that could work for my service (great API, content agnostic, ability to filter by item tags).

But still, I'm not sure a 3rd party service like this could ever provide the flexibility that I would eventually want (For instance, how could I control the rate at which new content is spread and eventually stops being recommended? how would I address potential issues of gaming?).

I'm not that knowledgeable about how rec engines operate. If anyone here understands the technical details regarding rec engines (or has experience building them) I would love to hear what you think. A third party system could potentially allow me to launch this thing very soon and would alleviate many scaling issues. But maybe it would be smarter to try to get a team together and build it from scratch so that I have complete flexibility.

10 comments

[ 9.2 ms ] story [ 186 ms ] thread
tortilla
I just read "Designing for the Social Web" and there's a chapter on "Collective Intelligence" (Chapter 6) http://www.amazon.com/Designing-Social-Voices-That-Matter/dp...

It doesn't get into any technical implementation details but does cover design, strategy, and issues you should consider.

You might want to browse that chapter at your favorite bookstore.

Also check out "Programming Collective Intelligence" by Oreilly. I haven't read it yet, but it is sitting on my shelf.

ojbyrne
I'll second the "Programming Collective Intelligence" recommendation. It's a good introduction to the field.
shafqat
Good question - I'd also like to know if anyone has had any success in building or interfacing with one of these external services like Strands.
keefe
It turns out this is a really hard problem. If you haven't seen it already check out http://www.netflixprize.com/ to set your expectations appropriately.
rcraswell
In Netflix's case it's not that baseline recommendation (whether based on collaborative filtering or something else) is such a hard problem, it's that getting something statistically better than what they have is so hard. Decent recommendation based on CF was a well-understood problem over ten years ago.
gaika
We've build our own, contact us if you're interested (link in the profile).
gunderson
You don't have to do anything too sophisticated to get some great results for your users.

Check out this article and collaborative filtering library:

http://www.igvita.com/2007/01/15/svd-recommendation-system-i...

I actually want to play with this more, so if anyone wants to create a google group to share implementation approaches, etc., just reply to this and I'll create it.

zacharye
I'm sure you'll find many a great solution here but for what it's worth, I've dealt with the Strands guys and they have a terrific team. I'm sure they'll try to work out your issues with you if they see potential...