What do you think of my startup idea?
I'm calling it a discovery engine, where the user can enter the name of a specific piece of content they have in mind, or something they are generally interested in, and receive recommendations of new content from like-minded people.
Just like Wikipedia issued a call to all people interested in making an encyclopedia, this would issue a call to all early adopters to be recognized as authorities and trend-setters. Think The Tipping Point by Malcolm Gladwell, but taking place online, efficiently, and transparently.
The project calls for combining social-bookmarking and user-generated media with an algorithm that both aggregates similar collections of content into networks and makes recommendations of content based on the evolving network structure. The ranks of "influence" and "in the know" are measured against networks of users with similar collections of content. These two rankings incentivize users to continually post relevant content because they want to remain "influential" and "in the know" in front of the people that are genuinely interested in the same content. The majority of users coming to Topiat for recommendations receive relevant recommendations fueled by the work of those who are genuinely influential and in the know.
But unlike current recommendation systems, which are domain specific and treat an individual as the sum of all their preferences (e.g. Netflix), the discovery engine would allow users to create networks based on all types of content (any combination of music, products, images, videos, URLs etc.) and enables users to explore different interests they have with the ability to create multiple groups of content on their profile. Each group of content becomes aligned with similar groups of content, from which recommendations are generated and delivered to the user (e.g. my oldies music compared with users with similar tastes in oldies music, my surfing group compared with other users that think of surfer the same way I do).
I'm putting together a Y combinator funding proposal based on this basic idea and am looking for feedback before I send it in. If there are any developers that like the idea and want to know more, let me know. Additionally, if you are good with machine learning techniques (e.g. neural networks) and are interested, let me know.
43 comments
[ 4.7 ms ] story [ 193 ms ] threadideas like this are tough because they're only really useful once an incredible amount of content has been submitted. what tricks can you do to make the site be sticky to the first 100 users, when nothing yet is submitted? what are the first concrete things the user sees, or does, when they arrive at the site?
i think that's why these recommendation sites are only successful in niches (e.g. travel (tripadvisor), food/entertainment (yelp), movies (netflix)).
What you would tell your little sister about the site is "find more things you like, and if you're good at finding new content, be recognized for it."
Then, it looks for people who have voted/commented on the movie, and extrapolates a probability of you liking other movies that they have rated highly or poorly. If your rating history is similar to theirs, it rates their other movies with similar pieces higher. If your history is very different, or if they voted Jurassic Park very poorly, then the other items they voted highly are placed low on your recommendations.
I think that's awesome.
It isn't clear why "domain-specific" recommendation systems don't work within a group.
Note that it isn't obviously true that separating by group actually produces better recommendations. In fact, the (inadequate) evidence that I'm aware of indicates the exact opposite.
BTW - If you don't have the algorithms mentioned, how much do you think that you actually have? (I can imagine lots of wonderful things that would come from a personal transportation device that got 100mpg, but if I don't know how to make one....)
Based on what you have put into your collection of content, and collections others have made, the algorithm aggregates those similar people into the same network. People in the same network get content recommended to each other from their like-minded peers that they have not discovered on their own.
With regards to the feasability of such an algorithm, I've talked about it with many mathematicians and machine learning programmers and the wheel does not have to be reinvented for this application. The tools already exist, and just have to be customized and tweaked for this application.
Why isn't the netflix recommendation system useful for generating recommendations from "groups of content" labelled "movies"?
If it works for movies, why can't it combine "groups of content" labelled "science fiction"?
For example I can make a group of content called "new living room" and add all of the things that go into my new living room. This includes the music and movies that I have stored there, the type of TV I bought, the type of couch, stereo receiver, speakers, or even the paint on the wall. When someone searches for something within that collection, the system knows that someone else, somewhere, has combined that "thing" with the other "things" in the collection, so they get rated higher as being compatible.
The critical difference with the discovery engine is the idea of a group of content that users fill themselves with content based on criteria they see as relevant. Yes, a users aggregate preference composition is important, but what is more important is their set of preferences regarding a specific collection of content. This way, a user can be really into classical music, horror movies, and modern furniture, and get relevant recommendations for each interest, connecting with people who are most in the know regarding each interest.
The idea is that if you search for "sony SSK70ED," on the "discovery engine," it will show you what other people have paired with those speakers, such as receivers, furniture, and televisions. In a way those things are "similar" to the speakers because they complement them. Of course, the system shows you similar speakers first, but the complementing items are interesting results to have when you're searching for a specific item.
That's wrong. The only thing that is movie-specific about the netflix recommendation system is the preference data that it runs over. It doesn't know movies from eyebrows.
If netflix (the company) also collected preference information about music, the recommendation engine would predict music preferences. And, since it would have both music and movie info, it would use music prefs to recommend movies and the reverse, just as it uses movie prefs to predict movie prefs today.
Amazon's "users who bought {something} also bought" is an example of "doesn't know anything about the domain". (They have to tone it down to keep it from recommending "strange" things that are way out of category.)
Disclosure: I know the guy who implemented NetFlix first recommendation system and have written a collaborative filter myself. I know what I can do with the fact that we both like the Pogues and Chunky Monkey. I still don't see what I can do with how we group those preferences.
The way in which this site will allow users to group their preferences seems like a slight organizational difference when compared with other recommendation sites that use collaborative filters, but it has huge implications. This post is meant to give people a taste of what I'm starting to try and find others who are interested. I'd love to talk about any and all specifics and their implications especially if you are programmer. If you want to chat my AIM is rocksld3.
IRC is pretty popular, I don't see why your idea wouldn't be among the less BitchX-inclined internet crowd. ;-P
You'd have to be brutal with the 'influence' system, though, to keep the trolls out of the 'ponies and barbie dolls' group.
The killer feature, to me, of an idea like this would be something like musicovery.com, where overlapping things lead you down new paths.
1.) One of the main problems I am trying to solve is in your second sentence: "Once you find a chan you like..." It's time consuming to find new content you like, especially in a setting as nebulous (for most people) as IRC. With the discovery engine, all a user needs to do is rate recommendations based on what they are interested in, and they are immediately connected with groups of like-minded people and what they know. Fast and easy.
2.) In addition to connecting to groups of like-minded people, the user gets recommendations from those who are most influential and most in the know. All content is not equally desirable, and users using the discovery engine will only get recommended that content which has empirically shown is wanted by people that think like them (relevant).
>only get recommended that content which has empirically shown is wanted by people that think like them
Seems like an OK idea, but I think the site might suffer if this is taken too far and the groups just become echo chambers, walled off from each other entirely. I just think a site could be more popular if it focused on finding common links in disparate groups instead of isolating them based on common differences. Think of it like radio: in my music directory, my biggest artists are The Crystal Method (41 songs), NIN (112 songs), and Sarah McLachlan (85 songs). No single radio station, AM, FM, or XM, that I am aware of covers a range like that. Wouldn't it be cool to find a place that let you keep close to your core group, but at the same time gave you tips on what other groups thought was cool, just in case you agreed?
I think an uber-short pitch here is "A better recommendation engine for everything" which of course only makes sense if you know what a recommendation engine is.
Re: ability to get recommendation engines working better, see the Netflix $1M competition as an example of just how hard it is.
demonstrate the idea by showing ... build an app... Show me the demo
Now I see. Even a mocked up static demo is better than nothing. The Oct 11 date is plenty of time. Now I get what "your going to have to work real hard" comment means especially when you come up against fixed dates.
http://quarkfactor.com/topiat_example.png
Are you applying for YC this round?
Send me an AIM IM: quarkfactor
Better. Now make some more that demonstrate your key points and that will also help you start building. Wish you'd put this up first as it starts to put flesh to words (idea description).
What we need is someone who is amazingly good at neural networks, algorithm design, and advanced mathematical engineering with programming languages. These people don't grow on trees; they're extremely few and far between. But the main asset for this idea isn't the idea itself, or even-- to some extent-- the user interface, but the way the math works behind the scenes. That's our innovative secret, and it's the reason I'm excited.
or... i might have said that because i want to discourage you from using this great idea and save it for myself... hmm...
This idea is about finding new things based on things you already know. It's not possible to manipulate the Google "machine" to discover things that are similar to what you know you like.
I don't use Mahalo but I have no doubt he'll be able to monetize that kind of business and eventually sell out for a decent payday.
Could say the same about Mahalo?
These are concrete ideas:
Facebook, just like your silly school book, but online, and you can add friends
Google - search engine
Yelp - reviews of restaurants and businesses
las.fm - music recommandations from your friends
pandora - discover music according to your tastes
reddit/digg/news.yc -- recommending news and stories
Yours is not a tangible idea. Narrow it down to one sentence the main idea, and just one paragrapsh on how it will work, and then you have something more tangible.
Facebook: Digitize your real relationships
Google: Find websites based on keywords
Yelp: Find restaurants, share experiences about restaurants
Last.fm: Discover new music based on popularity
Pandora: Discover new music based on constituent pieces
reddig/Digg: Discover news based on popularity
This idea: Discover new things based on constituent pieces, popularity, and relationships.
Seems like a natural progression to me, even if it seems complicated in the description.
Facebook: Better contact with more people
Google: Answers to questions. Solutions to problems.
Last.fm: Good music
Pandora: Good music
reddig/Digg: Inspiration, quick and easy learning about tech, internet, politics
This idea: ?
For example, what's the benefit of getting movies that are similar to Jurassic Park? If you say "comfort and delight" I'd like to know how it's more comfortable than checking the movie with imdb and checking the recommendations. It has to be a bit more comfortable than the way people find movies today. Why would they check your service instead of just walking down to the video store and look at the selection?
How are those not benefits?
The benefit of finding movies that are similar to Jurassic Park is that you find new movies that you might enjoy. This is very obvious. What is the benefit of finding new music on iTunes/Pandora? It's the same thing! Of course it's for comfort and delight. That's what you're supposed to be pursuing in life.
IMDB does not recommend movies at all. If it does, it does it horrifically badly because it has never helped me before. IMDB is not a recommendation engine, it's simply a database. And they don't seem to want to innovate beyond that.
Video stores are enormous and movies are ordered alphabetically by genre. There is no way to know which movie is good, or which one you will like, because all you have is a promotional paragraph on the back and a pretty picture on the front. This is also very obvious to me.
Users are focused on end goals, not means. The end goal for Facebook is to keep in contact with friends, meet new people, organise events, and show off their photos.
Nobody wants to digitize anything. If FB initially told its users they've made a service to digitize their real relationships, they wouldn't be very popular.
The jargon used makes it sounds like a conversation with contemporaries, colleagues or investors.
The application's complexity spiraled out of my control. I still believe in the core concepts, but execution of this thing in particular is very tricky. Theoretically speaking it could be the end-all recommendation engine, and that is, from my own experience, almost as lofty as building a Turing Machine. Granted that you have user input, it probably only needs to be half as strong, but that is already very strong, and very difficult (at least for me).
Now logic is one thing. One the same level of difficulty is how to make it user friendly; for this app, regardless of how hard the AI is, the UI is probably half the battle.
I'm not sure what kind of app you have in mind specificially. dcurtis's mockup is real slick and to the point. My implementation was a lot more passive, but I suspect internally the structures share commonalities with your proposal. If you are interested we can discuss. I won't be applying with my implementation though, because this the kind of site where without 100k users, a VC won't even bother. And no matter how good it is, it is much harder to get traction today, hence higher uncertainty, higher burn.
Tags are a little messy, and perhaps overly specific, so you may need to cluster them and then do recommendations on that.
I'm not sure I understand how you plan on making 'influence' work, if someone bookmarks a link recommended to them do you simply increment all those who have bookmarked it before?
Please don't use the word incentivize, its terrible :)
If I were you, I would try to, at least for the time being, try to pare the idea down into something much smaller and more manageable. Once you have that done as a proof of concept, if it's still too small you can expand to other areas. Along the way, you will've learned a lot about what you'll need to execute on the bigger idea. Why not start with the movie suggester you mentioned later on in the thread?
My point being that I don't see anything in there that really evolves the social content sharing platform in any significant way.