Wow. Surely Yelp can pick up on a continuous, consistent pattern of negative reviews.
The hard part is staying one step ahead of less-legitimate accounts. There's always somebody trying to cheat the system, and stopping these users often results in a tradeoff.
> Surely Yelp can pick up on a continuous, consistent pattern of negative reviews.
But a continuous, consistent pattern of one-shot negative reviews is also not particularly difficult for a competing business (or anyone with a grudge) to reproduce.
This is a hard problem, and I don't think the credit check-style identity verification questions suggested in the post are going to solve it. For one thing, I think an overwhelming number of people would find it creepy. They're trying to tell people about a bad moving company, not get a car loan.
Putting more of your "valuable" online identity at stake seems likely to help. I hate to say it, but "Log in using your Facebook account" suddenly gives Yelp a lot more data to exploit to decide just how much trust they should put in your reviews.
Why would they find it creepy? You wouldn't need to say your using credit history. It would be as simple as asking the user a question "Please validate your a real person by answering these few questions".
As far as the user is concerned it's just a few questions and answers.
Have you seen the questions they use? They range from "which of these addresses have you previously lived at" to "which of these companies do you have a loan with" to "how much is your loan payment" among others. They're pretty creepy and made even more so by the fact that these are the same questions you answer to prove your identity to, say, the credit bureaus or your creditors. It's not the sort of stuff one should be answering lightly.
> Why would they find it creepy? You wouldn't need to say your using credit history. It would be as simple as asking the user a question "Please validate your a real person by answering these few questions".
As far as the user is concerned it's just a few questions and answers.
If the questions were about my credit history then this is a deal breaker.
There is no way a site like yelp can get me to authenticate by answering questions about my credit history and I suspect that the vast majority of people would agree.
Now, facebook credentials, that's a different story.
"just a few questions and answers" that I would be, frankly, shocked, that Yelp knows out of nowhere without the explanation that they're hitting my credit file... and then with that explanation, it's equally creepy that they want to hit my credit file for me to leave a review of a pizza place on the Internet.
As far as the user is concerned it’s just a few questions and answers.
Just a few questions and answers that could hint at all kinds of things including relationship status, orientation, race, financial status, etc. Imagine if some stranger on the street, or even the cashier at a trendy retail store, asks you about some detail they have no business knowing. You would find it creepy. Now imagine it's some random web site (which is all Yelp is to most people).
I agree it's easy to reproduce. Not an easy problem by any measure.
Putting in my credit card (or revealing any information I don't deem relevant to the company / form) is always an immediate deal breaker. For Yelp, a critical mass of reviews is needed, I'm not sure they can afford that.
The problem is, a continuous, consistent pattern of negative reviews can just as easily come from someone who has it out for a business.
I have a friend who is currently a target of such a vendetta. He got into an edit dispute with someone who was trying to censor stuff on a wiki. This other person responded by flooding listings of my friend's previous employer (which has been out of business for a few months) on nearly every review site out there with a continuous stream of utterly fabricated reviews. These reviews range from calling him a pedophile (easy enough to get deleted) to pretending to be a client who was screwed by his incompetence. I'm not sure there is any way to differentiate the latter kind of reviews from genuine negative experiences.
The proposed solution sounds incredibly easy to game - as a moving company, I could create a bunch of fake, one-off profiles with complaints about one of my competitors and destroy their Yelp reputation. Unfortunately, I don't think it's possible for an algorithm to detect the difference between a bunch of angry one-time posters and a coordinated reputation attack.
This is why I've gone off sites like Yelp for recommendations, and instead stick to places like Foursquare which rely much more heavily on the behaviour of the people that I know. Seeing that several of my friends have been somewhere more than once is a much stronger recommendation signal than a bunch of strangers giving something five stars.
You need a valid, fully qualified, unique credit history for each new account. How is that easy to game? Are you assuming that the moving company will get all their employees to to signup to Yelp and certify their credit history against fake sock puppet accounts? They maybe able to do it for a few accounts but pretty soon they would run out of real people to use up... in the mean time real reviews will be submitted by an endless pool of real people with valid credit history.
It will be quite easy to tell which reviews are real and which are not. At the very least it will make the company look questionable because 50% of reviews will be 5 star and 50% will be one star. The main issue is we need to get valid reviews out of the filter so that Yelp users can more easily make their mind up.
As it stands a lot of real reviews are hidden and a lot of fake reviews are exposed. Simply by exposing real reviews you level the playing field.
Say what? Yelp is pulling credit histories of people who join the site? If they are, that is way out of bounds for what users typically expect when joining an online review site. As a simple reviewer, I'm not engaging in any business with them, they're not offering me credit, and pulling a credit report on some anonymous internet user would not only be expensive but would require some serious disclosure (as having credit reports pulled actually impacts one's credit score). Not only that, but if they are doing it, it doesn't seem to be working very well.
Besides, I never supplied sufficient information for Yelp to connect me to my credit report without doing some seriously ethically questionable data mining.
Why do you believe that Yelp does this? If so, that would be more surprising to most users than the pay-for-play behavior we're seeing here.
Did you read the article before commenting? The article proposes that Yelp should provide people the option to use their credit rating to prove they are not a sockpuppet in return for Yelp displaying their review. It is only an option, credit ratings are only used because they are a pre-existing identification and authentication source and most importantly YELP IS NOT DOING THIS YET.
As a matter of fact I did, and you're right: I mis-read the parent comment as present tense vs. potential future tense. My apologies.
That said, I think some of the problems with the credit report solution still stand; namely that pulling credit histories has a cost (both to the reviewer and to Yelp). I would worry about what they would do with that information beyond merely authenticating me as a "real person" and I would expect that fewer people would be willing to review a local donut shop if doing so required a credit pull.
The hiding of reviews has made Yelp virtually useless. I like the identity verification idea. I implemented the Equifax version and it's very simple to code. I believe it cost us in the neighborhood of $1/verification. That may outweigh the value of a verified Yelp user though, but I have no idea of the Yelp economics.
It would be interesting to see a crowd-sourced recommendation site based only on positive recommendations. Something like a Net Promoter score to tell me where is good. i.e. you can only leave feedback for a place if you would actually recommend it to someone.
Every business can have bad days, and a negative review can put a prospective customer off even if actually 99% of the time that's a good business.
This isn't really a comparative recommendation site like yelp, the scenario would be something like a user saying what they want to do (like eat burgers on friday) then people would only recommend place to go. Kind of like Quora for places.
Eventually you could build a decent picture of business that are recommended multiple times to start making suggestions beyond the crowd sourced suggestions.
FWIW I watched the video (all the women have moustaches!) but am no more informed as to how it works. I also looked at the business sign up page but couldn't find any thing there to help work out the mechanics of the process. It looks like you integrate with FB closely?
Thanks for your feedback! I have gotten so little. :)
I was unsure of the etiquette here at HN about pimping one's own site in the body of a post. I saw that others chose to put their site URLs in their profiles, but very few were posting them in comment text.
As 'iamgilesbowkett mentions above, maybe it would be more informative to search over an aggregation of different opinions rather than hash mountains of conflicting reviews into a single byte digest.
For the consumer, our site offers a way to casually discover nearby businesses based on how well they are liked by specific circles of people. All activity is contextualized by location; the target of attention is geo-tagged by lat-long, not the user.
We try to reduce the effort required to browse through large amounts of business listing and business review data. The user should be able to quickly explore a specific area for businesses that match a search term, or are physically close to another (perhaps unrelated) business.
Please feel free to surf around using the business search.
On the site, business search itself is open to the world, but the core social parts require an account to participate. You can enter an object term plus a location. Search URLs can look like this: http://spotlikes.com/discover/places/noisebridge/san-francis...
Due to the founders' interest in Napa and Sonoma counties, we have lots of comments describing San Francisco North Bay Area small businesses. One entry point for browsing might be someplace like: http://spotlikes.com/business/benziger-family-winery-glen-el...
During after-release QA, sometimes I try to do a kind of major metropolitan area hopscotch, visiting all the largest US cities using the fewest clicks.
For the SMB owner, our site offers businesses a place to make a simple listing - a page where they can post their basic details (phone number, URL, logo, etc) - and then sit back and watch the users stream that small referral link out in their activities.
Yes, the site is tightly integrated with Facebook. User comments on business listings can flow automatically into social media sites. If you authorize it, your "likes" will flow onto your FB wall/timeline. We have similar Twitter and Foursquare integration.
We have probably overlooked all kinds of stuff. We move pretty fast. If you would like to call out anything in particular, I am keenly interested in the feedback.
Personally, I use the negative reviews much more than the positives. Especially on business reviews on Angie's List or book reviews on Amazon, I get a lot more info from the negatives. The company itself is usually able to tell me the positives. I don't let a few negatives rule out my business, but it allows me to compare more accurately between alternatives.
If I'm in planning mode and I don't know where I should go eat burgers on Friday, I don't want to know where _not_ to go, I want to know where I should go.
Of course, if I want to look a specific place or product, pos and neg reviews are useful.
But if you're in a "I'm moving across the country" mode, a negative review of a moving company may be worth much more than a positive review. (Possibly $2,000 more, in this case.)
Question based identity verification can be very painless.
For you, but not the vendor. See PayPal.
Question based identity verification processes are already used and supplied under the hood by agencies such as TransUnion, Equifax & Experian.
And that leads to friction. Yelp got it's massive user base from frictionless technology.
I don't like Yelp's business practices just as much as the next person, but this is a human interaction issue that isn't solved by a simple algorithm, nor is it solved purely by (mind you, costly) human being based committees.
True, Yelp got the filtering completely backwards in this instance. But, do you base any economic decision on two 5-star reviews from two Yelpers with 0 friends and 4 reviews combined?
Not everyone is a sophisticated Yelp user with an understanding of social networks. A lot of people will simply see two 5-star reviews, zero negative reviews, and think that looks like a safe bet.
Are you saying it's Yelp's responsibility to protect unsophisticated users from themselves? Also, I am not quite sure "a lot of people" will behave as you describe.
If one has to become a sophisticated Yelp user and do a substantial amount of detective work in order to determine if Yelp reviews are legit in order to get any value out of it (or at least not get actively misled by it), then what value does Yelp provide?
Worth mentioning is Yelp's long (alleged) history of using review filtering as a de facto racket to 'encourage' businesses to sign up for their paid services.
A company I used to freelance for found that after they stopped paying Yelp for a premium account, they started seeing more negative reviews and a number of their positive reviews were filtered out. They got almost daily phone calls from Yelp marketing types essentially suggesting that if they renewed their subscription, the negative reviews would 'disappear'.
This was a few years ago, so maybe things have changed since then. Either way, I wouldn't hold your breath for a better review verification system.
That's the only thing I use them for. It's more convenient to search for "bar" and see a list of actual boozeries, vs. searching Google Maps and finding metal manufacturers, and various businesses that happen to have "bar" in their name.
I use Yelp mostly for its metadata, such as restaurants' '$/$$/$$$' price categories, or a quick, easy-to-read menu sampling. I give the reviews approximately zero weight at this point in Yelp's evolution.
Especially complicated when you realize that their mapping system is completely broken for numerous addresses. I used Yelp to try to find a good local cafe to eat at in Irvine. Drove right to the spot indicated on the map and found an industrial complex. Had to look up the business name on Google Maps to find that it was about five miles South of where Yelp told me the cafe was.
Same thing happened again in Vegas, looking for a nice place to eat, find something that looks interesting and then see that Yelp places it in an industrial complex nearby. This time I don't expect to find the place, but I drive by and find myself somewhere in a dark alley full of closed-for-the-evening auto shops.
I'll still use Yelp to find the name of a business, but then I have to switch to Google Maps and do a reality check to actually see if the business is near by and how to get there. A real PITA.
"Worth mentioning is Yelp's long (alleged) history of using review filtering as a de facto racket to 'encourage' businesses to sign up for their paid services."
EXTREMELY worth mentioning. If you've yelped Yelp, so to speak, you know to take their recommendations with a grain of salt and then some. Caveat emptor.
While I'm on the topic, the idea of a single company managing reputation for any given space is silly beyond words. Downright moronic, even.
Reputation is an aggregation of multiple opinions. No aggregating entity can aggregate without their aggregating principle spinning the results. Yelp is notorious for a corrupt and indeed outright criminal aggregating principle -- "we'll suppress bad reviews if you pay us and good reviews if you don't" -- but the same is true for any aggregator, whether they're aggregating based on good principles or bad, because aggregation is a form of authorship. I mean look at Hacker News vs Reddit vs Digg vs Slashdot. They all aggregate, and each has a distinctive voice.
Because of this, yes, Yelp is a useful resource, but no, don't use it for a serious purchase without going on Google. If you use Google and Yelp, you're at least polling multiple sources, so you're doing something which can access the fundamental aggregating dynamic which powers reputation in the first place. A single point of failure for reputation is a contradiction in terms!
That made me think of an interesting idea. No problem is not solvable with some level of indirection!
A review aggregator website where the value of reviews from various sources is weighted by how reliable they are. For example, if there is Yelp, Facebook and Google, it can weight a restaurant 0.7 Yelp, 0.2 Facebook and 0.1 Google.
we do. if you use Google as well as Yelp, every other site on the Internet which mentions the business you are curious about effectively acts as a separate review site.
This is supposedly happening all the time, yet no one has managed to catch it on tape for some reason. I can't think of a credible explanation for the lack of hard evidence (and lawsuits).
This happened to a driving instructor I know just 2 weeks ago. He had a 4 star average rating on Yelp, receives a call from Yelp sales pitching some kind of a premium service. He declined. Within a week most of his positive reviews were suddenly filtered out dramatically reducing his score.
It's not worth mentioning. Almost every case has been a coincidence and they've gone out of their way to explain the review filter (which functions similarly to Google's PageRank algorithm and attempts to filter out spammy reviews using various factors).
Each court case related to this has been thrown out.
If you trust Google to be fair with PageRank then there hasn't been any evidence that should make you more concerned about Yelp.
Business owners complain about getting demoted in Google's PageRank algorithm quite a bit, too.
I don't recall ever hearing people receiving calls from Google salespeople who imply that a failure to buy advertising on Google may result in a reduction in PageRank, or that purchasing advertising may restore it.
> Business owners complain about getting demoted in Google's PageRank algorithm quite a bit, too.
Google rankings fluctuate over time, but it is never the result of a direct targeting of an individual company or URL by google, whereas there is ample evidence that Yelp directly targets individual businesses depending on whether or not they are paying customers.
There's no technical workaround for having good references from trusted source for something as important as movers, a dentist, a skilled tradesperson, a doctor, et al. (edit: I don't care, it's true. Even Angie's List isn't great)
I don't know if he's just being coy by saying this is an accident on Yelp's part.
The actual reviews are here: http://www.yelp.com/biz/royal-transportation-moving-and-stor.... It's pretty obvious there is something fishy about the unfiltered reviews. Each reviewer first left a one line review "Amazing!!! one of a kind..one of the best sushi place" and "the best of the best" before their very detailed, glowing review of the movers. If that's what it takes to fool the spam filter then something is very wrong. By contrast one of the filtered reviewers has ten measured, thorough reviews but his review has been banned for ToS violations.
Yelp really needs to disclose which vendors have a business relationship with them. They should also make the filtered reviews easier to see (place the link next to the unfiltered review count) and remove the damn CAPTCHA. I regularly try to check the filtered reviews but can't because the CAPTCHA is too difficult to read.
really really really makes me suspicious. Just did a check to make sure that google does not integrate yelp reviews (it does not) since that is what I mostly use.
also worth noting that "bob" liked the company so much he cross-posted his review to google.
Rather interesting to see his google review history, as long as we are speculating . . .
Gotcha. Damn, I see what you mean. If that's all it takes to game the system, it's amazing that their system isn't completely overwhelmed by fake reviews...
That's literally jawdropping for me. I would have thought it a fantastic business... and then to find that all the filtered reviews are 1 star... unreal.
Maybe a shamless plug, but I work on mobile apps at Fodor's Travel. We've been doing print travel guides for 75 years. We now have apps, in addition to books.
We're professional curators. That means every restaurant review (and shops, hotels, etc) has been written by an actual human professional who's opinion we hope you trust (we're a little more upscale than Lonely Planet, but we have our hip moments). So in a way, we're the anti-UGC. When you have crowds of people, all the ratings become a noisey 3-4 stars. Including Burger King. That's why I think Fodor's content is a good alternative. We're in the travel space, not local, but, it's a refreshing alternative.
Places have good days and bad days (or even hours). Your reviewers could get made. While the reviews may be spot on, they are a one point in time random sample.
The other problem is that not everyone is the same. For example I absolutely hate "attentive" staff in a restaurant (let me eat in peace and if I want something I'll ask for it), but want the check as quickly as possible on finishing while others like to slowly wind down. A regular reviewer probably won't mention either of these because that is normal to them.
The existence of multiple reviewers means one will be more like me and would mention these issues which would make me go somewhere else while others who enjoy that kind of experience would be more likely to come. This doesn't make anyone right or wrong, but does show how different reviewers help and why the lowest review scores have the most informative reviews.
As a counterpoint, I don't like the same movies Roger Ebert likes. But his opinions are consistent, and I know well enough where we are different, that I can usually estimate from the text of his reviews how well I'd like a given movie.
Reviewers may prefer attentive staff, but the good reviewers (which exist) know that not everyone shares their preferences. That's the difference between a good review and a bad review: you can translate a good review into your own viewpoint and decide for yourself.
When I think of noisy one star reviews, I mostly think of the one star review for a microprocessor where the guy didn't use a heatsink and the motherboard melted. There are plenty of folk who come across well in writing but are monstrous in person, I'd never expect to trust their restaurant reviews, and I might not be able to identify them either.
The question is, are you going to find that one reviewer who is like you in the sea of 4-star-giving morons?
I love reading Ebert's movie reviews. I often disagree with him, but he is a mostly fixed point. I can tell if I'm going to like something because, over time, I have developed an understanding of where his tastes and mine align and where they differ. This kind of thing is a lot harder to do online—it requires real work and attention.
In a way, we're disagreeing about where we want to have our statistical anomalies. Do we want the reliable reviewer caught on a bad day, or perhaps recognized by the staff? Or do we want the sea of one-time reviewers, a few of whom had a statistically unlikely bad or amazing experience?
Huh, interesting, and I'm surprised I hadn't heard of Fodor's. I work at http://www.oyster.com/ -- we're kind of the anti-TripAdvisor as all hotels are visited and photographed by our pro photographers and reviewers. We've found the same thing with UGC in the hotel space. For the popular hotels on TripAdvisor, there are 1000s of reviews, and the aggregate rating is always 3-4 stars. In contrast, on Oyster.com there's one review written with the same guidelines as all the other hotel reviews (as well as 100s of high-quality, non-staged photos).
Ya, I get that a lot.... I guess with less bookstores around, we have a lot less notoriety. You're right, high number of crowd ratings regress to some mediocre mean, less absolute dump 1 stars, less stellar 5 stars, i guess.
Man, I'm 30 and this makes me feel old. :) In the print travel guides market, it's one of the big names. It comes from the generation before the more backpacker-oriented guides (Lonely Planet, Rough Guides, etc.) and targets more of an upper-middle-class family vacation type of audience, but it's still fairly popular, within the print-travel-guides space.
It does seem that their move into digital guides is quite recent and incomplete, though.
I just want to say, my mother has been using Fodor's Forums for years now to help her with planning her trips. So thank you for creating such a useful resource!
Q: Internet solution doesn't work. How do we fix it?
A: Use the 1995 solution but because it's not 1995 anymore you'll use Facebook and Twitter to ask your friends which is still using "the Internet".
But not in the "back to square one" meaning, it is rather "next level of a spiral staircase". I believe there's still room for improvement (definitely there is over the '95 low tech and probably over what we have now) in the way we can combine internet and the '95 solution to find good <mover / restaurant / hairdresser ... etc>
Time will tell - lots of money goes into social, someone will do it right or at least we'll know what obstacles prevent this from happening.
No need for the 1995 solution, Yelp is just doing a very bad job at detecting bad reviews. It's usually rule #1 that in any type of "bad" user filtering that many people that are good users may not engage your site.
This point doesn't apply very well to this case. The problem is that there was a lack of verifiable people on Yelp posting negative reviews. Yelp has many more semi-verifiable people on it (i. e. people who've left multiple reviews) than the typical person has friends with experience with a given rarely-used service.
If in this case the problem were likely to be solved by acquaintances, it would also be likely to be solved by Yelp, under its assumption that people need to leave multiple reviews to be considered credible.
Yelp has been having problems for a long time, especially with small business owners. The site is killing small business owners, just do a Google search and you will find endless complaints about the site. Truth is Yelp cannot control the reviews, they can't guarantee them and their Filter... if it works only 50% of the time then it is worthless.
Disclaimer: I don't work for Yelp but I have created other user-ratings-based sites before.
It's not entirely fair to criticize Yelp for under-emphasizing negative reviews.
One of the big hurdles to overcome on a user ratings site is that people are most motivated to write reviews when they are outraged. There is a massive selection bias, and if nothing is done to alleviate this, your site runs the risk of becoming a soap box for ranting.
You work harder to get positive reviews than negative ones. If you receive 2 bad reviews and 2 good ones, that does not mean 50% of people are dissatisfied.
This should be the motivation for preferencing good reviews over lots of one-off bad reviews.
BUT: Yelp made the cardinal sin of "objective" ratings sites: their profit comes from the entities they are rating. Ergo everything on their site is questionable.
If this bias occurs across all businesses on Yelp, then still, wouldn't some places will be rated higher than others (even if "higher rated" comes to mean "has fewer bad reviews")?
Like you say though, Yelp is just making the playing field uneven in favor of those who pay to play, so none of this matters anyway.
"people are most motivated to write reviews when they are outraged. ...your site runs the risk of becoming a soap box for ranting."
There's a great big gap between letting all your angry customers (which hopefully you don't have a lot of) leave rants and creating sockpuppets so you have nothing but great reviews. The space between is the basics of word-of-mouth marketing: ask your happiest customers to talk about you.
No one is stopping American Eagle Movers from asking their happy customers to head on over to Yelp and leave them a review. They just don't seem to have any happy customers. So they have to fake it. And then Yelp steps in with their [seemingly] shady selection practices. Hence Justin's disastrous experience.
People are way more willing to take the time to write a bad review if they feel that something wrong was done to them than take the same time to leave a positive review, even if you ask them to.
The only one that will leave a good review will be Yelp users that rates everything.
And maybe people that you did them a big favor and they feel they have to repay you someway.
I'm not sure I can agree with that. Negative reviews are almost always far more helpful than positive reviews. Review sites should be careful about referencing positive reviews. The gold standard is Consumers Reports which is much more balanced than the average interent review site (where everything gets 4 stars) and thus much more helpful.
Speaking as an app developer, sometimes people get furious over things that are clearly stated before they download it. It really is easier to get complaints, I can see usage stats that show many more people using my apps daily.
So I read the complaint, and that thing that was clearly stated before is now reiterated. If it doesn't bother me, I go ahead and buy your app.
Good reviews are worthless to me as a consumer. What I want are details, good and/or bad, that I can use to evaluate my purchase ahead of time. "This is great!!!", "I would use it again!" are devoid of information.
If negative reviews are more informative, then I assure you, you should welcome them over a bunch of five-star "great!"s.
Besides being informative for me, the user, they should be valuable to you, the business owner. Nobody maintains an "attaboy" database, but you better keep track of bugs and fix them if you expect to grow or even survive in this economy.
I do agree that a lot of valid reviews are being thrown out with the bathwater. There should be a better mechanism to show all unfiltered reviews.
The thing that Yelp that makes it stand above other review sites is that they incentive positive reviews. Like Yelp or not, the idea of "Yelp Elite" is actually a great idea, because it is able to draw out a lot of medium to great reviews, stuff you would rarely see on other sites. When you incentive reviews, either good or bad, then you do tend to eliminate this "one-and-done" bias that tends to focus on the negative.
That being said, my wife is/was a Yelp Elite for several years with over 800 reviews, but quit this year because she thinks that Yelp as a whole has started to go downhill, and doesn't like the pretentiousness of most people on there now.
That's absurd. Google's not going to see the filtered reviews unless they start doing a lot more JavaScript execution than they've been doing, and why would Yelp care if a browser extension or spammers see filtered reviews?
Google's not going to see them because they have purposefully made them difficult to scrape...
I'd imagine Yelp would rather not have a "Yelp Unfiltered" extension made and their filtering becomes less important. Same for helping spammers automate their processes by checking up on their reviews.
It's a race to the bottom though, the spammers will eventually win.
> One of the big hurdles to overcome on a user ratings site is that people are most motivated to write reviews when they are outraged. There is a massive selection bias, and if nothing is done to alleviate this, your site runs the risk of becoming a soap box for ranting.
This is the risk we take listening to ANYTHING depending on voluntary responses. Burying negative reviews to attempt some intrinsic rating dismisses the value in actual user input, with the paternalistic view that the review aggregator somehow knows better. Who decides which and how many negative reviews to hide? You? How about letting the user figure it out, knowing that trusting any opinion is a risk and learning from experience to calibrate reviews he reads to himself? It's the best we can hope for without resorting to the review site, in effect, clobbering consensus views to get something it "knows to be more accurate." In that case the users no longer review, but the site, which is not a user, does.
> If you receive 2 bad reviews and 2 good ones, that does not mean 50% of people are dissatisfied.
You absolutely do not know this. It could be that 70% were dissatisfied. You can't assume this without taking the role of a reviewer, which you cannot legitimately do without experiencing the item being reviewed.
I have no pity for this guy...seriously he only went by Yelp's reviews and did no further research.
This article just proves how stupid consumers can be when making important decisions.
When I read the article I didn't see the guy mentioning looking up the company on the BBB or even simply Google'ing "[Company] scams" or "[Company] scammers".
Yes, Yelp could implement better solutions to protect consumers but ultimately the consumer needs to do adequate research before purchasing.
So there's a resource out there that claims to let the customer do a good portion of that research by aggregating real customer reviews and you berate him for using that research? Calling the guy "stupid" for believing a bunch of supposedly satisfied real customers while at least 10 extremely negative customer experiences were hidden from him is bullshit.
Honestly the author is over thinking his solution to the problem. The way to solve this problem is to give people the option of viewing reviews that Yelp believes are of dubious origin. Add another section to the reviews page, and let the users decide how to process the extra information but frame it as possibly being untrustworthy with an explanation of why it was tagged as untrustworthy.
They already allow you to click through to the reviews they have filtered out. How about having two average ratings, one for only "filtered in" reviews and one for all reviews?
And the font is about 10px and 30%K buried in the bottom corner. I can understand that would be desirable if the algorithm was more accurate in filtering bogus reviews, but that's clearly not the case so it just seems unscrupulous.
I have a friend that works as a veterinarian. Yelp called his business often to try to get an advertising deal. After he declined, many of his 5 star reviews disappeared from the site. After losing a lot of business, he signed up for the advertising. The next day all of the 5 star reviews returns. This was just a few months ago.
This has been confirmed from tons of people. How on earth has Yelp not been dismantled by the courts and their CEO put in jail? This is insane. Businesses now have to pay "protection" money to the Yelp mafia. WHY HAS YELP NOT BEEN SHUT DOWN? I FEEL LIKE I'M TAKING CRAZY PILLS!
Then he should sue. Every case brought against Yelp has been thrown out so far. I'd imagine that if this racket stuff happened, at least -one- case would come to fruition.
Over and over and over again I read negative articles about Yelp. These kind of places are ripe for scammers. These businesses know that you're randomly searching for a service, don't know you and don't plan on ever seeing you again. Yelp's system is, apparently, very easy to game. They've been accused of extortion. And yet, billion dollar valuation. It's laughable.
We're so out of touch that people are talking about how this space is ripe for disruption and a new startup. Umm, Yelp is an extremely young company. I guess once you're public you're seen as old hat? Pump and dump, and move on to the next thing. Unbelievable.
Is there a startup opportunity for an identity verification service whereby you do the ID verification once and then each service that wants to use it gets charged a smaller amount?
Example: consumer goes to idverificationsite.com, registers there for free- they ask out of wallet questions to verify identity, potentially do a minimal background check too.
You then use this login with Yelp, AirBnB, Getaround, etc.
Yelp, Getaround, Airbnb pay idverificationsite.com some small amount per verified person. Removes some of the friction of ID verification/spreads cost around.
Google actually used to do verification on people with Knol; you'd type in a credit card number and it would look up your name on the CC and match it against the name in your profile. I think that functionality got retired.
It would be nice if this service was available more globally.
There was an interesting article in Inc. magazine a while back about a company that tried that. They found out that no sites wanted to use the service, even though they saw how it might solve their problems, because it would immediately make their non-verified users less valuable.
There are at least two startup opportunities for something like this. One is the creation of the credential/verified identity certification and the second is the transmission/exchange of the credential. These are two very disparate set of competencies.
Several years ago, I was working with Honesty Online, doing exactly this identity verification startup prop and where it became very evident that this is really two businesses. Now we, in another startup, sigkat.com, are very early stage in the second business - creation/exchange/transmission of any sort of credentials so that vertical apps like identity verification can focus on their domain.
165 comments
[ 4.7 ms ] story [ 220 ms ] threadThe hard part is staying one step ahead of less-legitimate accounts. There's always somebody trying to cheat the system, and stopping these users often results in a tradeoff.
But a continuous, consistent pattern of one-shot negative reviews is also not particularly difficult for a competing business (or anyone with a grudge) to reproduce.
This is a hard problem, and I don't think the credit check-style identity verification questions suggested in the post are going to solve it. For one thing, I think an overwhelming number of people would find it creepy. They're trying to tell people about a bad moving company, not get a car loan.
Putting more of your "valuable" online identity at stake seems likely to help. I hate to say it, but "Log in using your Facebook account" suddenly gives Yelp a lot more data to exploit to decide just how much trust they should put in your reviews.
As far as the user is concerned it's just a few questions and answers.
If the questions were about my credit history then this is a deal breaker.
There is no way a site like yelp can get me to authenticate by answering questions about my credit history and I suspect that the vast majority of people would agree.
Now, facebook credentials, that's a different story.
Just a few questions and answers that could hint at all kinds of things including relationship status, orientation, race, financial status, etc. Imagine if some stranger on the street, or even the cashier at a trendy retail store, asks you about some detail they have no business knowing. You would find it creepy. Now imagine it's some random web site (which is all Yelp is to most people).
Putting in my credit card (or revealing any information I don't deem relevant to the company / form) is always an immediate deal breaker. For Yelp, a critical mass of reviews is needed, I'm not sure they can afford that.
I have a friend who is currently a target of such a vendetta. He got into an edit dispute with someone who was trying to censor stuff on a wiki. This other person responded by flooding listings of my friend's previous employer (which has been out of business for a few months) on nearly every review site out there with a continuous stream of utterly fabricated reviews. These reviews range from calling him a pedophile (easy enough to get deleted) to pretending to be a client who was screwed by his incompetence. I'm not sure there is any way to differentiate the latter kind of reviews from genuine negative experiences.
This is why I've gone off sites like Yelp for recommendations, and instead stick to places like Foursquare which rely much more heavily on the behaviour of the people that I know. Seeing that several of my friends have been somewhere more than once is a much stronger recommendation signal than a bunch of strangers giving something five stars.
It will be quite easy to tell which reviews are real and which are not. At the very least it will make the company look questionable because 50% of reviews will be 5 star and 50% will be one star. The main issue is we need to get valid reviews out of the filter so that Yelp users can more easily make their mind up.
As it stands a lot of real reviews are hidden and a lot of fake reviews are exposed. Simply by exposing real reviews you level the playing field.
Besides, I never supplied sufficient information for Yelp to connect me to my credit report without doing some seriously ethically questionable data mining.
Why do you believe that Yelp does this? If so, that would be more surprising to most users than the pay-for-play behavior we're seeing here.
That said, I think some of the problems with the credit report solution still stand; namely that pulling credit histories has a cost (both to the reviewer and to Yelp). I would worry about what they would do with that information beyond merely authenticating me as a "real person" and I would expect that fewer people would be willing to review a local donut shop if doing so required a credit pull.
Every business can have bad days, and a negative review can put a prospective customer off even if actually 99% of the time that's a good business.
This isn't really a comparative recommendation site like yelp, the scenario would be something like a user saying what they want to do (like eat burgers on friday) then people would only recommend place to go. Kind of like Quora for places.
Eventually you could build a decent picture of business that are recommended multiple times to start making suggestions beyond the crowd sourced suggestions.
Would you mind looking at (the web site listed in my profile) and describing what is still missing?
edit: removed link
FWIW I watched the video (all the women have moustaches!) but am no more informed as to how it works. I also looked at the business sign up page but couldn't find any thing there to help work out the mechanics of the process. It looks like you integrate with FB closely?
I was unsure of the etiquette here at HN about pimping one's own site in the body of a post. I saw that others chose to put their site URLs in their profiles, but very few were posting them in comment text.
As 'iamgilesbowkett mentions above, maybe it would be more informative to search over an aggregation of different opinions rather than hash mountains of conflicting reviews into a single byte digest.
For the consumer, our site offers a way to casually discover nearby businesses based on how well they are liked by specific circles of people. All activity is contextualized by location; the target of attention is geo-tagged by lat-long, not the user.
We try to reduce the effort required to browse through large amounts of business listing and business review data. The user should be able to quickly explore a specific area for businesses that match a search term, or are physically close to another (perhaps unrelated) business.
Please feel free to surf around using the business search.
On the site, business search itself is open to the world, but the core social parts require an account to participate. You can enter an object term plus a location. Search URLs can look like this: http://spotlikes.com/discover/places/noisebridge/san-francis...
Due to the founders' interest in Napa and Sonoma counties, we have lots of comments describing San Francisco North Bay Area small businesses. One entry point for browsing might be someplace like: http://spotlikes.com/business/benziger-family-winery-glen-el...
During after-release QA, sometimes I try to do a kind of major metropolitan area hopscotch, visiting all the largest US cities using the fewest clicks.
For the SMB owner, our site offers businesses a place to make a simple listing - a page where they can post their basic details (phone number, URL, logo, etc) - and then sit back and watch the users stream that small referral link out in their activities.
Yes, the site is tightly integrated with Facebook. User comments on business listings can flow automatically into social media sites. If you authorize it, your "likes" will flow onto your FB wall/timeline. We have similar Twitter and Foursquare integration.
We have probably overlooked all kinds of stuff. We move pretty fast. If you would like to call out anything in particular, I am keenly interested in the feedback.
Of course, if I want to look a specific place or product, pos and neg reviews are useful.
For you, but not the vendor. See PayPal.
Question based identity verification processes are already used and supplied under the hood by agencies such as TransUnion, Equifax & Experian.
And that leads to friction. Yelp got it's massive user base from frictionless technology.
I don't like Yelp's business practices just as much as the next person, but this is a human interaction issue that isn't solved by a simple algorithm, nor is it solved purely by (mind you, costly) human being based committees.
If you guys made a competitor to Yelp, how would you improve on it?
I'm not sure how it compares to other sites, but you can see the URL in my profile.
A company I used to freelance for found that after they stopped paying Yelp for a premium account, they started seeing more negative reviews and a number of their positive reviews were filtered out. They got almost daily phone calls from Yelp marketing types essentially suggesting that if they renewed their subscription, the negative reviews would 'disappear'.
This was a few years ago, so maybe things have changed since then. Either way, I wouldn't hold your breath for a better review verification system.
But if I don't trust the reviews, Yelp effectively degrades into a complicated remix of Google Maps.
Same thing happened again in Vegas, looking for a nice place to eat, find something that looks interesting and then see that Yelp places it in an industrial complex nearby. This time I don't expect to find the place, but I drive by and find myself somewhere in a dark alley full of closed-for-the-evening auto shops.
I'll still use Yelp to find the name of a business, but then I have to switch to Google Maps and do a reality check to actually see if the business is near by and how to get there. A real PITA.
EXTREMELY worth mentioning. If you've yelped Yelp, so to speak, you know to take their recommendations with a grain of salt and then some. Caveat emptor.
While I'm on the topic, the idea of a single company managing reputation for any given space is silly beyond words. Downright moronic, even.
Reputation is an aggregation of multiple opinions. No aggregating entity can aggregate without their aggregating principle spinning the results. Yelp is notorious for a corrupt and indeed outright criminal aggregating principle -- "we'll suppress bad reviews if you pay us and good reviews if you don't" -- but the same is true for any aggregator, whether they're aggregating based on good principles or bad, because aggregation is a form of authorship. I mean look at Hacker News vs Reddit vs Digg vs Slashdot. They all aggregate, and each has a distinctive voice.
Because of this, yes, Yelp is a useful resource, but no, don't use it for a serious purchase without going on Google. If you use Google and Yelp, you're at least polling multiple sources, so you're doing something which can access the fundamental aggregating dynamic which powers reputation in the first place. A single point of failure for reputation is a contradiction in terms!
A review aggregator website where the value of reviews from various sources is weighted by how reliable they are. For example, if there is Yelp, Facebook and Google, it can weight a restaurant 0.7 Yelp, 0.2 Facebook and 0.1 Google.
we do. if you use Google as well as Yelp, every other site on the Internet which mentions the business you are curious about effectively acts as a separate review site.
http://abcnews.go.com/Blotter/business-bureau-best-ratings-m...
Each court case related to this has been thrown out.
If you trust Google to be fair with PageRank then there hasn't been any evidence that should make you more concerned about Yelp.
Business owners complain about getting demoted in Google's PageRank algorithm quite a bit, too.
> Business owners complain about getting demoted in Google's PageRank algorithm quite a bit, too.
Google rankings fluctuate over time, but it is never the result of a direct targeting of an individual company or URL by google, whereas there is ample evidence that Yelp directly targets individual businesses depending on whether or not they are paying customers.
Can you link to any of this "ample evidence"? Preferably not just allegations from business owners who don't like their reviews.
Yelp's filtered reviews fluctuate as well. Reviews that are filtered can be unfiltered if the system decides the reviews aren't actually spam.
http://chicago.cbslocal.com/2012/02/27/2-investigators-busin...
Wait, Yelp gets money from the businesses they list? Yeah, that's not a conflict of interest.
The actual reviews are here: http://www.yelp.com/biz/royal-transportation-moving-and-stor.... It's pretty obvious there is something fishy about the unfiltered reviews. Each reviewer first left a one line review "Amazing!!! one of a kind..one of the best sushi place" and "the best of the best" before their very detailed, glowing review of the movers. If that's what it takes to fool the spam filter then something is very wrong. By contrast one of the filtered reviewers has ten measured, thorough reviews but his review has been banned for ToS violations.
Yelp really needs to disclose which vendors have a business relationship with them. They should also make the filtered reviews easier to see (place the link next to the unfiltered review count) and remove the damn CAPTCHA. I regularly try to check the filtered reviews but can't because the CAPTCHA is too difficult to read.
really really really makes me suspicious. Just did a check to make sure that google does not integrate yelp reviews (it does not) since that is what I mostly use.
also worth noting that "bob" liked the company so much he cross-posted his review to google.
Rather interesting to see his google review history, as long as we are speculating . . .
http://maps.google.com/maps/user?uid=213714492309658467573&#...
_____
edit: also a bit of fun to compare these two reviews:
http://www.mymovingreviews.com/movingreviews/myself-and-my-f...
http://www.yelp.com/biz/royal-transportation-moving-and-stor...
Q: How can you verify the quality of a < mover / restaurant / hairdresser / babysitter / dentist / chiropractor / handyman / etc.>?
A: Ask your < friend / relative / neighbor / workmate / schoolmate / etc.>.
2009
Q: Same question.
A: Don't need no stinkin' people. We have the internets!
2012
Q: Internet solution doesn't work. How do we fix it?
A: Use the 1995 solution.
If a friend recommends Avengers Assemble, I go see it. If I end up at the cinema out of boredem, I'll let imdb help me.
I want some website to come along which solves this for me, and neither Facebook nor Hunch nor IMDB do really.
We're professional curators. That means every restaurant review (and shops, hotels, etc) has been written by an actual human professional who's opinion we hope you trust (we're a little more upscale than Lonely Planet, but we have our hip moments). So in a way, we're the anti-UGC. When you have crowds of people, all the ratings become a noisey 3-4 stars. Including Burger King. That's why I think Fodor's content is a good alternative. We're in the travel space, not local, but, it's a refreshing alternative.
Oh, and all our apps are FREE.
http://www.fodors.com/mobile-apps/
The other problem is that not everyone is the same. For example I absolutely hate "attentive" staff in a restaurant (let me eat in peace and if I want something I'll ask for it), but want the check as quickly as possible on finishing while others like to slowly wind down. A regular reviewer probably won't mention either of these because that is normal to them.
The existence of multiple reviewers means one will be more like me and would mention these issues which would make me go somewhere else while others who enjoy that kind of experience would be more likely to come. This doesn't make anyone right or wrong, but does show how different reviewers help and why the lowest review scores have the most informative reviews.
Reviewers may prefer attentive staff, but the good reviewers (which exist) know that not everyone shares their preferences. That's the difference between a good review and a bad review: you can translate a good review into your own viewpoint and decide for yourself.
When I think of noisy one star reviews, I mostly think of the one star review for a microprocessor where the guy didn't use a heatsink and the motherboard melted. There are plenty of folk who come across well in writing but are monstrous in person, I'd never expect to trust their restaurant reviews, and I might not be able to identify them either.
I love reading Ebert's movie reviews. I often disagree with him, but he is a mostly fixed point. I can tell if I'm going to like something because, over time, I have developed an understanding of where his tastes and mine align and where they differ. This kind of thing is a lot harder to do online—it requires real work and attention.
In a way, we're disagreeing about where we want to have our statistical anomalies. Do we want the reliable reviewer caught on a bad day, or perhaps recognized by the staff? Or do we want the sea of one-time reviewers, a few of whom had a statistically unlikely bad or amazing experience?
Man, I'm 30 and this makes me feel old. :) In the print travel guides market, it's one of the big names. It comes from the generation before the more backpacker-oriented guides (Lonely Planet, Rough Guides, etc.) and targets more of an upper-middle-class family vacation type of audience, but it's still fairly popular, within the print-travel-guides space.
It does seem that their move into digital guides is quite recent and incomplete, though.
Not for Nokia, they aren't. (AUD 6.60 each)
http://store.ovi.com/search?q=Fodor%27s+Avante
Q: Internet solution doesn't work. How do we fix it? A: Use the 1995 solution but because it's not 1995 anymore you'll use Facebook and Twitter to ask your friends which is still using "the Internet".
But that kind of blows up your point/argument.
But not in the "back to square one" meaning, it is rather "next level of a spiral staircase". I believe there's still room for improvement (definitely there is over the '95 low tech and probably over what we have now) in the way we can combine internet and the '95 solution to find good <mover / restaurant / hairdresser ... etc>
Time will tell - lots of money goes into social, someone will do it right or at least we'll know what obstacles prevent this from happening.
If in this case the problem were likely to be solved by acquaintances, it would also be likely to be solved by Yelp, under its assumption that people need to leave multiple reviews to be considered credible.
It's not entirely fair to criticize Yelp for under-emphasizing negative reviews.
One of the big hurdles to overcome on a user ratings site is that people are most motivated to write reviews when they are outraged. There is a massive selection bias, and if nothing is done to alleviate this, your site runs the risk of becoming a soap box for ranting.
You work harder to get positive reviews than negative ones. If you receive 2 bad reviews and 2 good ones, that does not mean 50% of people are dissatisfied.
This should be the motivation for preferencing good reviews over lots of one-off bad reviews.
BUT: Yelp made the cardinal sin of "objective" ratings sites: their profit comes from the entities they are rating. Ergo everything on their site is questionable.
Like you say though, Yelp is just making the playing field uneven in favor of those who pay to play, so none of this matters anyway.
There's a great big gap between letting all your angry customers (which hopefully you don't have a lot of) leave rants and creating sockpuppets so you have nothing but great reviews. The space between is the basics of word-of-mouth marketing: ask your happiest customers to talk about you.
No one is stopping American Eagle Movers from asking their happy customers to head on over to Yelp and leave them a review. They just don't seem to have any happy customers. So they have to fake it. And then Yelp steps in with their [seemingly] shady selection practices. Hence Justin's disastrous experience.
The only one that will leave a good review will be Yelp users that rates everything.
And maybe people that you did them a big favor and they feel they have to repay you someway.
Good reviews are worthless to me as a consumer. What I want are details, good and/or bad, that I can use to evaluate my purchase ahead of time. "This is great!!!", "I would use it again!" are devoid of information.
If negative reviews are more informative, then I assure you, you should welcome them over a bunch of five-star "great!"s.
Besides being informative for me, the user, they should be valuable to you, the business owner. Nobody maintains an "attaboy" database, but you better keep track of bugs and fix them if you expect to grow or even survive in this economy.
Legitimate complaints are welcome, I never implied they were not.
The thing that Yelp that makes it stand above other review sites is that they incentive positive reviews. Like Yelp or not, the idea of "Yelp Elite" is actually a great idea, because it is able to draw out a lot of medium to great reviews, stuff you would rarely see on other sites. When you incentive reviews, either good or bad, then you do tend to eliminate this "one-and-done" bias that tends to focus on the negative.
That being said, my wife is/was a Yelp Elite for several years with over 800 reviews, but quit this year because she thinks that Yelp as a whole has started to go downhill, and doesn't like the pretentiousness of most people on there now.
What's tricky about it now? Click "X filtered" reviews and you can read to your heart's content.
I'd imagine Yelp would rather not have a "Yelp Unfiltered" extension made and their filtering becomes less important. Same for helping spammers automate their processes by checking up on their reviews.
It's a race to the bottom though, the spammers will eventually win.
This is the risk we take listening to ANYTHING depending on voluntary responses. Burying negative reviews to attempt some intrinsic rating dismisses the value in actual user input, with the paternalistic view that the review aggregator somehow knows better. Who decides which and how many negative reviews to hide? You? How about letting the user figure it out, knowing that trusting any opinion is a risk and learning from experience to calibrate reviews he reads to himself? It's the best we can hope for without resorting to the review site, in effect, clobbering consensus views to get something it "knows to be more accurate." In that case the users no longer review, but the site, which is not a user, does.
> If you receive 2 bad reviews and 2 good ones, that does not mean 50% of people are dissatisfied.
You absolutely do not know this. It could be that 70% were dissatisfied. You can't assume this without taking the role of a reviewer, which you cannot legitimately do without experiencing the item being reviewed.
This article just proves how stupid consumers can be when making important decisions.
When I read the article I didn't see the guy mentioning looking up the company on the BBB or even simply Google'ing "[Company] scams" or "[Company] scammers".
Yes, Yelp could implement better solutions to protect consumers but ultimately the consumer needs to do adequate research before purchasing.
If you're not a customer, you're the product.
I find that to be the most shocking statement in the article.
We're so out of touch that people are talking about how this space is ripe for disruption and a new startup. Umm, Yelp is an extremely young company. I guess once you're public you're seen as old hat? Pump and dump, and move on to the next thing. Unbelievable.
Example: consumer goes to idverificationsite.com, registers there for free- they ask out of wallet questions to verify identity, potentially do a minimal background check too.
You then use this login with Yelp, AirBnB, Getaround, etc.
Yelp, Getaround, Airbnb pay idverificationsite.com some small amount per verified person. Removes some of the friction of ID verification/spreads cost around.
Sidenote: the BBB is the worst offender of these protection rackets - http://feefighters.com/blog/bbb-accreditation/
Also - I've just decided to start posting reviews to Google rather than Yelp, they lost me as a product after reading this thread.
It would be nice if this service was available more globally.
http://www.inc.com/magazine/201112/case-study-josh-levy-and-...
Several years ago, I was working with Honesty Online, doing exactly this identity verification startup prop and where it became very evident that this is really two businesses. Now we, in another startup, sigkat.com, are very early stage in the second business - creation/exchange/transmission of any sort of credentials so that vertical apps like identity verification can focus on their domain.