"Businesses that try to weasel their way to a higher rating (paying off grumpy clients, for instance) have their Yelp pages branded with a red warning."
If you really hate a place, just give it some scam-looking reviews and watch it burn.
I've been a heavy Amazon shopper for years and there is no doubt that there has been a huge spike in fake reviews recently, and they all seem to be marked as Verified Purchase to the point that the Verified Purchase tag now makes me more suspicious that it's a fake.
I have noticed a proliferation of "I received this product for {free | discount | etc} in exchange for my unbiased review" notices tagging reviews on what seems like every product I look for on Amazon these days. It's a shame the reviews are so gamified now that they are losing their usefulness, at least to those that realize the scam.
The last few times I had to buy a product, I just googled stuff like "site:reddit.com usb charger" and read the comments and suggestions of actual people; much better than the fake stuff from amazon.
I wonder how much it will take until someone starts a botnet discussing products on reddit..
I agree, Amazon reviews used to be useful, but are now largely useless. I used to find the reviews really helpful, to the point where I was much more likely to buy something with only a few minutes research on Amazon than on any other site (although amazon's money back guarantee helped too). Now though, I usually won't buy there without doing a lot more research, which in itself imposes a cost on me, and reduces the amount I buy.
Some possible markers of fake reviews I look out for, completely anecdotal:
- products that have high hundreds or thousands of reviews, with almost all 5's and a few 1's.
- products with a large disparity between the Amazon reviews and sites that (I think) are much less likely to be targeted for fake reviews, e.g Walmart.com or bestbuy.com
It should be no harder to figure out an algorithm to stop this than to stop spam email, which makes me think sites like Amazon are not really motivated to stop the practice.
> I agree, Amazon reviews used to be useful, but are now largely useless.
If you're buying tech stuff, I find that the following approach works well:
* Find the thing you want on Newegg.
* Read a few of the five-star reviews.
* Read a few of the one-star reviews.
* Read most of the four and three-star reviews. You can use these reviews to get a sense for the actual faults with the product, and to see if the inevitable DOA complaints in the one-star reviews are a real problem, or just bad luck.
* If the product checks out, buy it from wherever you like.
It's pretty ironic, really, that Yelp, a company famous for extorting small businesses for advertising subscriptions (with the threat of hiding positive reviews), is complaining about the validity of reviews on their site.
Their whole business model, afaict, is a shady scam against people who are too busy running their business to learn how to do SEO.
+1 for this. I know a ton of small restaurant/bar owners. None of them like Yelp. All of them have been called and offered many versions of the "we can make your bad reviews go away if you just fork over this subscription fee" scam, even if not using those exact words.
I currently automatically assume that reviews online are fake. The thing about paid reviews is that I don't have a problem with them in principal. I would even like to see "trusted" groups or even companies who are paid to give thorough reviews and post them to places like Amazon and Yelp so that you get something other than "sux" "cover didn't look right" or "food wasn't spicy" that is utterly unhelpful and throws off the score coming from people who actually give fair reviews to products.
There would obviously be a conflict of interest for the people doing the reviews which is why I think it would make sense for companies to pay an intermediary who pays another firm to do the actual review. Companies that give reviews could earn "trusted" status through Amazon or Yelp (even though the are the mafia) etc.
How accurate are Amazon's and Yelp's fake review detection algorithms?
* How many false positives are there? How many businesses do they unfairly penalize (and how many reviewers do they humiliate)? For example, perhaps some behavior trait common to a location or social class is misinterpreted as a fake review.
* How many false negatives are there? How many fake reviews are given extra legitimacy in the eyes of users because the algorithm gave them a pass? Maybe the algorithm weeds out only the bad fake reviewers, or a certain kind of fake reviewers. Perhaps a cottage industry of techniques to game the algorithms will arise, like SEO. I do know that decent shills know how to fool amatuer slueths; heck, I could do it: Write something long, with specifics, add a little balance to look reasonable, mirror what some others say, etc.
I'd be interested to know how Yelp and Amazon can test their algorithms. They would need a large set of proven fake reviews, and a representative set would be much more valuable. Where does one find those?
Why can't Amazon or Yelp, etc. weigh the trustworthiness of their reviewers? Newcomers low till proven otherwise, long-time reviewers who prove trustworthy higher. Trust low reviews more than high reviews, except for trustworthy reviewers. Punish reviewers with lower weight when they break the trust.
How to know what's trustworthy? That's more difficult to establish, but someone like an Amazon has the capacity to build something like this. So would Yelp and they both have incentives to make the reviews published on their platforms believable -else we'll all believe it's just spammy astroturfing.
I'm sure they do this to some extent, but I'd guess that prolific users are rare, and the vast majority of reviews are written by users who have only written a small number of other reviews. So the danger is that downweighting or hiding reviews from newcomers might mean throwing away the vast majority of your data, and also discouraging new users from contributing.
They could possibly incentivize (honest) reviewers or new reviewers with a next day prime upgrades on a future purchase, for example.
Still, a few honest reviews are better than a flood of unbelievable reviews because at that point it's as if there were zero reviews.
Moreover, it's my understanding most of the astroturfing is from "bought" reviews. They ought have a way to detect that (accurately), or at least give them suspicion.
It's not as easy as it seems. Many systems have already been devised, but fake reviewers adapt; it's an arms race, just like with email spam. Bing Liu, who has done work on sentiment analysis, has written about some techniques dealing with spam/fakes on review sites. Here's just a sampling of popular methods and flags:
- Duplicate checking (same user on different products with similar reviews, for example)
- Meta-reviews ("was this review helpful?" - can also be gamed)
- user rating averages (all highs or all lows sometimes considered a flag)
- ratio of "first product reviews" to total reviews on a per-product basis
- "Super-reviewer" status (questionable, but often a flag)
- Products with low sales ranks
- Review ring detection (IP block, post times, etc.)
- Early reviews
- Users who give high ranks, while most other reviews are low
- Positive reviews for one brand's products, and negative for others
- "Verified" purchases
A lot of these already have countermeasures, and fake reviews have already come up with counter-counter measures (like review-time staggering, using multiple IP blocks, shipping empty boxes to defeat verified purchases, etc.)
The "how to know what's trustworthy?" is the million dollar question, and it has many answers that happen to change over time. Not easy to solve.
Could they add a kind of 2FA when they ship product to the customer? Understandably, this would drive down the rate of buyers posting reviews, but could minimize astroturfing rings. In your confirmation email you get a link which can only be used to review that product, if product is subsequently returned, then weigh review less, specially if positive, if too many products are returned, remove reviews for those products.
I'm not aware of any efforts in that direction. As far as the review link, I don't think that would defeat the newer countermeasure involving 3rd party sellers shipping an empty box (which never gets returned.) It still looks like a verified purchase. Second-parties could require all items be shipped through them (meaning they receive the item and inspect it, then reship it to the customer), but the added expense seems like something they'd be unlikely to spring for.
22 comments
[ 3.1 ms ] story [ 52.6 ms ] threadIf you really hate a place, just give it some scam-looking reviews and watch it burn.
I wonder how much it will take until someone starts a botnet discussing products on reddit..
Some possible markers of fake reviews I look out for, completely anecdotal: - products that have high hundreds or thousands of reviews, with almost all 5's and a few 1's. - products with a large disparity between the Amazon reviews and sites that (I think) are much less likely to be targeted for fake reviews, e.g Walmart.com or bestbuy.com It should be no harder to figure out an algorithm to stop this than to stop spam email, which makes me think sites like Amazon are not really motivated to stop the practice.
If you're buying tech stuff, I find that the following approach works well:
* Find the thing you want on Newegg.
* Read a few of the five-star reviews.
* Read a few of the one-star reviews.
* Read most of the four and three-star reviews. You can use these reviews to get a sense for the actual faults with the product, and to see if the inevitable DOA complaints in the one-star reviews are a real problem, or just bad luck.
* If the product checks out, buy it from wherever you like.
It's pretty ironic, really, that Yelp, a company famous for extorting small businesses for advertising subscriptions (with the threat of hiding positive reviews), is complaining about the validity of reviews on their site.
Their whole business model, afaict, is a shady scam against people who are too busy running their business to learn how to do SEO.
There would obviously be a conflict of interest for the people doing the reviews which is why I think it would make sense for companies to pay an intermediary who pays another firm to do the actual review. Companies that give reviews could earn "trusted" status through Amazon or Yelp (even though the are the mafia) etc.
https://en.wikipedia.org/wiki/Consumer_Reports
* How many false positives are there? How many businesses do they unfairly penalize (and how many reviewers do they humiliate)? For example, perhaps some behavior trait common to a location or social class is misinterpreted as a fake review.
* How many false negatives are there? How many fake reviews are given extra legitimacy in the eyes of users because the algorithm gave them a pass? Maybe the algorithm weeds out only the bad fake reviewers, or a certain kind of fake reviewers. Perhaps a cottage industry of techniques to game the algorithms will arise, like SEO. I do know that decent shills know how to fool amatuer slueths; heck, I could do it: Write something long, with specifics, add a little balance to look reasonable, mirror what some others say, etc.
I'd be interested to know how Yelp and Amazon can test their algorithms. They would need a large set of proven fake reviews, and a representative set would be much more valuable. Where does one find those?
But, meh tragedy of the commons.
How to know what's trustworthy? That's more difficult to establish, but someone like an Amazon has the capacity to build something like this. So would Yelp and they both have incentives to make the reviews published on their platforms believable -else we'll all believe it's just spammy astroturfing.
Still, a few honest reviews are better than a flood of unbelievable reviews because at that point it's as if there were zero reviews.
Moreover, it's my understanding most of the astroturfing is from "bought" reviews. They ought have a way to detect that (accurately), or at least give them suspicion.
- Duplicate checking (same user on different products with similar reviews, for example)
- Meta-reviews ("was this review helpful?" - can also be gamed)
- user rating averages (all highs or all lows sometimes considered a flag)
- ratio of "first product reviews" to total reviews on a per-product basis
- "Super-reviewer" status (questionable, but often a flag)
- Products with low sales ranks
- Review ring detection (IP block, post times, etc.)
- Early reviews
- Users who give high ranks, while most other reviews are low
- Positive reviews for one brand's products, and negative for others
- "Verified" purchases
A lot of these already have countermeasures, and fake reviews have already come up with counter-counter measures (like review-time staggering, using multiple IP blocks, shipping empty boxes to defeat verified purchases, etc.)
The "how to know what's trustworthy?" is the million dollar question, and it has many answers that happen to change over time. Not easy to solve.
Could they add a kind of 2FA when they ship product to the customer? Understandably, this would drive down the rate of buyers posting reviews, but could minimize astroturfing rings. In your confirmation email you get a link which can only be used to review that product, if product is subsequently returned, then weigh review less, specially if positive, if too many products are returned, remove reviews for those products.