I'm currently planning a trip to India. I've been to several countries around the world and never worried about reviews too much, but India really seems to align with this story. I'll see hotels with countless 5 star reviews (hundreds or even thousands!) that are overwhelmingly positive and get ready to book. But with one hotel my BS meter got triggered, since it was in a rather dingy seeming part of town but still had thousands of great reviews, the pictures were gorgeous, and it was fairly cheap. Also, the reviews were all written in the same voice. "The owner is a very good sir top number one" style.
The low reviews had pictures of holes in the walls, bugs, exposed wires, and dozens of comments of people saying "I don't remember posting it, but this hotel posted a 10/10 review under my name. I think they hacked my account" or "the owner forced me to write a positive review before checking in" kind of stuff.
Since then, I've gone back and given a second look at all the hotels I've reserved so far and order reviews by negative first. So many are just fake or coerced reviews. I compared against places I've stayed in the past and I've seen no such thing. Delhi, in particular, seems overwhelmingly fraudulent and with owners openly getting pissed if you don't leave positive reviews. Small towns in India seem like they're probably okay (guests posting smiling selfies around the hotel and nothing overtly suspicious or anything), but I'm definitely feeling anxious about my upcoming trip.
> Since then, I've gone back and given a second look at all the hotels I've reserved so far and order reviews by negative first. So many are just fake or coerced reviews.
Of course, there's also a risk of fake negative reviews, from competitors.
That's something I've definitely worried about and seen some people mention in reviews around India. I mostly try to identify the voice of the comment writer. If they're from America but talking in odd English with a 10/10 review, I doubt it's real. Same if it's 1/10 from America and written with strange grammar.
Society is very good at adapting to such things. It will be an arms race for sure, but I don't think there is going to be a crisis of people being suckered by AI generated reviews. If it gets bad enough people will just stop giving them any weight at all, or even treat a lot of good reviews that are well written as suspicious. Then we will find some other way to find out quality of goods and services which will then be gamed at a greater and greater level until that too becomes unusable.
I've heard from establishments after leaving reviews on several occasions. Typically they will either offer a free meal or a refund of the unsuccessful meal. I will update a review to let people know that I was taken care of, but I haven't ever taken down a review. I would consider doing so if my experience had been one-off, and I understood that it would not happen again (e.g., if an employee acted terribly and was no longer working there).
One frustrating incident I had was when I stayed in an airbnb and found cockroaches, lack of hot water, and general filthiness. I wasn't able to leave a review because airbnb moved me to another unit (after many hours spent on the phone), and my account showed I had never stayed in the problematic unit. I can see why hosts would like a policy like this, but it's terrible for guests.
They were doing much better before their cost advantaged disappeared when they had to pay local taxes, and before current interest rates whacked hosts.
Amazon seems to encourage this. I ordered a pair of cheap wireless headphones that were highly rated. I get them and they are crap. But, included in the package is a flyer offering an Amazon gift card for more than I paid for the headphones if I gave a 5-star review. My integrity is worth way more to me that that. So I leave a one-star review with a picture of the flyer, hopefully someone else will see that and realize the reviews are fake. Amazon rejected my review because it didn't meet their guidelines since it mentioned "Reviews given in exchange for cash, discounts."
If it happens again, I'll go straight to blogging without bothering to report it to Amazon. I may not have that many readers, but it's more than the zero that I'd have through a rejected Amazon review.
I know I'm not adding anything new but I just wanted to chime in and say I've had the exact same experience as another data point for how widespread it seems to be.
I’ve had this experience probably 20+ times over the last few years with Amazon. Stars on Amazon cannot be trusted. Even products on Amazon from verified sellers can’t be trusted to not be fake. My general policy is don’t buy anything from Amazon for which I have any quality expectations.
Yeah I've been wanting to have that "comment on anything" for a while.
It's possible in technical theory. But sharing the comments would be hard. Federation might work. Spam moderation could be hard. And most websites refuse to go into iframes, so your options are limited to browser extensions, SOCKS proxies, and Opera Mini style apps that re-render the page. Which means you'll get less than a percent of a percent of people to use it.
Ditto. I imagined it like leaving a post-it note on a page. If you found a fix for a product you could leave a note on the company's FAQ page. Problem is spam, in all it forms. Can you imagine political pages?
This has been floated out a couple of times, but there’s one more factor you’ve missed: The companies really want comments/reviews done by a platform they control and they will fight against 3rd party methods.
If it's done as a browser extension, the way Keepa is (it's used to monitor price history on amazon.com), there's absolutely nothing the companies can do about it.
Tactics that have been used against similar tools:
- Changing the generated HTML so that the extensions selectors no longer grab the right data (this becomes a cat and mouse game and there’s a non-zero chance that the extension authors tap out, especially if there’s not enough money to sustain the workload)
- Filing nuisance complaints (DMCA is a standard one here, but there are lots of others)
- Filing a copyright lawsuit in court
- Filing other court lawsuits (lost income, slander, etc)
In many cases the business knows they are fighting a losing battle but all they need is more resources, more time, or more patience.
Yeah, we already have a few extensions similar to this for viewing price history, such as Keepa. I use this one all the time since prices jump around so much; when I look at a product page on Amazon, Keepa inserts a big window with a price graph and other pickable info.
I see it as, somebody acts unethically, and I can either (1) go along with their unethical behavior and personally benefit and encourage further unethical behavior, (2) refuse to participate, or (3) refuse to participate and discourage the unethical behavior.
How is this not a violation of some kind of truth in advertising law? Is the US government just of the position now that "if you violate the law and make us your bitch it's totally cool as long as you're automating your crime and committing it at scale?"
I received a “warning” insinuating that my Amazon account would be suspended for a bad review of a product that offered me a gift card for more than the value of the product in exchange for a screenshot of my 5-star review.
I am not sure what to make of this, I have considered that possibility the company has an employee on their payroll.
Ratings should always be on a curve for the rater, and on a 10 point scale, with no official meaning attached to the number of stars. If you rate everything 10 stars, you just gave everything 5 stars. If you want your 10 stars to mean 10 stars, you need to rate a bell curve of other stuff too.
(you might say "but if I only select the best places, and the ratings are always confirmed, that would be downvoting them." No, it wouldn't, that's exactly the type of situation that Gaussians know how to handle. Whatever is the average rating and whatever is the excellent rating will emerge from the collective.)
if you wish to attack my plan, I'll only listen if you offer a defense of the status quo, or offer a better improvement plan.
> if you wish to attack my plan, I'll only listen if you offer a defense of the status quo, or offer a better improvement plan.
Sorry but you don't get to set the terms of your own critique. If you refuse to listen then that impugns your character, but it doesn't make the critique invalid.
You method is flawed because some people only leave reviews when they good service or only when they get bad service. Everyone has a different reason why they do things, and what motivates them to do things. This will only work if you force people to leave reviews, and in that case your chances of getting a legitimate rating is much smaller.
You system would be better suited to de-rate the importance of people who leave a lot of reviews that are consistently skewed towards one end, but apply such a system to everyone is not going to make the results more fair.
> If you want your 10 stars to mean 10 stars, you need to rate a bell curve of other stuff too.
If you want your positive rating of something to have the most impact (for whatever reason), then you need to give negative ratings to other things (even if they deserve better)?
My issue is that we’ve been trained to see things on a broken scale - 10 is good, 9 is good, 8 is probably ok, 7 is questionable and everything else is really bad.
I get it’s chicken/egg, but I feel like you can’t adjust ratings without retraining people on what ratings mean. I guess that means I think we should probably do away with the numbers and think about a more qualitative approach? Not a strong belief.
there needs to be a way to indicate something exceptionally good, and everything can't be that.
if 5 is the center of the new scale where the current grade inflation puts everything, people will not need to be retrained. Then, let them vote. You don't need to understand grading on a curve to be graded on a curve, and the grade you get is the right one.
Steam has other problems with user submitted genre tags. Make a DLC that people thought under-delivered and you be may surprised to learn that you shipped a hentai game. Though that is definitely on the user’s side.
I've heard the arguments for switching to binary choice (netflix also did a big writeup about it), but I'd still prefer the ability to give a bit more nuance in my rating. It feels weird giving a movie that is okay and a movie that is excellent the same rating.
Rotten Tomatoes goes further to emphasize the "love it or hate it" dichotomy - they display a ripe red tomato for a movie with a rating of 60 and a splattered green tomato for a movie with a score of 59.
Stuff like this doesn’t naturally fall on a bell curve. Most hotels are great. Most restaurants are pretty good. There is no such thing as an objective 5.
in a normal distribution, you expect the mode to be at the mean to be at the median. If that's where most hotels are, and most restaurants, welcome to what 5 objectively means.
You expect that people will rate most things to be 5s based on averages, since if you aggregate your best experiences as 10s and your worst as 0s, then most everything is a 5, right?
But human brains don't work like that. We don't compare a dinner at a restaurant to literally the best dining experience we ever had or a trip to a theme park with the birth our child, so you can't realistically expect people to operate in that way when filling out a review form.
yes, and that's why figuring out the distribution of your opinions and normalizing them to convolve them with the rest of the raters will extract useful information from your ratings.
According to your logic we should be failing half of all school classes because half the kids shouldn't be above average.
Not many people are going to think that a system which weights everything to be distributed that way is useful as a metric for evaluating human experiences.
Maybe this could help with rating inflation by legitimate reviewers, but I don’t think it would help (and could possibly hurt) with the problem of fraudulent reviews. It would just mean that fake reviews would counterbalance their fake positive reviews with fake negative reviews of other products, making all reviews, in both directions, totally unreliable.
I once gave an Airbnb 4 stars because I was annoyed they had no way to make coffee, even though it was basically a hotel being run as an Airbnb. Like, not even a cheap Keurig in the lobby
They DMed me and told me that was bullshit and if I wanted coffee, I should've filtered by that amenity
Which is amazing because coffee is so expected you can't even search by it
Nah, they're 100% wrong. Maybe a coffee maker isn't as fundamental as, say, a stove or refrigerator, but I'd be unhappily surprised if the place I booked didn't have one. To my recollection, I've never been somewhere without coffee. It wouldn't occur to me to filter for such a thing. Should I also be looking for places that have pillows?
I do see what you're saying, but that's an expected basic amenity.
I feel like it depends somewhat on where the unit is located. If it's in a downtown area with half a dozen coffee shops within a few blocks, maybe it's not necessary. But if it's in the suburbs, then yeah coffee is pretty much required.
I wonder what percent of people make use of provided coffee. Some people are pretty strict about what kind of coffee they like, and wouldn't want to use whatever is provided. And some people don't drink coffee, of course.
“Lovely room, but I have to get up and dressed before I can get caffeine, which makes me subconsciously associate it with a headache and stuffy sinuses.”
A study I just looked up said 74% of American adults drink coffee every day. Of those, approximately 9 of 10 would prefer not to have to leave the house for their morning eye-opener, and 1 in 10 prefer papercuts then lime juice in their eye.
Seriously, no innkeeper who isn’t personally opposed to coffee for some reason would risk pissing off 75% of their customers, especially when it’s so cheap and easy to supply. I would genuinely rather a place lack pillows than on-site coffee.
Thanks for sharing those facts. As someone who has never drank coffee regularly, I don't know the extent to which coffee-drinkers would prefer to roll out of bed and drink whatever is on offer in the unit, versus getting dressed and grabbing a Starbucks/Peets/etc. a few minutes later.
It’s all personal preference, of course. For me, unless the place next door was spectacular, I’d much rather have a cup of mediocre-or-better coffee to sip while I’m getting ready.
The best situation is having a decent cup while I’m washing up, then a mind-blowing cup a little later while out and about.
You may expect that as an amenity but nobody else is obliged to share your expectation. The fact is that AirBnB provides hosts the ability to try to SET expectations about amenities, even ones as mundane as a coffee maker. Given that, I'd judge this guest to be wrong, not their host.
It's a review. Customer didn't like something, why not mention it for others? It's supposed to be a service for people like you in the future.
The problem is that the scale is now so compressed that 5* means "OK" and 4* means "not OK". 3* is assumed by the system to be a user abusing the review system.
And yet coffee isn't so expected that there isn't a setting for it in a listing's amenities. It's quite the contrary. In fact, there are settings for both "coffee maker" and "coffee", there for anyone to see who takes the time to see. Your AirBnB was in the right and you were in the wrong.
That’s just wrong. Plenty of useful information in reviews, even when most of them are gamed. Bad reviews are always useful so you should leave them, especially if they exist to counter the good fake ones.
I disagree. The best way to counter a wall of fake reviews is one thorough, real review. Add a few photos, if you can.
I review every hotel I stay at. From time to time, I switch on a VPN, go into private browsing, and get a bit of joy when I see that my 500+ word Google reviews are the first thing people see.
Honestly, I can't believe it is 2024 and people still believe/trust user-generated reviews of any sort. Isn't it widely known that these are always gamed, manipulated, algorithmically distorted and just totally unsuitable for making any kind of judgment about a product? I'm shocked that people and companies still pay attention to these streams of garbage.
I partially disagree. Just as an example, the 1-4 star reviews on Amazon are usually spot-on. It's always worth spending a minute or two to skim through those, starting with the lowest ratings first, while ignoring the 5 star reviews.
Ultimately a professional reviewer wouldn't be so easily swayed by this kind of sleazy behavior, but this is the price for having a stack of "reviews" for every gin joint in every town in the world, all written by Joe Public. Ultimately I think the old system was better: for most restaurants or trips, the "cost" of finding out whether you personally like a place is relatively small, and for bigger outlays we had the Michelin guide, Zagat, or what have you.
I stayed in a hotel recently via Hotwire. After my stay I received an email asking for my review using two separate icons, presumably two different links: one thumbs up and one thumbs down. I clicked on the thumbs down and received a 404. I gave up.
Airbnb also puts pressure on guests not to leave bad reviews: If you try to leave a 3 star review they ask you to explain using their criteria. And noise and lack of space aren't on that list. And they say they'll tell the host what to work on, as if paying full price for discomfort is acceptable.
Most of the time, I just leave 4 star reviews for places I didn't like and then mention all my hassles in my public comment.
Booking is much better: They ask you what you didn't like.
(I've used both Airbnb and Booking over 50 times over the last 6 years)
87 comments
[ 2.2 ms ] story [ 164 ms ] threadThe low reviews had pictures of holes in the walls, bugs, exposed wires, and dozens of comments of people saying "I don't remember posting it, but this hotel posted a 10/10 review under my name. I think they hacked my account" or "the owner forced me to write a positive review before checking in" kind of stuff.
Since then, I've gone back and given a second look at all the hotels I've reserved so far and order reviews by negative first. So many are just fake or coerced reviews. I compared against places I've stayed in the past and I've seen no such thing. Delhi, in particular, seems overwhelmingly fraudulent and with owners openly getting pissed if you don't leave positive reviews. Small towns in India seem like they're probably okay (guests posting smiling selfies around the hotel and nothing overtly suspicious or anything), but I'm definitely feeling anxious about my upcoming trip.
Then there's the famous case where Thailand tried to arrest a man for a not positive review: https://www.theguardian.com/world/2020/sep/29/american-wesle...
Of course, there's also a risk of fake negative reviews, from competitors.
Something you can see patterns of petty complains, and other times you can detect bigger issues that may make you look elsewhere instead
Most of these are present in major cities across India. They're a little on the expensive side though.
(I'm NOT affiliated in any way with any of the hotels mentioned in this post).
We were dropped at a house fifteen minutes walking from beach.
The next day I rented a motorbike and we relocated.
We never saw a refund.
One frustrating incident I had was when I stayed in an airbnb and found cockroaches, lack of hot water, and general filthiness. I wasn't able to leave a review because airbnb moved me to another unit (after many hours spent on the phone), and my account showed I had never stayed in the problematic unit. I can see why hosts would like a policy like this, but it's terrible for guests.
If it happens again, I'll go straight to blogging without bothering to report it to Amazon. I may not have that many readers, but it's more than the zero that I'd have through a rejected Amazon review.
I thought I could sift through the rubble to discern fake from real but now I know Amazon reviews are absolutely worthless.
Could be useful to fill gaps on other sites too, for example when an uploader turns off comments on youtube.
It would be a pretty big middle finger to amazon if the primary site for amazon seller reviews was not actually amazon, but a third party site/app.
It's possible in technical theory. But sharing the comments would be hard. Federation might work. Spam moderation could be hard. And most websites refuse to go into iframes, so your options are limited to browser extensions, SOCKS proxies, and Opera Mini style apps that re-render the page. Which means you'll get less than a percent of a percent of people to use it.
- Changing the generated HTML so that the extensions selectors no longer grab the right data (this becomes a cat and mouse game and there’s a non-zero chance that the extension authors tap out, especially if there’s not enough money to sustain the workload)
- Filing nuisance complaints (DMCA is a standard one here, but there are lots of others)
- Filing a copyright lawsuit in court
- Filing other court lawsuits (lost income, slander, etc)
In many cases the business knows they are fighting a losing battle but all they need is more resources, more time, or more patience.
Like, I don't judge. And when I was younger and broke half the time, I might've leapt at the easy money. I just wouldn't do it today.
I see it as, somebody acts unethically, and I can either (1) go along with their unethical behavior and personally benefit and encourage further unethical behavior, (2) refuse to participate, or (3) refuse to participate and discourage the unethical behavior.
You choose 2 and I choose 3.
I am not sure what to make of this, I have considered that possibility the company has an employee on their payroll.
Someone should ask Bezos - was that a one or two-way door decision?
(you might say "but if I only select the best places, and the ratings are always confirmed, that would be downvoting them." No, it wouldn't, that's exactly the type of situation that Gaussians know how to handle. Whatever is the average rating and whatever is the excellent rating will emerge from the collective.)
if you wish to attack my plan, I'll only listen if you offer a defense of the status quo, or offer a better improvement plan.
Sorry but you don't get to set the terms of your own critique. If you refuse to listen then that impugns your character, but it doesn't make the critique invalid.
You method is flawed because some people only leave reviews when they good service or only when they get bad service. Everyone has a different reason why they do things, and what motivates them to do things. This will only work if you force people to leave reviews, and in that case your chances of getting a legitimate rating is much smaller.
You system would be better suited to de-rate the importance of people who leave a lot of reviews that are consistently skewed towards one end, but apply such a system to everyone is not going to make the results more fair.
If you want your positive rating of something to have the most impact (for whatever reason), then you need to give negative ratings to other things (even if they deserve better)?
I get it’s chicken/egg, but I feel like you can’t adjust ratings without retraining people on what ratings mean. I guess that means I think we should probably do away with the numbers and think about a more qualitative approach? Not a strong belief.
if 5 is the center of the new scale where the current grade inflation puts everything, people will not need to be retrained. Then, let them vote. You don't need to understand grading on a curve to be graded on a curve, and the grade you get is the right one.
Binary choice leaves less room for interpretation and fuzzy ness, you either like something or you don't.
Maybe make it a really tiny box in the middle that you need to zoom in to click to encourage mostly positive or negative reviews.
Rotten Tomatoes goes further to emphasize the "love it or hate it" dichotomy - they display a ripe red tomato for a movie with a rating of 60 and a splattered green tomato for a movie with a score of 59.
But human brains don't work like that. We don't compare a dinner at a restaurant to literally the best dining experience we ever had or a trip to a theme park with the birth our child, so you can't realistically expect people to operate in that way when filling out a review form.
Not many people are going to think that a system which weights everything to be distributed that way is useful as a metric for evaluating human experiences.
"Absolutely awful service" / "dissatisfied" / "basically what I expected" / "positively surprised" / "incredibly excellent"
This way, most customers would click "basically what I expected", making 3 star the default.
Adjusting all reviews based on the general mean and standard deviation after the fact makes sense, though.
They DMed me and told me that was bullshit and if I wanted coffee, I should've filtered by that amenity
Which is amazing because coffee is so expected you can't even search by it
I do see what you're saying, but that's an expected basic amenity.
I wonder what percent of people make use of provided coffee. Some people are pretty strict about what kind of coffee they like, and wouldn't want to use whatever is provided. And some people don't drink coffee, of course.
A study I just looked up said 74% of American adults drink coffee every day. Of those, approximately 9 of 10 would prefer not to have to leave the house for their morning eye-opener, and 1 in 10 prefer papercuts then lime juice in their eye.
Seriously, no innkeeper who isn’t personally opposed to coffee for some reason would risk pissing off 75% of their customers, especially when it’s so cheap and easy to supply. I would genuinely rather a place lack pillows than on-site coffee.
The best situation is having a decent cup while I’m washing up, then a mind-blowing cup a little later while out and about.
The problem is that the scale is now so compressed that 5* means "OK" and 4* means "not OK". 3* is assumed by the system to be a user abusing the review system.
It is like a complaint of a laptop. If you sit on the keyboard, it bends a little bit. If that is the biggest complaint, it's fine.
I review every hotel I stay at. From time to time, I switch on a VPN, go into private browsing, and get a bit of joy when I see that my 500+ word Google reviews are the first thing people see.
Most people don’t know reviews are largely gamed, so that’s why people believe them.
I think this often gives at least an indication of what to expect.
Most of the time, I just leave 4 star reviews for places I didn't like and then mention all my hassles in my public comment.
Booking is much better: They ask you what you didn't like.
(I've used both Airbnb and Booking over 50 times over the last 6 years)
Same thing probably happens on other platforms. Example: Sometimes restaurant ratings on Uber Eats are a full 1 star above what they are on google.
Qatar Airways Bans YouTuber for Negative Review
https://news.ycombinator.com/item?id=38677344