Does the internet have a 1 star review problem?
This morning, as promised, a 1 star rating shows up on every review platform. She claims the hotel is ugly, the staff was rude and racist, and even that someone had pushed her.
Everything about this claim is a lie. My partner and have been here 4 days and everything has been fantastic. Every single person on the staff goes out of their way to be helpful and the property is beautiful.
Looking more at their reviews online, there are a handful of 1 star reviews with some crazy claims. One even looks to have photoshopped an image to make the pool appear dirty. Almost every 1 star review mentioned not being upgraded.
I've become especially sensitive to this after listing a property on Airbnb last year. Almost all guests and reviews are wonderful, thoughtful comments about my home and it's location. But then occasionally get a 3 star review with an explanation of something like "the dryer vent had lint in it". Come on...you're staying in a cabin in the woods, not the St Regis.
As a business owner and a technologist, I feel a ton of empathy for people that have it all on the line and can be so vulnerable. How can rating systems better account for/normalize against this kind of behavior?
26 comments
[ 3.1 ms ] story [ 76.3 ms ] thread1. This would be at a tertiary level. Customers would need to dig quite deep to even get to the response. The average number rating isn't impacted and it's the primary signal a customer is evaluating.
2. An honest response from a business owner can feel distasteful, even if it's earned. This is even worse for franchise owners because it feels like a big corporation is responding this way.
I had a contractor fail to do the simplest possible task (write a XML layout for a login screen). Gave him a 1 star. He threatened to give me a 1 star too if I didn't change the review, and did so as promised.
However, I have seen owners explain that certain reviews were part of a targeted attack to damage the owner's reputation, so I think it's the fairest approach.
Having the system to have contextual data (photos, comments) from both the sides – reviewed and reviewed – will be enough tools to let any independent person make a sound decision.
I can’t think of a non oppressive way to regulate unhinged ratings. On sites like here or Reddit, I’ve noticed users do a pretty consistent job of policing through karma. But I’ve also seen comments get downvoted because again, low credibility, human emotion blah blah.
It would really have to be a niche site that gate keeps reviewers. Only allow ones that've made columns about food, gained community status. Like Bon Appetit but strictly reviewers, on a larger scale. Something that proves they aren’t a chicken running around with its head cut off.
They do a decent job at downvoting/flagging spam and obvious harassment.
However, if you post something that is just ever so slightly unpopular or polarizing, you will receive the same torrent of downvotes. This makes it very hard to discuss things where you don't agree with the majority.
I would assume reviews have the same problem. Even if you had a wonderful time at a ressort, it's unlikely that this is true for evrery guest. Some guests will have a poor experience, and if they post about it they will be called a liar.
This of course depends on whether or not mods remove unpopular opinions, and/or ban people providing them.
When I look at my post history, I see that my most thoughtful and balanced comments are either ignored or downvoted. My most upvoted comments are usually rants that lots of people agree with, but I don't consider them worthy of the attention.
But some that's valid-but-unpopular also shows up. The specific characteristics can vary a lot by subreddit.
You can also use this sort on a personal page (https://old.reddit.com/u/<username>) to see which of their comments / submissions stirs up dispute.
Collaborative filtering / user-based moderation absolutely has its issues and limitations. You can eke some more information from it through the "controversial" sort, however.
Crowd-pleasing ranking systems like voting tend to surface crowd-pleasing content.
(There are reasons I spend little time on Reddit these days.)
So I don't think we have so much of a 1-star review problem, but more of a polarised review problem. Everything is either 1 star or 5 star with not much in-between.
Something I think the next generation of reviews should take into consideration, though: places change. A place that used to be good can take a turn for the worse, and a place that used to be bad can improve. Besides an overall rating, I would like to see how the most recent reviews assess an establishment.
If you have a problem with every hotel you visit, it's probably you. Right now negative reviews cost nothing... if negative reviews are expected to be a minor part of your overall review pattern, you have to be selective about expressing negativity or know that your review's relevance will suffer.
This item has 4.7 stars: https://shopee.com.my/product/110830198/5438249668?smtt=0.0....
It's a Monopoly game by the "Paker Brothers". I see 5 star reviews that say "board has been ripped, street names are wrong, tokens are made of cheap plastic, delivery a week late. But I got what I paid for." There's several which praise it for good quality. Come on, I can see "Company of Distribution of Electricity" from your video.
It has a 100% authenticity guarantee by the platform too.
https://en.wikipedia.org/wiki/Nosedive_(Black_Mirror)
Anyway, the problem with 5 star based rating is that everything below 5 star will actually hurt your rating, so it become sort of binary: 5 Star is OK, everything else is bad.
Being "average" and having "average" prices, we tend to have a few 5 star reviews, many 3 and 4 star reviews and a handful of 1 or 2 star one, of course.
Last 1 star review we had was about a couple that around 23:00 on a saturday night (having checked in at 20:00) lamented that they could hear the noise of an air conditioning machine (true, for some reason there was a vibration that could be heard from the room in the silence of the night) and pretended that we should fix it right now and then, immediately, refusing to move to another room (perfectly silent) that was offered to them because "it was too late".
Surely enough the next day there was a 1 star review, of course without saying that a room change was immediately offered.
The one before this was a couple years before (literally):
>Terrible
>The light on the bedside wasn't working.
What I noticed since the advent of these booking/review sites are two things:
1) the customers instead of asking for a room change or telling immediately to the reception whatever is wrong stay silent and then a day or two after post the bad review, often just the valuation without comments, so that there is no way for us to understand what "triggered" the 1 star
2) a number of people (could it be because they use smartphones and they mis-click on something?) post (good or bad) reviews that are clearly related to other hotels, I have a 4 star one (so, a good one) stating how the location is perfect, near the falls (there are no falls that I know of in a 200 km radius from here) though the swimming pools are a tadbit dated (we have no swimming pool) and it is a pity that check-in is not possible after 23:00 (we are open 24/7)
I always try to filter out all the positive reviews from Americans when travelling in Italy or France as Americans tend to view things more positively, romanticize average food and places and in general cant really judge the quality of the establishments in those countries.
On the other hand I very much value americans commenting on Michelin* and similarly priced restaurants as americans visiting these places usually have more experience (and obviously $) dining in such places and can objectively judge quality of the food.
So the answer is yes - the internet in general has a review problem, but knowing that upfront can at least help parse the information.
What I think is needed is for companies to incentivise higher quality reviews. Breaking review in categories so I can understand exactly what's 5/5 and what's 1/5 about the product/service. Weight ratings with detailed explanations and photos. Moderate spammy / unhelpful comments. Allowing the owner to respond. All these massively increase the value of a review.
I'm not sure there's much you can do about the odd unfair 1 star rating. There are always going to be people who dislike your product/service and they should probably have the right to express than dislike. However, I think in the case you're describing if the rating was broken into categories and the reviewer 1 stared every category, you can probably assume it's low quality and negatively weight it. The owner should be able to respond to comments like this and provide their side of the story. I use Trustpilot a lot and I'm often happy to over look the odd negative review when a company provides a convincing explanation and apology in response to negative comments.
I guess to answer your question, I think the issue is more that we tolerate low-effort reviews where people can just hit 1 star button with a one sentence explanation and their review is weighted the same as someone who spends 30 minutes providing a detailed review.
solutions 1:
Obviously most 1 star and 5 star reviews are not credible so you can just ignore them and get a rating based on the other ratings but the users cannot know that you are ignoring said reviews
solution 2:
Rating system from 0 and 10 as it gives the user more room to describe their experience, it also has the secondary benefit of being something you can normalize. If the average review is a 8/10 , that becomes your new 5/10 and you can devise some kind of mean and standard deviations from the mean system. again you can ignore like 0/10 or 10/10 ratings for the same reasons as previously mentioned.
solution 3:
Could use some combination of sentiment analysis and correlate it to the numerical rating.
Ultimately the user should not know how the sausage is made, much like the youtube algorithm, some things are obvious but its vague enough to make gaming the system harder than a basic 5 star rating of equal weight.