214 comments

[ 3.5 ms ] story [ 229 ms ] thread
Interesting conclusion made by the authors about why reviews on Airbnb tend to be more positively biased than elsewhere.

> Reviews on Airbnb may all appear similar on the surface, but they nonetheless hold power as each one increases the amount of unique information available to other users, potentially reducing uncertainty for future hosts and guests. Our study confirms previous research focusing on star ratings: positive ratings and reviews are clearly the norm on Airbnb . No doubt, many positive reviews are the result of consumer experiences which frequently are genuinely pleasant. Nevetheless users should be mindful that there may be a number of other reasons for the strong positive orientation in so many Airbnb reviews: negative aspects of experience may be minimized, or left unmentioned, in reviews, due to factors such as: sociocultural norms of politeness, established trust among host and guest, review and rating reciprocity, lack of anonymity, as well as Airbnb ’s possible removal of reviews which violating their guidelines. Therefore, less-than-positive experiences may be concealed in lukewarm reviews where reviewers avoid overt negativity: for instance, in comments such as “Interesting stay in a nice neighborhood.” As a result, users should be aware that meaning resides not only in the information that is given, but also in the information that is excluded

How many reviews would you have to read to get the unique information from each one?

I'm not sure people read that many reviews.

I'd love a service that gleans interesting tidbits from long reviews and surfaces it all in a summary. I recently went to Curaçao and had a great time. The main reason it was a fun trip is because my girlfriend loves reading reviews and found all the pro-tips in them.
Amazon and other places seem fond of word bubble type information.
This is exactly how I read reviews on Airbnb and Booking.com. "Great location, friendly staff" and nothing else almost definitely means a very shitty place at a nice location.

Speaking of which, despite its horrible UI somehow I find Booking.com to be more reliable than Airbnb in this regard. I think Airbnb's game of mutual reviews (that are not published until both parties agree) eventually proves to be disfunctional and useless. Booking.com gives the clients the freedom to say anything without the fear of retaliation. Of course there will always be very unhappy customers even if the place is amazing, but at least you know you can rely on the averages and specifically the wording of the reviews.

Booking asks for the good parts and the bad parts seperately and then only shows the good parts. You really have to look around for the negative parts. I always do that and it is very helpful.
I wonder if there were ever attempts to replace reviews with questionnaires with some motivation to fill them like points or micro-discounts. It should provide more standardized flow and force users to disclose more about their experience.

I am curious why nobody does this, do I miss some obvious disadvantage?

They should like G2 and some others do for software products.

One obvious disadvantage would be that the friction increases drastically, hence reducing the number of reviews.

G2 and Capterra et al are gate-keeping cancer.

Change my view.

After my last stay in an airbnb I was asked a series of questions about my stay, facilities, amenities, the neighborhood, etc. They seemed to be tailored in some way or pulled randomly from a larger pool.
I'm sure they tailor it based on the amenities listed. They won't ask you if the pool was clean if the host doesn't advertise having a pool. They probably also tailor it based on historical complaints. If a guest has complained about a particular home being dirty, then future guests are asked more often about cleanliness.
I would not say the ratings are all positive. They are all between about 4.5 and 5 stars. So 4.5 is "worst", 4.75 is "medium" and 5 is "best".

The question is if there is a correlation of these and your preferences.

I have stayed in 13 AirBnBs last year and rated them in a spreadsheet. Putting them in one of 3 bins: good, ok or bad. The average AirBnB ratings for my bins are:

    good: 4.78
    ok  : 4.78
    bad : 4.92
So for now the correlation is negative. Not sure what to make of it.
So if you follow the AirBnB host forums at all, anything less than a five star(absolute best possible) review is considered bad.

Hosts flip out all the time on 4-star reviews. They loose can "superhost" status and other such things after a couple non-5 star reviews. Giving a 1-star review is probably worse than causing $100+ in damage for some of a lot of these hosts.

Hosts will rarely rate guests < 5 stars, simply for fear of retaliation.

The problem here is that guests are never told that 5 stars = normal stay. This leads to problems. If AirBnB was more up front about expectations it would make their review system a lot better.

There is a real sense of rating inflation happening here. It makes a 4-starred place seem especially sub-standard, even though it might be the result of a single unreasonable guest, or even a reasonable guest providing a muted review.

Yelp is hated in the restaurant industry for similar reasons.

This seems to be a problem with most similar review systems. I believe Uber has similar issues with anything less than 5 stars being considered unacceptable and causing problems for drivers.
Do users really evaluate like that?

For me, it's pretty obvious that rating is arbitrary and depends on how optimistic/perfectionist/obsessive/etc. the reviewer is and that there is not much difference between 4 or 5 stars.

It's not users, it's the platform itself. AirBnB and other companies like Uber and Lyft will significantly deprioritize you if your average rating isn't very close to 5 stars. Rating your Uber driver 3 stars can significantly affect how many fares they're offered and therefore how much money they can make.
But if users don't care, why do platforms care? Maybe it's actually a reliable signal?

Obviously it's also more reliable in the context of other evaluations by the same reviewer which the platform has easy access to, i.e. if someone always gives 5 stars, then a 4 star review from him is much worse than one from someone who never gives 5 stars.

Users caring might mean that a user is 15% more likely to select a 5 star host over a 4.5 star host, not that the user will say a 5 star rating is a top concern.
I feel a better system is just thumbs up/thumbs down. Either it was satisfactory, or it wasn't.

Basically the same as our current 5 star/1 star dichotomy but it makes things more clear to people who are tempted to give 4 stars.

Even though it seems like shrinking the available options will lead to less fine-grained data, sometimes it seems like it actually works out better. I really liked how Four Square only had thumbs up and thumbs down as the allowed ratings. I feel like it did a very good job of capturing how aligned with expectations the restaurant experience was. It also made it really painless to give feedback.
Yup, I really miss UrbanSpoon for the same reason (now owned by zomato and totally ruined). I found it's percent-thumbs-up metric to be so much more valuable than Yelp's 5-star average. If you really want a finer-grained system, have multiple thumbs-up, thumbs-downs.
YouTube originally had a five star rating system for each video but switched to the current thumbs up/down system. I believe the reason given was that almost all viewers either give five stars or one star, which made the middle three ratings pointless.
How do we make a single thumbs down not trigger a 5 alarm fire in the customer service department? Anyone else dread the onslaught of follow up from a company after you give a bad rating?
Is there a dichotomy? I don't see the 1 star being more common than 2 or 3.
I changed my music player/library from the default 5-star rating system to a 1-star system, basically replicating the "like"/"save" functionality from Spotify. It makes it easier to handle, because I don't have to think about whether I like this particular song 4 stars or the full 5 stars.

I just mark favorite tracks and favorite albums. The track and album ratings are separate, because I might love an album as a whole, but also have a couple of individual favorite tracks on that album.

And I don't have any music in my collection that I don't like or am completely "meh" about. If it's in my library, I already love it, the question is whether it's an all-time favorite or not.

They could circumvent that problem by replacing the five stars with an up/down system -- since that's the way it's interpreted anyway. I don't think most users are capable of giving more nuanced reviews anyway. If they can, it would be better expressed in words than as numbers.

If I have a stay that's genuinely superior, I'll write an enthusiastic review. Other than that, "it was fine" or "don't come here" is really all I need to say.

I wonder how changing the rating system in cases like these from a five star scale to a thumbs up/down system would affect user behavior and system usefulness. It seems like ebay, uber, airbnb, etc all suffer from the same "anything less than 5 stars is effectively negative" issue.
Once you have a large enough userbase the thumbs up / thumbs down method might be the best option. My favorite example of numbered rating systems failing is metacritic where almost every user score is either a 10 or a 0. They have a weird dynamic where users will give a 10 to counteract a low score or a 0 to counteract an inflated score. Giving a 0 or a 10 makes your vote more impactful so why would you score any different?

At the end of the day most people just want to know if people think a service, movie, driver, seller, etc. is worth it or not * . If you need more info you are going to read the reviews anyway.

* An exception might be niche user review communities like myanimelist or rateyourmusic

Who looks at the user reviews on metacritic?
Lots of people?

The difference between the critic rating and the user rating is useful information to have when making a viewing decision. If they are both low, that tells you something. If one is low and one is high, that tells you something.

Grading scales can work, for instance if people are rating and reviewing products in online shops.

You're looking at a jacket and you see that while some people rate it 5/5, most of the ratings are 4/5 and 3/5, and 85% of the reviewers say they would recommend the jacket to a friend.

So you read the reviews, and gather that most of the medium scores are because the arms are a little shorter than expected and the cut across the back is rather wide. You also notice that everyone is praising the quality of the fabric and worksmanship. Good information, since if you have short arms and the back muscles of a powerlifter, that jacket would probably be a good purchase.

That is good use of a scoring system, but it requires people to care about it and about writing reviews. It works for niche sites, but completely breaks down somewhere like Amazon, or on Uber/Airbnb where you get punished for anything less than a perfect score.

Ebay has a positive/neutral/negative review system for sellers, right? Does that have any interesting side effects?
This reminds me of the idea of net promoter score in customer satisfaction surveys. It's a variation on the idea that it's better to have imprecise knowledge that is relevant than precise knowledge that is irrelevant. Net Promoter Score replaces a litany of questions about satisfaction with a single question ('would you recommend this product to a friend' (yes/neutral/no -> 1/0/-1)? You take the average of this score ((yes - no) / total) and it corresponds roughly to the health of a product. The basic idea is that asking a bunch of questions doesn't really add much knowledge, and giving more choices and asking more questions can actually lead you down a false path.
Everyone who has ratings does it. I used to rate 3 = professional and fully acceptable until I got a call from Dell asking what they had done wrong, and then from Honda. I realized ratings are all inflated and staff are effectively penalized for less than perfect, and since I'm not going to screw someone because I disagree with their rating scheme, I just set 5 to acceptable and go off that.

The sane way to do it would be to ask yes no questions, but that would require thinking.

So why leave ratings?

Nearly every time I have a bad experience with a company the problem is systemic, but a bad rating will be associated with the random person who served me. So I don't participate. If the problem is particularly bad I find another supplier.

(comment deleted)
If they did their job, I'll give them the "you did your job" 5 stars.

Even if the company has systemic problems, they still need to pay bills as long as they still have the job.

Finding another company is a smart option. I'd also write to the management to explain why that problem is blocking you from doing business with them, without mentioning who you spoke with.

It's the same thing with Uber and Lyft. Dropping below 5 stars can result in termination.
And then when guests know this about the rating system, a lot of them will go even further to avoid giving a low rating because they don't want to harm the hosts' livelihoods unless something truly outrageously bad happened.

Just like a lot of people I know will tip 20% on a restaurant bill without even reflecting on the level of the service, because it's so ingrained into people that wait staff rely almost entirely on tips for their livelihoods so people feel guilty using this quality signal mechanism the way it was initially intended.

Because the way it was intended is inhumane if you even barely reflect on it
It's inhumane to give someone a bonus for good performance?
It is not a bonus, they are legally exempt from minimum wage laws.

The entirety of their livelihood is dependent on our judgements of their service.

If they don't make enough in tips to reach minimum wage, the employer still needs to close that gap. It's not as exploitative as we're led to believe, or at least no more so than working a minimum wage job is normally.
Yes but this never happens. If you cost your employer money at the end of the month because you didn't make minimum wage then you'll be fired. Employers don't want to pay out, employees don't want to be fired so nobody ever speaks of it when it happens and employees just accept less than minimum wage.
You are mistaken. Firstly asking your employer to close the gap is a great way to get fired. Secondly minimum wage is so low that you cannot possibly live on it. It's in fact so low that not even walmart pays minimum perhaps because the kind of people who would work for it aren't liable to be worth employing.

Federal minimum wage is 7.25 after taxes this is about 250 per week wherein most people in this situation are paid every 2 weeks. That is to say that one cannot expect to have more than 2 paychecks per calendar month for a grand total of $1000. If you didn't need food, clothes, medicine, or anything else this is about enough for rent.

What this means is that the employee/employer relationship not to mention the individuals ability to live indoors and have food is actually predicated on the ability to achieve greater than minimum wage. Tipping out even if they can do so isn't enough to keep anyone afloat. Not tipping in a state that has a tipped minimum wage is not that much different from dining and dashing its just not illegal because its never been illegal to screw someone if they are substantially poorer than you.

(comment deleted)
cactus2093 referred to how it was originally intended, not how it currently works. I believe the original intent was for workers to always be paid a decent wage by default, and to get a bonus for exceptional service. Now it's inhumane not to tip because tipping is built into salary expectations, but that's unrelated to the original intent.
it depends: that isn't true in san francisco, FYI.
The jobs aren't blanket exempt. That's a federal rule. A ton of states don't exempt tipped jobs from minimum wage. In California, for example, where it is the case, the usual minimum wage is $12 or $13, depending on the number of employees, and a dollar less for tipped jobs.

But repeating the pattern, that's a state rule. Different parts of the state don't exempt jobs from minimum wage. (God, I'm glad I'm not a lawyer). In San Francisco, for example, the minimum wage is $15.59, and as far as I can tell, it holds whether you're tipped or not.

Minimum wage didn't even exist when tipping started.
Is it not expected that when you pay money for a service then the performance should be good? Good service should be the default. If the service is outstanding then that is when we should start giving a bonus.

US tipping etiquette has nothing to do with good service and all to do with propping up underpaid staff.

I think we agree, it's more of a matter of adjective grade. I expect service to be adequate. Some of it is frankly good. Once in a while it's outstanding.

As for US tipping etiquette, cactus2093's point was that current US tipping doesn't really reflect what was originally intended. But kingkawn made a claim that extended past the current US tipping practices.

The argument that you should always tip food service ~20% is generally based on the claim that "quality signal mechanism" is not actually the intent of the tipping system, but merely an excuse (that was perhaps true at some point in the past) to justify legally paying workers low wages and artificially reducing the sticker price of food.
> If AirBnB was more up front about expectations it would make their review system a lot better.

I guess from a strategic point of view it might look better for the AirBnB's and Ubers of this world to have lots of '4.7 out of 5' ratings than 'xx% of people wanted the service provider to be punished', especially to first-time users. But yeah, if Uber in particular wanted to honestly represent what the ratings actually represented it'd be a simple upvote/downvote.

Ironically, on platforms more focused on allowing consumers to choose based on qualitative reviews than penalising based on quantitative scores, I tend to find 4* reviews are much more likely to be well-informed enough to influence me into buying the product than the 5* ones, as well as more likely to be a genuine reflection of the purchaser's sentiment.

How would retaliation work? I don't think I've ever been able to see my hosts review of me before submitting mine for them.
> The problem here is that guests are never told that 5 stars = normal stay. This leads to problems. If AirBnB was more up front about expectations it would make their review system a lot better.

They’re not entirely explicit about it, but to leave anything less than 5 stars AirBnB you have to leave a text explanation of why you did. So it becomes clear quickly that 5 is the norm and less is for an actual problem.

If you score around three stars each time then you’ll see the prompt each time and not think anything of it.
Yes, this is such an anti-pattern.

Once Uber started asking me a thousand questions about why I was leaving someone 4-stars, I stopped leaving reviews entirely.

Lyft does the same. And when you tap on 3-stars it even says "Just OK" yet it treats it like it was a horrible ride.
If a Californian told me a restaurant they went to was "Just OK", I would avoid going to that restaurant unless there were no other alternatives. I think that might be part of the disconnect.
That is something that I find weird in US "ratings". To me, "ok" means that it was... Ok. Definitely nothing bad, from completely satisfactory to good, but not exceptional. Which for some reason seems to translate in US to something like "Awesome, best food ever in my life". Now, my question is, what do you say when you actually get awesome food that is best you have ever eaten?
"OK" and "Just OK" are two different things. Californian culture tends to shy away from direct conflict unless it's deliberate so there's a lot of damning things with faint praise.
Speaking of subtle differences, what's your take on "A-OK"? That's Yelp's label for 3 Stars.

I also think that it's different for restaurants than for drivers. When looking for restaurants whether in my home city or traveling, of course we look at 4 stars or above. But never have I once looked a driver's rating, I just call a car and take the first one that is assigned to me, cancelling only for reasons not related to ratings at all. Maybe it's just me that does that?

> Today the average rating on Yelp sits in between “A-OK.” (3) and “Yay! I’m a fan.” (4).

src: https://blog.yelp.com/2018/09/restaurant-ratings-on-yelp-are...

Has that actually happened? I’ve lived in the US for basically my whole life and I’ve never met someone who disingenuously described a mediocre eating experience as the best in their life. I think it would be hard for them to make friends if they can’t even be honest with how they feel about meals.
I think that depends on where you live. I’ve lived in places where there were not that many restaurants better than “just ok” unless you wanted to spend a lot of money. Nowadays, I feel as you said, that I never have a reason to go out of my way to new place that’s “just ok” but that’s because I live in LA (and previously SF) where there are “seriously great” options everywhere. My standard for affordable food has gone way up after living in California.
My dad always says "Yeah, it was a pretty average house. Nothing terrible, nothing amazing, 2.5 stars". He then clicks in the middle of the middle star, and ends up giving 3 stars.

Hosts must hate him.

sounds uber and airbnb should ditch the 5 star system and switch to thumbs up/down like netflix.
With Netflix the 5 star system made sense and they shouldn't have ditched it. There was no issue of someone losing their livelihood if you left a 4-star review on Netflix, so Netflix movie ratings were pretty accurate I think.
I used to give most Uber drivers three stars until someone told me it’s not supposed to be like that. There’s nothing in the app to guide you that the scores are shifted like this.
Great so now I feel bad for all those less than 5 star reviews I left.
Sounds like eBay, buyers can't get anything other than a positive review. You can write whatever you'd like in that review but no one reads them.
The fundamental problem is that as soon as a metric is used for financial incentives, it becomes useless as a metric.
Not only do they flip out (which means you leave a 5 star review to avoid drama) but if you have a really crappy host that cancels your reservation or run into any weird issues you are not ALLOWED to leave a review. Therefore the only reviews you are seeing are "successful" stays. I'm never going back to AirBnB. It's been hot garbage all the way down. Can't trust a thing on there.
An automated review is posted when the host cancels, indicating how far out the cancelation from the stay happened.
Do these reviews include stars? Ie would be good to know how the cancellation was handled. There was a recent example in NZ where a host cancelled on a family based on their ethnicity (Covid-19 fear related), would the automatic reviews provide insight into such issues with hosts?
The current system can be improved, certainly.
I've mostly ran into issues where the host is not able to work out a significant issue (unresponsive for days, does not provide access to property, property smells so badly of smoke you cannot breathe adequately - as a former smoker this is a high threshold). This is not technically a cancellation but an "ended stay" and you are not able to leave a review once you have entered the property in my experience. That said, I'm sure it's improved (hopefully in the year+ I haven't touched it)

In general, it's been extremely difficult to read between the lines on reviews since they're all five stars, and sometimes you end up reading things unintentionally.

It sounds like they should switch to a binary review system (thumbs up/down). Leave the nuance to the actual review text.
> If AirBnB was more up front about expectations it would make their review system a lot better.

Better for consumers but not better for AirBnB. The fact that potential customers assume that 5-stars indicates an exceptional rather than standard review is an advantage for AirBnB. Revealing the actual rating expectations would cheapen the perception of all the existing reviews.

TFA's thesis is that people convey negative information -- just not primarily through the star rating.

As a longtime Airbnb user I can't imagine staying at a place just because it has a good star rating. You have to read the reviews and see what people liked and didn't like. If a place has a bad star rating I generally wouldn't consider staying there at all, but a 4.8 star rating with lots of substantive reviews and no real complaints is very different from a 4.8 star rating with a bunch of people noting that there's deafening construction noise from 8AM to 8PM.

Also, how would anyone retaliate on Airbnb? You can't see the host's review until after you've posted your own.

Wouldn't a binary rating system work a lot better here? Either positive or negative + a comment, sounds like it would work way better.
> Either positive or negative + a comment, sounds like it would work way better.

Exactly. This is how CouchSurfing works. It's just another way of asking "Would you recommend this place / this person to your friend?". And that's what people really want to know, after all.

It's worse than that - they actively edit/delete negative reviews.

https://qz.com/1333242/airbnb-reviews/

I wonder if they did this with our only negative review. Her comments were rambling but the visible review was vague.

We charged 20$/night since it was our first Time doing it. We put in the description, No TV. But the guest asked for a TV and Chromecast.

So we moved a spare living room TV upstairs. I went to the store and bought a 35$ Chromecast.

We got a negative review for the TV being Dusty.

It made us paranoid and we tried extremely hard from that point forward. Got some sort of "best host award"

Then our city shut us down.

(comment deleted)
You're a rounding error. Most of Airbnb's volume is professionals letting out dedicated spaces on an ongoing basis
I wonder what their process is for deciding if a review should be removed. I just checked the listing page for a place I stayed at two months ago, and the (poor) review I left was still up, along with a few others.
Yep, and don't forget that savvy hosts have ways to prevent you from leaving a review (I don't know how, but I do know that when we had to leave a cockroach-infested unit, we were unable to leave a review). That means that there may be people who want to leave an informative (and bad) review, but are unable to do so.
What do you mean ways to prevent?

I did notice that the last review I left isn't actually displayed on the site. It was far from negative, but I did leave 4 stars and note the extraordinary thinness of the walls, as it made hanging out inside of the apartment a bit of a exercise in stealth. Strangely, review was never listed.

We left a unit due to numerous undisclosed defects (effectively no hot water in shower, cockroaches) and moved into another Airbnb. We were not allowed to leave a review for the problem unit. It was as if we had never stayed there.
The host has to agree to allow your review for it to appear.
Has to agree to what? I thought reviews were between the reviewer (host/guest) and the platform, and that the reviewed party can only see it after they submit their own review.

In my case, there was not even a chance to write a review, not that I wrote a review and it didn't appear anywhere.

It seems that Airbnb's policy effectively hides the worst reviews (ones where the guest is moved to another unit and the host terminates the stay in a way that prevents a review from being possible), which contributes to the 'grade inflation' that we see in Airbnb ratings.

I’m a host. How do I do this? I’ve never had the ability to hide reviews
They didn’t hide a existent review. They canceled the stay in a way that made it disappear from our dashboard so that we couldn’t review it.
Dispute a review. Offer to pay back partial funds to get review removed.
That's not at all the same as what was claimed in this thread that hosts can prevent a review from appearing in the first place and that hosts have to allow the review before it's published.
No, as reviews often mention specific details of the location - even if those details are mostly positive.
I've written a few negative reviews that were totally honest. They never made it to the profile. So yeah, pretty meaningless.
I had the same experience. I called AirBnB and they acted like it was a really strange occurrence and they would look into it. My review never appeared on the host's page. I have zero trust in AirBnB.
You’re right in having no trust in them. Recently I complained about my price increasing by a couple % once I signed up. Nobody could tell me what the problem was. It took them several messages to finally spill up that they were charging me more due to my country’s taxes.

Why is this not clear? Airbnb is so opaque. It would be really easy to list this extra tax but they don’t, so I end up creating a new account.

mine showed up on the profile couple of weeks ago. Its still there.
There is probably a similar issue in academic grade inflation. I'd be curious if professors have any strategies for dealing with this (aside from resorting to more traditional word of mouth approaches).
Is this also a thing on ebay and amazon and ali? I frequently review as "did exactly what they said, delivery on time, no issues" and I do not intend implying negative outcome, I am just choosing to be terse. "the flat was fine, streetscape boring" doesn't mean it was a gang-house in zombie-land.
Amazon have started to ask questions about specific aspects of the product.

It seems to be based on AI/ML, I think. I bought a Dyson vacuum cleaner which was terrible at picking up hair from the carpet, and when I went to add an Amazon review, it asked me an extra question which was something like "Good at picking up hair?". That must have involved either human interaction, or AI, or it's a question they ask about every vacuum cleaner and I'm showing a cognitive bias.

AirBnB and similar platforms have business models based on quantity of transactions, not quality of experience. If you can obscure an honest assessment of the quality, you can increase the quantity of transactions (at least in the short- and mid-terms). Inflation of ratings and hiding bad reviews are not accidental, that’s their business model.
Yes. I had a nightmare Airbnb over new years (had to leave because my health was deteriorating) but couldn't leave a bad review because of course i'd get one back in retaliation.
The owner can't see your review until after they review you. But, that won't stop future owners from seeing your bad review.
I think op is saying that they were given another airbnb to replace the bad one, and in doing so, you lose the ability to review the first, bad location.
No we were not. The host knew we were mad at the state of her house, any review made on our part would trigger her retaliation (as she would assume correctly it wouldn't be positive).
Is it too early to jump to the conclusion that keeps nagging at me, which is that all the various attempts at reputation management that try to mitigate the basic inherent detachment/uncertainty of the remote, network-enabled business model that we all increasingly depend on, were doomed to fail all along? Mainly I guess I'm thinking of this: https://en.wikipedia.org/wiki/Campbell's_law
After a stay at an AirBnB, I provided a review. A part of the review question was "What would you suggest as room for improvement?. AirBnB told me they will keep this answer confidential. In a few hours, I had multiple passive aggressive messages from the host about the "confidential" feedback.
I mean, likely AirBnB did relay the information confidentially... but doesn't take a genius to figure out which guest suddenly provided the confidential feedback which was sent on the day after your stay ending.
If you shared the content, you didn't keep it confidential. Maybe you are confused with anonymous?
Are you sure it said confidential? As far as I know Airbnb has always had a field for “A private note for the owner”. This is “confidential” between you and the owner, as in “not public”, but not between you and Airbnb.
Who did you think the feedback was for? AirBnB employees?
I wonder if this was why Netflix switched to a thumbs up or thumbs down system.
Do you remember how people used to ship blocks of ice a century ago? It was a massive industry.

Thousands of pounds shipped over thousands of miles of water. A miracle for its time. The improvements in food storage that came afterward were nothing short of a marvel. The ability to preserve food, to hold it in statis; it bought people time to ship and eat food (a very precious commodity indeed).

Looking back now and it seems strange. So much effort for so little gain, relative to what we're capable of doing now. Plug in a refrigerator and you're done. Ice cold for a lifetime.

There's still a lot of effort going on, but much of it is hidden—a landscape of electrical grids, mathematics, steel and gold mined from the Earth to create the technology in our kitchens.

Most people probably didn't even know things could be much better; that's all they ever knew. The leap to refrigeration was by no means an easy or obvious one. And it seemed to happened slowly as the infrastructure was slowly built up bit by bit over the course of a few decades.

Review systems remind me of the ice shipping industry: A marvel for its time, but only for its time. The ability to filter information about locations and products; to make informed choices by aggregating a large set of human opinions is remarkable. You can make choices about things you've never experienced, across a wide variety of domains: Amazon, Airbnb, the App Store, Shopify, Google Play.

But it's slow. It's tedious and subject to error. It's easy to miss great insights. It's the lowest common denominator for gaining information about a product, subject to manipulation and noise.

Nowadays, whenever I go on a 5-star review website, I'm thankful for the information when I can get it, but I always wonder when my refrigerator is going to arrive.

Nowhere is this more evident than when you want to do comparison shopping on an expensive product like a camera or a car. I am actually in the market for a new car right now. I know pretty much what I want. But trying to figure out which product is the best fit for what I want is a nightmare. I go on a manufacturer's web site and it's all marketing. Trying to extract technical details is a chore, and trying to figure out the actual differences between different models and trim lines can range from a nightmare to downright impossible.
With car shopping I’ve always hunted down fan forums. There you can usually find out if there’s any issues and whatnot - and even sometimes when deals are about to happen.
I just tack reddit on to the end of every search.
Cars (and cameras) are even more complicated because of prestige and reputation influencing price so greatly. Hyundai and Buick end up as two of the best values right now because nobody wants a used Buick. Cameras have a similar but different thing going on where most people think they need a "DSLR" to take professional pictures, and they buy something three times as big as they need because they dont want to "downgrade" to a mirrorless.

Obviously branding and status play a factor in many purchases, but the larger ecosystem / maintenance agreement ones are a different magnitude. Buying a camera is buying into a camera ecosystem, and buying a car is both a product and its serviceability. The Buick can be greatly reliable, but if its perception is that of something less reliable, you pay for perception not reality.

This is why professional reviewers and critics exist.
This has been my experience with any complex product. All our media is regressing to the layman's mean. Why bother posting technical specs and information if 90%+ of your customers won't bother looking at them?

Frankly it makes us all dumber.

i absolutely love this comment, you’ve got a great writing style
reminds me of joe from halt and catch fire
(comment deleted)
Then maybe you can clarify what its actual insight is? I still don't understand how ice shipping is analogous to reviews, or what the common dynamic is. Simply expressing that [1] got me 30 net upvotes and no attempts to explain.

I think a more reasonable hypothesis is that the comment is fluff -- it doesn't communicate anything relevant but somehow makes you feel you learned something and changed your mind. That is not what I want to see more of on HN.

[1] https://news.ycombinator.com/item?id=22507740

Can you think of some innovation that might replace review systems in the same way that refrigeration replaced ice transportation? This seems more like a cycle where review systems will be nice until they inevitably get gamed, at which point some new company will come out with expertly-curated reviews and recommendations, but then that company will sell out and compromise their curation, at which point the new crowd-sourced review website will take over. Ad infinitum.

I suppose you can come up with ideas like AI curation or spam prevention, or some decentralized reputation system. Those sound pretty wild, but hey, I reckon refrigeration would have sounded pretty wild too.

There needs to be an independent outsider that can give uniform and unbiased reviews. i.e. Michelin reviews for airbnb. Question is how many people are willing to pay for these.
You could imagine a network of trust, where some people have more trust than others. And there could be clusters where people with a lot of trust are not necessarily trusted by everyone. So this would be more general than a central authority like Michelin.
Isn’t that Consumer Reports? Although Airbnb listings are too extensive for them to independently review.

Edit: I suppose they could step in and evaluate it as a platform and validate whether issues get addressed.

Shipping ice was inefficient, but it worked: it actively provided the same benefit as refrigeration later did and it didn't have an "adversary" (as in, people or food that were actively undermining it). In contrast, the research on five star rating systems is that, mathematically, they don't work (as no one is using scales that are compatible, you cannot do algebra on the values to get an "average"). Meanwhile, they are actively gamed, and so can be worse than noise, making people believe something harmful. If you want a better analogy, it is like a bunch of people relying on some combination of apple cider vinegar (which might do something, or might not, but may as well try it), snake oil (which definitely does nothing, but is claimed to do everything), and garlic (which does something, just not that well, but is often placebo effect) in an industry already attempting to indiscriminately deploy antibiotics (which works, but only temporarily, as once an infection adapts it is all downsides and no upsides)... and no one is even sure if a cure is possible. Ice works, we knew it worked, we knew why it worked, and all we needed was more engineering to make it efficient and scalable (by eventually figuring out how to simulate ice using carbon, chemicals, and metals): don't pretend for a second that rating systems have anything in common with something that worked.
Ice keeps things cold, and someone has to keep bringing new blocks of it to your icebox somehow.

Reviews are free data, and you have to know how to navigate them.

I don't see the difference.

Right, but with ice, the mechanics are pretty simple: heat gets in and warms stuff up, and its behavior is predictable. OTOH, reviews are "free data", but you don't know whether it's a real review or a fake review. If you decide incorrectly (eg. believing the fake reviews over the real ones), you would end up in a worse position than you started out with.
Ice is not an adversary.

If reviews, or any post-review substitute technology, become the thing that helps imperfect humans make decisions, then that becomes the target for attack by misinformation. If reviews drive business, then so do fake reviews. Reviews are valuable, so there's a market in reviews.

> You can make choices about things you've never experienced, across a wide variety of domains: Amazon, Airbnb, the App Store, Shopify, Google Play.

Reviews steer hundreds to thousands of dollars of spending from every consumer. Nothing that valuable can be allowed to remain authentic.

Perhaps this future review system could be used for voting in elections, or perhaps even for fair wealth distribution.
I have no idea what the analogy is supposed to be here or what the insight is.
(comment deleted)
I never read positive reviews.

If the negative reviews sound whiny, I ignore them. If they bring up valid concerns I stay away from the product/business.

Yes, if your message contains only 1s, then its expected information content (aka Shannon entropy) is zero.
AirBnB takes down negative reviews. I had a poor experience and wrote about my experience. When I asked for a refund, a small percentage was refunded by the host. AirBnB immediately took down my review and then refused to communicate.
Goodhart's Law - reviews were useful ten years ago, they are utter trash now because they are totally tainted.

This is going to reverse horribly on all the "platform" businesses because they are totally conflicted. As ever, we will go back to the same stuff that people always used: word of mouth, building a trusted brand, etc.

All the "successful" unicorns are only distinguished because of their utterly trash brands. Every one of these companies grasped the short-term gain (whilst saying that private capital meant they were long-term thinkers), they hired staff who had no idea how to run a business and were interested only in enriching themselves, and they are basically toxic now. AirBnb, to my mind, are one of the worst. There is no coming back from this.

In Germany (and Austria AFAIK) employers are legally obliged to write letters of recommendation for their employees. These letters must be true, complete and favorably. So we have a similar situation where employers want to express their opinion but are required to be benevolent.

Does this make letters of recommendation from German companies meaningless?

Some will say yes, but regardless of that, what is certain is that the whole situation developed a kind of flourish language that can express at lot about the employee without ever being obviously negative. Some sentences can even be translated into school grades.

There are numerous books about this language and even more websites and they all more or less agree, so this language is pretty standardized since at least maybe 30 years. A last interesting point about this language is that some things are said by leaving out certain phrases or words, so you say something by not saying something.

Finally I'll give you few examples [+]. May they serve as an example for a future AirBnB language;-)

He has specialist knowledge and a healthy dose of self-confidence.

Meaning: This person is an arrogant jerk.

He's quickly become popular with clients.

Meaning: This person can't negotiate at all.

She had great empathy for the needs of the staff.

Meaning: This person flirted more than they worked.

His sociability helped to improve the atmosphere a work.

Meaning: This person is an alcoholic.

[+] Translated by me to the best of my knowledge. I'll spare you the German originals to avoid clutter. If you want sources shoot me a message, E-mail is in my profile.

(comment deleted)
Can you give us some examples of recommendations that would translate favorably?
Sure. The ones that come to my mind are little bit harder to translate though, because they rely on nuances in German grammar, but I'll give it a try.

To give a person grade A in their performance evaluation you would write:

Er erledigte seine Aufgaben stets zu unserer vollsten Zufriedenheit.

To give grade B you would just change one word slightly:

Er erledigte seine Aufgaben stets zu unserer vollen Zufriedenheit.

Both translate into English as:

He always completed his tasks to our complete satisfaction.

The difference is that in the first sentence the word complete (vollsten) is the superlative which makes as little sense in German as it makes in English and is grammatically incorrect.

Grade C would be expressed like this:

He completed the tasks assigned to him to our complete satisfaction.

Here the word always is missing.

Another positive example would be:

His behaviour towards superiors, colleagues and customers always set an example.

Here the order of superiors, then colleagues and then customers is vital.

EDIT: I guess the first example could be translated literally like: "He always completed his tasks to our completest satisfaction." I didn't write that above because I think vollsten in German doesn't sound nearly as bad as completest sounds in English, at least to my ESL ear.

"most complete" instead of completest sounds fine.
I like the examples to illustrate both sides. I think if you had to always be positive in American you would say:

A: "He always completed his tasks to our complete satisfaction" ("complete" being the superlative)

B: "He always completed his tasks to our satisfaction" ("always" emphasizing the consistency of our satisfaction)

C: "His work was satisfactory"

F: <something that omits any mention his work>

Although B sounds a little awkward to me.

Alternatively,

F: "He makes an effort at completing his tasks."

“He strives to complete his tasks to our satisfaction”
full and fullest serve for quite natural substitutes for vollen und vollsten i.e. full satisfaction
Those are some very subtle hints, would anyone reading these letters really analyze and interpret them in such detail? I do understand the German, even if it's not my first tongue, and i can tell the superlative is obviously stronger in the first version, but couldn't that difference equally be attributed to personal writing style? Compared side by side it's easy but if given just one how would you know?

From my experience a CV is usually skimmed through very quickly because you have a huge stack to process, so picking up on these hints would require significant effort and understanding of who wrote it.

Most of those phrases and their associated "translations" are common knowledge, so it's really just a hidden rating system with bad reviews embedded in everyday positive language.
(comment deleted)
I was always told that in the UK it's illegal to give a negative recommendation (in itself an oxymoron, and who gets to define 'negative'?).

The way around it is that employers can decline to make any statement at all, which is then taken as a negative.

The good news is this advice is completely incorrect. According to the government website advice: “a reference must be fair and accurate” and can be challenged.

Source: https://www.gov.uk/work-reference

I was always told it's perfectly legal, but writing anything negative opens the path for future legal headaches. It's less hassle to only confirm the dates worked.
Interesting - I'll file that under 'common misconception'

From the former employer's point of view not giving a reference is probably a better use of time, unless they especially wanted to torpedo someone

> in the UK it's illegal to give a negative recommendation

This is not true.

I understand this... but god is it fucking stupid. This is literally the epitome of what taking away free speech does.

Nothing is fucking true and humans are god damn stupid.

To the extent this is accurate, it has to lead to misunderstandings and accidents.

In your examples, it sounds plausible, but if you are suspicious (and surely everybody reading such a thing is) you could interpret any statement as expressing something negative in a positive way. Ultimately it's more like a Rorschach test for the reader.

So much for freedom of speech, I guess. That sounds awful.
Germany doesn't have free speech, at least not by American standards.
Not even America has free speech by American standards. E.g. you can't say "fuck" on TV and you can only create and sell pornography under certain circumstances.
> E.g. you can't say "fuck" on TV

Broadcast TV, and it gets messy when it's a fleeting expletive. And you can wear a shirt saying "Fuck the Draft" in a courthouse[1].

[1]: https://en.wikipedia.org/wiki/Cohen_v._California

America has a lot of problems, major ones. But there are glimpses of hope, truth, justice and fairness that still resonate in the ethos of America - thanks for sharing.
Pretty much my point (I lived in Germany some years ago, although born and lived most of my life in the US).
>In Germany (and Austria AFAIK) employers are legally obliged to write letters of recommendation for their employees. These letters must be true, complete and favorably. So we have a similar situation where employers want to express their opinion but are required to be benevolent.

Does it actually even happen often that people who leave jobs under hostile or negative circumstances ask for recommendations?

Personally, i'd never even think to ask for a recommendation from the few jobs i've left or lost under bad circumstances. I only ever use references from jobs where i've parted amicably anyway. Do other people actually try and get recommendations from places they were fired from or left because of not so good reasons?

I may be wrong but I think here (BC, Canada), employers aren't allowed to give negative recommendations, I don't think they're required to give one though. I've never checked, but this is what i've been told over the years, so I might be wrong about this part.

Afaik a recommendation letter from your previous employer is practically mandatory to provide in any application in Germany.
> Does it actually even happen often that people who leave jobs under hostile or negative circumstances ask for recommendations?

It's not called a "recommendation". It's "Arbeitszeugnis" which translates to more of a testimonial or certificate making it a kind of must have. Not having it would be like finishing school without having grades. So yes, it's especially a big topic in hostile or negative circumstances as it's (even the wording) usually part of an agreement at the labour court.

My favorite of this ilk: “His attendance is good.”
Meaning?
The person shows up to work. If that’s the only nice thing you have to say about an employee, that says something :)
Why is there a law that legally obliges employers to write positive reviews? How is such a law implemented and what must be the reasons for supporting such a legislature?
The judgement (Urteil vom 26.11.1963 VI ZR 221/62) which dates back to 1963 says that the employer needs to show general "good will" in issuing the work certificate.

It doesn't say that you are always entitled to get the best one though because another judgement says that the certificate needs to be the truth.

There are thousands of lawsuits every year because of that. In the end the employer needs to bring up solid proof for wrongdoing to justify a bad certificate. So usually, if you get an job applicant with a bad work certificate, you can be sure that he REALLY messed up on his last job. Unusually it would probably be better for the applicant to say he/she has been unemployed than show up with a bad certificate.

I always asked for an English version of the letter in addition to the German one. It then loses all the nuances and just looks nice.
It makes you a slave.
i am happy to give a 5 star rating if all my expectations were met. That doesn’t mean my words will be flattering to the property or host. Actually i give 5 stars because it delivers on its promise but if there is feedback on the property i add it constructively. Uncomfortable beds, not safe for children are all things other people might find useful. No if all properties have 5 stars there is no value it forces us to read recent comments and be wary of properties that don’t have recent comments.
Any review I write on any of these platforms will blandly positive, anodyne, uninteresting. I understand why that doesn’t work at scale but I’m not a practitioner of radical honesty, and honesty is not incentivized. Unless something is really egregious, 5 stars every time.
I’ve always found the reviews useful. Never had a bad experience. Reviews will mention downsides even if five stars. And will be superlative if the place is great.

They also between 4.5-4.9, with those at the upper end better. Review volume is indicative too.

Have stayed in dozens of airbnbs, always had a great stay.