At least Uber stars are about something that happened, not something hypothetical. This point:
As you can see, Dan’s 9 NPS data points vary, from 5 up to
10. What this data doesn’t tell us is whether the
respondent ever did what the question asked. We don’t know
if they recommended the company to a friend or colleague.
alone would be enough to discard NPS, even disregarding the kinda moronic 11-point scale reduced to 'upvote-downvote' via arbitrary cutoff
The article ignores that sometimes, feedback is necessary.
I had a Lyft ride recently where the driver drove a car I believed to be unroadworthy, deliberately ignored the sat-nav adding 10+ minutes onto a short journey, and deliberately ran a red light (possibly because they didn't want to brake moderately hard, due to the aforementioned unroadworthiness).
Now, I'm all for not trying to interfere with someone's livelihood, and have given quite poor Uber/Lyft drivers 5* in the past for that reason, but this can also sometimes a necessary feedback route. Trying to prevent this either via guilt, or because of this idea that engaging with the feedback system lets companies minimise other forms of better management, is ultimately preventing legitimate and sometimes very necessary feedback.
I think the article is more about the fact that 5* covers every experience from "slightly below average but not enough that I want to crush my driver livelihood" to "This was the better ride of my life, bar none" and everything in between.
Doing so, it's mostly useful to employers and not to users. The only case where it's useful to both is the case you describe, the obvious 1* ratings.
Rating every single ride on a transportation service is nuts. It's a burden on the client to have the feeling that he needs to rate and it's a stress on the driver.
If there's something wrong with the ride, like tardiness, inefficiency, bad vehicle or filth you should be able to report that explicitly. That way the driver can learn and improve. When the complaints exceed a threshold, there's a consequence.
The popularity contest with the stars is meaningless and stressful.
To be fair, the system works pretty well on the whole. One of the reasons I switch to Uber/Lyft because of the extreme attention on customer service. Drivers are very motivated to keep customers happy. That's a big difference to cab rides.
I've had many one-star cab rides (joy ride several times as long, verbally abusive, etc.). I've only had one bad experience on a Lyft/Uber. But the quality of the typical experience is much higher too.
It's imperfect, but I think the problem is more with:
* unreasonable performance thresholds;
* misuse of statistical significance;
* use of data in isolation; and
* misuse of data
The other issues described mostly wash out after a few hundred rides. On the other hand, automatically firing people who fall below 4.6 stars without so much as a conversation is a little bit insane. On the other hand, many human managers do things which are insane too; nothing's perfect.
On the whole, eBay has been pretty reliable for me due to ratings, but I did get cheated twice because of similar misuse.
In one case, I ordered a premium item for about 50 bucks. They sent a low-end item which costs about 10 bucks. It wasn't obvious; for the most part, people wouldn't realize it until months later. Most ratings were high, with maybe 1 in 100 people pointing out the item was not as advertised.
eBay wasn't concerned. Credit card required an expert appraisal. Federal trade commission doesn't deal with this sort of thing. Seller appeared to be raking in about $500k per year profit on fraud. After no one cared, I decided to follow eBay's, credit card's, and FTC's lead, and say that for $50, it wasn't worth my time either.
But I stopped shopping for anything which could be forged on eBay (chemicals, fabrics, materials, jewelry, SD cards, etc.).
If eBay combined normal mechanisms with ratings, it'd be pretty easy to stay reliable. Alibaba does this -- there's a real conflict resolution process.
When you go to a restaurant, most of the time you probably leave a normal-size tip, unless something has gone terribly wrong during your visit. On ebay, a seller with a rating of less than 95-96% is probably not trustworthy. Most of the time we're not really grading, it's more of a pass/fail system--either things went more or less as expected, or it was bad.
After every service at my Honda dealer the service writer reminds me that if I get a survey from American Honda anything less than a 10 rating in any category is a failure.
My reaction to that might be to rate the service at zero and then get my servicing done elsewhere accompanied by an email to Honda complaining about it.
Well, rating someone 4/5 is only a problem if everyone else rates 5/5 for merely acceptable service. So perhaps we should evangelise for lower ratings, rather than squeeze everyone into 5/5?
It's in this vein that I rate 3* as a base (get me from A-B without crashing or similar incident) and go up/down from there based on the overall experience.
There's people in this thread stating they've given drivers who've provided bad or unacceptable service 5*, just because. It is, as you state, this behaviour which is the root of the problem in these systems.
I feel like this might be a generational thing where the expectation was that 'C' meant average and 'A' means going above any beyond. The rating system I'm used to is something like Randal's https://xkcd.com/1098.
I'm sorry, someone else's psychosis (the bizarre interpretation of a scale) is not my problem and I refuse to adopt their derangement. You'll get an accurate, honest rating from me, or none if that's an option. Otherwise I'm being coerced in to a lie and I don't deal with coercion very well.
Holding a gun to the service workers head and claiming that I'll kill them if I don't rate them FANTASTIC!! isn't an argument.
> The rush to rate every experience and interaction has much the same effect. The ever present manager monitoring your output is a reality and we’re all involved.
I'd say the biggest problem with relying customer ratings is that once customer's learn how valuable they are, some customers will exploit it to try to get more than they deserve.
Without consistency the ratings are pretty worthless. You need someone who will honestly rate the employee based on the value they provide to the business, and the business is the only entity that can do that.
20 comments
[ 2.6 ms ] story [ 59.4 ms ] threadA good takedown was written about the Net Promoter Score[0], but the principles are generally applicable to any one of these idiotic schemes.
[0] https://articles.uie.com/net-promoter-score-considered-harmf...
I had a Lyft ride recently where the driver drove a car I believed to be unroadworthy, deliberately ignored the sat-nav adding 10+ minutes onto a short journey, and deliberately ran a red light (possibly because they didn't want to brake moderately hard, due to the aforementioned unroadworthiness).
Now, I'm all for not trying to interfere with someone's livelihood, and have given quite poor Uber/Lyft drivers 5* in the past for that reason, but this can also sometimes a necessary feedback route. Trying to prevent this either via guilt, or because of this idea that engaging with the feedback system lets companies minimise other forms of better management, is ultimately preventing legitimate and sometimes very necessary feedback.
Doing so, it's mostly useful to employers and not to users. The only case where it's useful to both is the case you describe, the obvious 1* ratings.
If there's something wrong with the ride, like tardiness, inefficiency, bad vehicle or filth you should be able to report that explicitly. That way the driver can learn and improve. When the complaints exceed a threshold, there's a consequence.
The popularity contest with the stars is meaningless and stressful.
I've had many one-star cab rides (joy ride several times as long, verbally abusive, etc.). I've only had one bad experience on a Lyft/Uber. But the quality of the typical experience is much higher too.
It's imperfect, but I think the problem is more with:
* unreasonable performance thresholds; * misuse of statistical significance; * use of data in isolation; and * misuse of data
The other issues described mostly wash out after a few hundred rides. On the other hand, automatically firing people who fall below 4.6 stars without so much as a conversation is a little bit insane. On the other hand, many human managers do things which are insane too; nothing's perfect.
On the whole, eBay has been pretty reliable for me due to ratings, but I did get cheated twice because of similar misuse.
In one case, I ordered a premium item for about 50 bucks. They sent a low-end item which costs about 10 bucks. It wasn't obvious; for the most part, people wouldn't realize it until months later. Most ratings were high, with maybe 1 in 100 people pointing out the item was not as advertised.
eBay wasn't concerned. Credit card required an expert appraisal. Federal trade commission doesn't deal with this sort of thing. Seller appeared to be raking in about $500k per year profit on fraud. After no one cared, I decided to follow eBay's, credit card's, and FTC's lead, and say that for $50, it wasn't worth my time either.
But I stopped shopping for anything which could be forged on eBay (chemicals, fabrics, materials, jewelry, SD cards, etc.).
If eBay combined normal mechanisms with ratings, it'd be pretty easy to stay reliable. Alibaba does this -- there's a real conflict resolution process.
It's in this vein that I rate 3* as a base (get me from A-B without crashing or similar incident) and go up/down from there based on the overall experience.
There's people in this thread stating they've given drivers who've provided bad or unacceptable service 5*, just because. It is, as you state, this behaviour which is the root of the problem in these systems.
5/5 - No problems worth mentioning.
4/5 - Minor problems.
3/5 - Major problems.
2/5 - Unused.
1/5 - Unused.
Holding a gun to the service workers head and claiming that I'll kill them if I don't rate them FANTASTIC!! isn't an argument.
"Please rate our new iOS App" → no bonus for you
"Please rate our new redesigned webpage" → no bonus for you
"Please take a customer satisfaction survey on our new product" → No promotion for you
"Rate me on elance/upwork" → no more work for you
This exact problem was explored in a Black Mirror episode, where every interaction was rated with a star system: https://en.wikipedia.org/wiki/Nosedive
Without consistency the ratings are pretty worthless. You need someone who will honestly rate the employee based on the value they provide to the business, and the business is the only entity that can do that.