And not only that, but also removing their existing collections of videos and replacing it with their "Made by Netflix" shows that mostly amount to shovelware.
I'm about ready to drop them, just like they've dropped MASH, soon to be X-Files, and many many movies.
Same. They dropped Peep Show and that was about the only show I Liked that was still on. Plus King of the Hill and Spongebob, but they took those off ages ago.
Lots of assumptions, presumptions, guesses and maybes here, but:
- Netflix's licences to stream shows are presumably for a limited period of time.
- There's only so much money in the pot, and they want to spend it on the renewals and new acquisitions they think mostly likely to retain existing subscribers and attract new ones.
- Sometimes the rights owner may not want to renew Netflix's licence: they may strike an exclusive deal with another platform, or may feel that having the show on Netflix is cannibalising their DVD sales.
They originally negotiated licenses super cheap as the studios thought they were just some niche upstart. Now they're eating the market the studios are realizing their mistake and either not renewing licenses or jacking the rates up massively - hence the move into original content which can't be taken away.
I read something (on HN actually) awhile ago about how Amazon seems to be actively trying to kill Netflix, not by going after subscribers but by going after licenses for programming by over-bidding.
I think Netflix though is making more off of their in-house produced content these days so I wouldn't say they're in danger, they just might look quite different in a few years.
I use amazon channel instead of Netflix now. I subscribe to channel for a while and watch their content exclusively till I am ready to move on. I am currently watching the excellet Xive Tv channel on amazon [1].
Netflix is waay overhyped for what they actually have in their catalog.
They used to have more. As they've built up their own programming they've been losing a lot more of the other content, but like I said, I think they're doing it out of necessity.
> but also removing their existing collections of videos and replacing it with their "Made by Netflix" shows that mostly amount to shovelware.
You are not wrong
"US Netflix offers only 31 of IMDb's top 250 movies, study shows"
"David Wells, Netflix’s chief financial officer, was quoted by Variety stating that the company wants half of its content to be original productions over the next few years." [1]
Netflix vs non Netflix content aside, the low amount of movies is a direct result of viewing data. TV shows are shorter so can be squeezed into smaller spaces for viewing and have longer plot lines to hook viewers. Can't blame them for putting there money where people waste their time.
They're becoming their own content creator...like HBO, and thus reducing their dependence on often hostile networks for content and able to offer more content to more international markets, there is nothing wrong with that.
I find myself watching more of the stuff they've produced then the licensed content, so for me, it's been an improvement. If I run out of things to watch, maybe I'll care then, but at least I'm not tied to a cable contract, I can just switch to another streaming provider until they fix that issue. At this point though, I can't keep pace.
Why do so many not seem to understand that, in most cases, Netflix is likely not "dropping" anything? Netflix does not own the rights to the content and the owners can decide on the terms of the agreement on how long the content will be on Netflix.
People threatening to leave, or actually leaving, lowers Netflix's leverage to try to get and keep the content in their system.
This is why Netflix is creating their own content and bringing in content that's not normally within your area. So they can keep it up longer and have more control over their own destiny.
I really don't care how they run their business. Not my problem. What is my problem is:
I started watching X-Files for the first time (yeah, never go around to it for the initial airing). Started watching in late February. And starting March 1st, is a warning on the upper left of my monitor saying "Show will be removed April 1".
From my viewpoint, I'm paying the same, and getting less. Not only less, but specifically something I'm trying to watch.
And yes, I tried watching that psychic alien abducted Netflix show. The sex scene at ep1 annoyed me, and had no interest still in ep3. Boring is putting it mildly.
I watched until the T-1000 showed up. Can't see him as an organic lifeform. But in all seriousness it's a huge UI improvement that they give you a few weeks heads up now. I've narrowly missed the ending to some damn good shows and all I needed was a few days to get to a spot with some closure. I get that the rights aren't entirely in their control, so at least it's something.
I'm just pointing out that such complaints are aimed at the wrong target, that's the way it shall be with any service as long as people don't complain to the companies that control the rights to such content.
Sure, that makes sense, but as a consumer, all that matters to me is whether they have the content I want or not. If they don't, Amazon will gladly sell me the season DVD's that Netflix hasn't been able to host. I'm not personally interested in 'investing' in Netflix creating their own content (which I generally haven't been that into honestly).
I'm cool with that, I'm just bothered by the constant blaming Netflix for something that is not of their creation. These problems will exist with any streaming service as long as people point their complaints to the wrong companies. People should complain to the content rights holders, not the streamers.
It's just as possible for the content owner to tell Amazon to stop selling their content if they wanted for some strange reason.
Yeah I think anyone who's been following Netflix should know that keeping content they didn't create available for streaming is actually quite difficult and not just because 'Netflix sucks' or something.
I'd hardly call most of their shows shovelware. A great deal of them are at least decent or comparable to top shows on TV. Also, you do realize they don't set prices on licensing and probably drop the shows after people already went through them or stopped watching them. If you like a show, watch it while its still on there, don't put it in your queue for 2 years then get mad when it disappears before you got around to it.
I support Netflix out of philosophy. I want to support media which is offered without restriction and most importantly without advertisement. Netflix goes out of its way to make sure it's as watchable as possible (sans being able to download their stuff without DRM, but I can't blame them for that, it's too risky for their business).
I have similar reasons to support HBO. However, I will not send money towards FOX/Comcast/Disney/Viacom because they still want to restrict their content and make it as hard as possible to watch it.
Of course, the danger is they could just start introducing advertising again by way of product placement in the story itself, but if it goes too far, I guess I'll cancel then.
After you hand Comcast your money, you can also watch their content on your phone or tablet, anywhere, both live and on-demand. You can also use the phone or tablet to access what's on your DVR. Comcast is actually doing a good job of making it quite easy to watch the content on any device.
I haven't looked it up in a long time, but I don't think it's as easy as Netflix. Also, I want zero ads, not just the ability to stream.
It's a business model problem, and the entertainment model media providers offer is just not comparable to what is available on the internet.
Currently, I don't think the interest of Comcast's shareholders is the same as mine, they want to protect and grow their existing revenue streams, which depend on ads and outdated licensing contracts. My interest is to watch what I want, when I want, with as little friction as possible. Netflix and HBO provide that, none of the other big media companies do.
Also, Comcast has an interest in opposing net neutrality and providing a shitty internet experience so as to support their dying ad and TV business, so I also don't wish to support them for those reasons.
Bummer. I understand why they do it but I think it's too coarse. I'd much rather know how many people thought a movie was fantastic and not just decent.
The current rating system is not a running average of how all users ranked the movie, but represents users with similar taste to yours. The same move could show a different rating for different users.
It would be neat if they gave each user a set number super thumbs (obviously with a better name). You can use them so many times per month or year. I think month would be ideal, but if they went with the year route, make it possible to remove your super thumb like to use it on another movie.
On the contrary, I think that superthumbs would be a brilliant name. If you introduce something as dry and complicated as a quota for strong upvotes, you'd better do anything possible to make it more playful. A certain dosage of silly naming might be a good start.
You're forgetting all the other metrics that Netflix collects, like which shows get binge watched, finished, at what episode are people hooked or more or less guaranteed to finish a show after, etc.
The problem is that everyone has a different idea of what the 1-5 scale means. My girlfriend never gives a 5 except for her favorite movie since nothing can match its greatness.
Meanwhile I rate everything 1-star vs 5-star.
So I'm not too convinced there's any nuance in star averages, much less a difference between "decent" and "fantastic". It might just be a more divisive movie that averages out to a 3-4.
Rating is a hard problem. No system works universally well. IMDb (Amazon property) for example uses 10 stars, Amazon uses five stars. Facebook use thumb up ("likes"). Games ratings are often in percent 1-100% (summarized by metacritics.com and others). School systems around the world use A, B, etc or numbers like 1-5 or 1-6 for grades.
You're lumping together different kinds of rating systems. Netflix just wants to recommend you stuff you might like, which is a very different problem from scoring things on an absolute scale (grades, IMDb, game critics). Facebook likes have a social interaction component that's not there in Netflix, so it isn't comparable either.
Netflix lacks the issues that lead Facebook not to implement a thumbs-down. Netflix should still want a thumbs-down.
You should also distinguish between aggregate ratings and individual ratings. Metacritic and other aggregators normalize to some scale, but not all the individual reviews use the normalized granularity. A common movie case is Metacritic having to normalize a 5-star, in half-star increments, review to a 100-point scale. I believe Metacritic also allows user reviews to use a 10-point scale, but present the average on a 100-point scale (the first ten integers but with one decimal point, technically allowing 101 ratings from 0.0 to 10.0). Aggregate ratings always have more detail than their constituent parts.
I can safely say up/down voting is pretty terrible across all social media. You end up with HN and Reddit: popularity contests. Meanwhile, popular movies are rarely the ones worth watching, and the top 100 movies on metacritic is a grab bag of (great) movies that probably don't fit your taste profile.
Meanwhile, I have to like all the things the same amount. This is just going to lead to me not rating the vast majority of mediocre content on the platform.
But the UX of the current rating system sucks. Rating stars are very small and you have to hover them to know you can rate. Only if you watch a movie to the very end you will see a clearer rating functionality, but even then you have few seconds to use.
I bet many new users don't even know they can rate. I wonder if people aren't rating much because the UX sucks, not because it's a 5-star system.
I believe what they display is just a guess at how much you will like it (based on people with similar viewing/rating histories), not a public rating as such, so once you've actually rated it, there's very little reason to show the other figure.
I don't believe you can ever see a public rating. Before you rate a title, you can see netflix's prediction of what you will rate it, but that's not public.
That's very rare cases. Cinema is the most unhuman conveyor of entertainment content consumption. Modern movies are nice looking moving pictures generated based on big data and emotion exploitation.
I think it's a great move. Maybe on a similar note, one of the reasons I really like and find more use out of Foursquare is the way their rating system works compared to the five-star Yelp rating system. For example, in Yelp I found that all "good" restaurants are in the above 4-star rating, and that's all the information I can get out of that rating. With Foursquare, they have turned the upvote/downvote/neutral rating into a number between 1-10 that tells me more. Above average is 7+, something unique about the place 8+, truly exceptional 9+.
It just seems to me it's a simpler way to rate something by a user, and at the same time classify it more adequately.
If you have many reviewers, you don't need to get more information than thumbs up or down from each person. How much people thumbed it up is a good indicator.
Star-rating can be too much detail, anyway: if you're comparing two shows, and one has a higher fraction of five-star ratings, but the other has a higher fraction of ratings that are four or above, which is better? Star ratings can be too much detail and cause confusion.
If you want more detail from each person, you can ask specific questions with a yes/no answer, like, "Were parts of it boring?" or "Was it violent?" That's probably better than star ratings.
That and Netflix has far more data that indicates likability to a far greater degree, such as how many people finished watching the show, over what period of time did they finish watching the show, after what episode are people more likely to get hooked and finish the series or binge on it, etc.
Haha yes! Where the hell do these people learn that??
The other thing that annoys me with ratings in general, specifically places like amazon, is when people pan a perfectly good item because of some third party factor.
"Item was great, but it came in a beat up box from the post office, I'm giving it one star."
WTH does someone drop kicking your box have to do with the product?
I have to think that some of those reviews are just trolls.
Bad reviewers will always be part of the system. You have to expect it. Netflix has tons of reviews that go "I turned this off before the opening credit sequence finished, worst movie ever!!!! -- 1 star".
IIRC there was a blog post on HN a while past that complained about it, and it seemed to be people that went with 1-star so that their review was potentially more visible.
The other thing you see in ratings: "I've patronized this business every day for 5 years and always loved it. Then the service was kinda rude the other day. 1 star. Can't believe they stay in business."
Very true. There could be socio-cultural reason as well. If a movie hurt the sentiment of a particular section of the society, you can expect 1-star ratings even from people who weren't going to watch the movie anyway.
Haha totally agree. I've often wished they would put the actual decimal rating (4.12, 3.56 etc.) because we have no idea what their rounding strategy is. Does 3.99 become 4? How about 3.75? 3.51?
I do find IMDB's ratings breakdown fascinating to explore, though.
For example, you can look at the 50 Shades films or A Dog's Purpose or (especially) Gunday and have a pretty good idea that lots people are giving them bad ratings without having seen the films. (Their weighted average ratings are supposed to combat that, but they don't seem to be doing a very good job.)
Interesting point. Would polarised reviews make you less or more likely to watch a film (or download an app or whatever)? In other words, two items with the same average, but different distributions.
Actually, Netflix is poised in a better position to filter between people who just rated the item vs people rating it AFTER viewing it (for a minimum duration). The latter class of people will have have more weight attached to their voting.
That's true, but it doesn't sound as fun to poke around in.
(Obviously, "collect a dataset that philh finds interesting" should not be a business goal of Netflix. I'm not suggesting this is a reason to keep the star ratings. It's just a thing that I like about having ratings more detailed than yes-no.)
The goal isn't to broadly determine which movies are good and bad; the goal is to make accurate predictions about which titles a particular person will enjoy.
One good way to accomplish this is to pool users into cohorts of similar tastes, then suggest movies enjoyed by their cohort but not yet watched by a given user.
The question is whether doubling the frequency of rating a title is more valuable than a slightly more nuanced understanding of what each person likes. 'More valuable' meaning they can build you a better cohort from which to make title suggestions.
Everything you've said makes sense, but I don't see it being contradictory to what I said.
If you decide you want a more nuanced understanding of what a given reviewer likes, I think it's better to evaluate different dimensions (Was it funny? Was it violent? etc) than shades of grey* of the same one.
> The streaming service said it had been testing thumbs up and down ratings “with hundreds of thousands of members” in 2016 – and it led to 200% more ratings being given.
I don't understand why engagement is the right metric here. If someone isn't sure how they feel about a movie, why is it a benefit to have them spew their half-formed thoughts into a like/dislike rating?
Seriously. Lots of low fidelity measurement is sometimes better than a smaller amount of higher fidelity measurement. Especially when that higher fidelity measurement has errors.
Here's the benefit - getting a clearer picture of your tastes, which allows them to make you better recommendations. Their recommendation engine would be based on finding users similar to you and telling you what they liked. More ratings is better than nuanced ratings for this engine because you're more likely to find overlap between users.
The goal isn't making a database of the "best" and "worst" movies and TV shows. We already have the International Movie Database for that. This is just about finding out what you like, and finding other stuff like it.
>If someone isn't sure how they feel about a movie, why is it a benefit to have them spew their half-formed thoughts into a like/dislike rating?
That's pretty harsh.
I've watched plenty of movies that I enjoyed and would give a thumbs up but aren't sure whether it should be a 3 or 4 star movie. There's movies I love but consider too flawed to give 5 stars. And I find little difference between 1 and 2 stars other than the level of regret for wasting my time.
Overall I rarely bother rating things on Netflix because it just doesn't feel worth it. So they get nothing out of me, whereas with a thumbs up or down I am more likely to chime in.
Thumbs up/down also misses an important metric, for me: the "meh" factor. Maybe something sucks, maybe it's great, but what if it's like...I have nothing to do, I guess I'll watch this show.
You could say "well that's the same as not rating it!" but it's not. I'd like there to be a clear distinction between "I abstain from voting" and "I thought this content neither vomit-inducing nor life-changing."
What I really want to avoid is the mediocre content i.e. 2.0 - 3.5 stars, very good and very bad can both be entertaining..
Now that mediocre content will be scattered across the two categories I really care about; I guess IMDB/Rotten tomatoes will steer clear of this ridiculous binary rating systems.
Yes it is, and it sometime gives some strange ratings like "Ash vs Evil: season 2" having a 100% rating[0]. Though it's probably a good show, it's not absolute perfection like the rating would imply.
Aha, I misunderstood, I thought Netflix was going to recommend movies or not based on that rating system, without giving us the background. I would still prefer something more fine grained though.
I've seen so much good content on Netflix rated at 2 stars or less. Movies and TV shows are way too subjective for user reviews to be used with any kind of reliability. Or maybe I just have low standards.
A problem I'd like to see a good solution to is selection bias. Unless there is some kind of reward, only people that really care are likely rate something
I think Netflix's move might help this. It certainly lowers the cost rating.
There's another type of selection bias as well. I'm not going to watch a movie unless I think I'm going to like it (based on reviews, comments, description, etc.) As a result, with the exception of a few real disappointments, I'm generally at least sorta OK with most of the movies I watch.
Even most of my disappointments tend to be in the vein of "It was OK I guess but I don't see why people think this is so great."
Therefore, any movie you've watched more than once becomes pure gold for Netflix! Spread that across several million subscribers and get enough data to be useful.
From the linked Variety article, "However, over time, Netflix realized that explicit star ratings were less relevant than other signals. Users would rate documentaries with 5 stars, and silly movies with just 3 stars, but still watch silly movies more often than those high-rated documentaries."
But the signal isn't just for Netflix; it is also for users, who might sometimes be in the mood for something silly and sometimes in the mood for something good. Also, people might rather get more suggestions of good movies even if they are more likely to watch bad ones. (Of course, people might also just overrate documentaries.)
But I want Netflix to surface good documentaries as well as good silly movies. I'm more likely to watch a silly movie, so with this change my queue is more likely to be filled with silly movies, making me more likely to unsubscribe because it appears that Netflix only has silly movies. I watch more silly movies than good documentaries, but I pay for Netflix mostly for the good documentaries. I can get mindless pablum elsewhere easily.
To me it sounds like they should instead use categories as hints for how to weigh ratings of different users for different categories.
I definitely don't want my feed to be filled with silly comedies all of a sudden.
I'd be pretty happy if netflix managed to actually suggest things there's a vague chance I'll be interested in rather than whatever they recently released/bought and desperately want viewers for..
It's a good thing netflix created original programming, because I would have long ago been gone from netflix if they didn't have /something/ nobody else had. Their nonoriginal selection has gone to total shit. Even hulu is competitive with netflix's selection excluding originals
Thumbs up or thumbs down is just as good of a signal for the users. Everybody has some innate star rating threshold where they won't watch a series or film. If it's below 3.5 stars I won't even give it a chance. Thumbs down accomplishes the same thing.
I'm going to rely on Netflix's recommendation engine to put things in front of me that I am going to enjoy.
The problem with that approach is that it assumes that liking is normally distributed.
I'm reminded of Jerry Garcia's famous quote about the Grateful Dead: "We're like licorice. A lot of people don't like licorice - but the people who like licorice REALLY like licorice".
A 3.5 could either mean 50% of users give it a 3, and 50% give it a 4, or it could mean that, say, 30% of users give it a 5 and 70% give it a 3 or less. How do you know you aren't part of the 30%? Great art is often polarizing.
I'll admit that I have had Netflix recommend to me movies that it thought I would rate at one star, and I have watched that movie and then rated it one star. Not only did their current system predict that I would watch that movie, they predicted my relative opinion of the movie's quality. And yet I watched the movie, and thus felt like their recommendation system served its purpose. I have to say that going into it expecting a one-star rating allowed me to set expectations for the film.
If they remove the axis of quality and merely opt for "will you watch this: yes/no", it will make it harder to select "good popcorn trash" versus "high-quality, high-enjoyment" to fit my mood.
I don't get it. Don't they already have the signal available to them of "did the user watch all/most of this video, or did they abandon it early" ? That seems like it does most of what they want, and thumbs up/down will add only very little information on top of that?
I wish there was a "meh" option. Sometimes things are ok, but not good or bad. This will leave me not rating a bunch of things unless they're good or bad.
Is this just part of the trend to dumb everything down to a polarising dichotomy? Because the trend to stop people from having a more complex and rational opinion than just picking sides doesn't seem to exactly improve discourse or debate, or inform and educate...
Wouldn't it be more beneficial to have fewer people give a more detailed review to describe what they like and dislike about something? What use is "good" or "meh" compared to for example "I rate this product low because the shoes are very narrow for my normal feet" so you can actual relate it to what you actually are looking for? One person's "meh" could be another's "good"...
From the article:
"This makes sense – giving a five-star rating takes some thought, especially for something like a movie or TV show.
A binary “yes or no” option is much easier for viewers to commit to, [...]
I think it's more a reaction to a usage pattern of their star system. YouTube abandoned their five star rating system citing that most people voted only 1 or 5 stars with almost nothing in between. [1] I'd guess a vast amount of Netflix users do the same.
YouTube is partly to blame for this. Since YouTube reported the average stars, if you saw a 4-star video and thought that it should be 3-star, then the optimal strategy was to vote 1-star to pull it down as fast as possible. If instead YouTube had reported the majority option (and publicized that fact) then it would have incentivized people to vote what they truly think.
Your suggestion has its own problems. Imagine two videos with a majority rating of three stars. Say 60% of raters have rated it that. But one video has the remaining 40% one-star, and the other has the remaining 40% five-star. Your algorithm will ignore this data, showing a completely wrong result that both videos are equally good.
I think one problem is that people have vastly different opinions on what 3 stars mean. For some it's quite bad, for some it's in the middle.
A binary option actually has three options. Did it prompt me to rate it as bad, or did it prompt me to rate it as good, or was it bland enough where I don't want to sway the rating of it at all?
If I rate a movie on IMDB as 5-6, and maybe even 7, the possibility of me recommending it to someone "depends". In that case I might as well not rate it at all.
Ratings are inherently tricky, because we all like different things. I don't think you lose anything by going to a like/dislike system. Recommendations should be based on aggregate maps and matching you with similar viewers that have a certain overlap in ratings. That's made easier with binary choices.
IMDB ratings are only partially informative to me anyways.
From my perspective, that shouldn't be a problem. Ultimately (as a user) the point of the rating is for me to get stuff that I like -- so it's all relative to me, regardless of individual rating differences.
Now, I realize that's almost certainly not the level of individualization offered by their ranking algorithm (I'm convinced there's way more of a "groupthink" dynamic), but that's partly my point: if your ratings and recommendation system isn't capable of supporting individual differences, then your algorithm is broken.
If you have a complex problem (personalized recommendations absolutely are complicated) then you shouldn't fix a broken algorithm (Netflix recommendations are, at least for me, a bit meh) by simplifying the inputs (switching from stars to up/down). You should fix the broken algorithm, by creating a better algorithm.
I realize it isn't that simple, particularly for a publicly-traded company whose stock price would absolutely be influenced by a decision to scrap part of their core technology, but from my perspective as a user, this is going to hurt my experience with the product.
I don't have Netflix, so I 'm not clear if the rating shown to users is changing. Or does it not show a rating, just use it to decide what movies to surface for you?
In any case, simplifying the ratings users can bestow makes a lot of sense, unfortunately. We like to imagine people carefully considering the merits and flaws like a professional reviewer, but the fact is that the vast majority of users only ever use two ratings anyway: if they like it, 5 stars; if they didn't, 1 star. There's maybe a third option where if they're ambivalent they just don't rate it.
So star ratings aren't actually very useful for evaluating products. This xkcd https://xkcd.com/1098/ made me realize that an Amazon product which has a 20% chance of exploding when you open the box, and otherwise works normally but unexceptionally, is gonna average out to a four-star rating for a product you really shouldn't buy. I always look at the one-star reviews before buying stuff now, no matter how few of them there are; knowing a thing's failure modes is much more useful than a bunch of praise.
I think a lot of people are reacting here as if Netflix is moving from the only signal they have being the star and review system to the only signal they have being the up/down system, and thus confused as to why Netflix would throw away the vast majority of their rating ability.
But that's not even close to true. In addition to it technically being up/down/no rating, we've got how long people watch the show for, how the shows they watch form a pattern, how all the patterns of what people watch show global preference patterns, whether they rewatch a show, when they watch what sort of shows, what individual scenes are rewatched vs. skipped... Netflix is swimming in a sea of preference data, not sitting here trying to figure out "Gosh, um, if the user likes this movie 3 vs. 4 stars, uh... what does that mean?"
It makes perfect sense to me to optimize this one-data-stream-among-many to increase user participation and get more bits of information from more people engaging with the simpler system, rather than trying to squeeze bits out of the few people willing to use the star system and the even fewer willing to write useful reviews.
It isn't really even as shocking as it may seem at first. The star system has 6 states, "no rating", 1, 2, 3, 4, and 5 stars. That's 2.6 bits, with some simplifying assumptions [1]. The thumb system has up, down, and no rating; that's 1.6 bits [1]. To make up for the bits, you need only see ~40% increase in participation over the current star system... think that's going to happen?
[1]: The simplifying assumption is that all outcomes are equally likely, but that's not true. I don't have the numbers to run a more complete information theory analysis, but it's not hard to imagine the "no star rating" case is so common that it produces such a small fractional bit that a higher-participation-rate yea/nay/no rating (if you puth this UI in their face, "no rating" becomes much more meaningful, too) straight-up produces more bits of information on average, and is thus simply an improvement even before considering the superior UI experience. I rather suspect this is the case, some very sensible assumptions would suggest this, but I lack the ability to prove this; the "assume all outcomes are equally likely" is at least a concrete case I can discuss.
Yes, but the ratings aren't just for Netflix to use, they're also interesting information for users.
This move seems very inward-focused: Netflix is thinking about the information it wants to get from users, but I wonder if they surveyed people about how much they want to see percentage-likes instead of stars.
Does Netflix offer star histogram charts the way Amazon does? I don't recall seeing them. If not, there's no difference to a user between a 6.8/10 generated by thumbs and 6.8/10 generated by stars. They are already synthesized numbers, not just an average across all users or anything.
> To make up for the bits, you need only see ~40% increase in participation over the current star system... think that's going to happen?
For some people, such as myself, it will decrease participation (making the "no rating" indistinguishable from "it was okay, but I don't love it and don't hate it, so I wouldn't know whether to upvote or downvote"). A 40% increase is also quite a significant change. I'd be astonished if they got that much more engagement out of it.
There's also quite a difference between having the data for their internal algorithms and what people want to see. Just like Facebook eventually added alternatives for just the "Like" button.
I don't mind having a percentage displayed rather than a star rating (I automatically convert everything in my mind anyway, even though the star ratings are more easily visible at a glance when scrolling through a list), but I mourn losing a lot of expressiveness in the ability to record what I really thought of the video that I watched.
Terrible idea, thumbs up or down rating only promote political reviews, a movie that makes critics of feminism? thumbs down despite the acting or contents inmediatly, a movie showing the good parts of religion? thumbs down by viceral atheists, some one just needs to look at youtube and see how this doesn't work,
I feel like star ratings, if just given in general, are pretty useless for anything that has multiple qualities. For example, maybe I like the cinematography of a movie but I think the acting is shit, or maybe I think the taste of the beer is nice but it's too acidic or sour and wouldn't work with specific foods because it's overpowering.
Airbnb does this pretty much spot on, in that they ask for 1-5 stars for five categories, rather than just "did you enjoy this Airbnb?" I also think that Untappd does this wrong, because in general beers can't be described in just a 1-5 star rating without deeper explanation.
I think that moving from a pure star rating to a thumbs up and down rating is better overall, if only because it makes me, as a watcher, not have to think as much and therefore give a rating where I might not have before. If I want to go more in depth, I can explain more too.
Incidentally, Steam ratings are also simple up/down, with total and sliding window 30 days aggregate, and an Amazon-style helpful/unhelpful/funny meta-rating.
Steam ratings have their own problems. Like the creators changing something minor and getting thousands of protest negative reviews from upset people. Which doesn't reflect the quality of the game at all. But does discourage new people from trying the game.
My favorite example is Fallout 4. There are tons of negative reviews from people who put 600+ hours into the game. How the hell do you put 25 days of your life into a game you don't like? They claim it "got boring". Not many games will not be boring after 600 hours...
> I feel like star ratings, if just given in general, are pretty useless for anything that has multiple qualities.
This is only true when those star ratings are taken out of context. Individual reviewers can have completely coherent star ratings that are very useful for making determinations.
The golden age of Netflix discoverability, for me, was back when they added social networking features. I connected my account with many friends and family and was really able to get a sense for everyone's taste in movies. When it came time to discover new movies/tv to watch, I'd open up the ratings history for someone who's tastes I felt aligned with the mood that I was in. Some of my friends would only rate high quality content with 5 stars, others would typically rate mass market, mindless entertainment with 5 stars and others tended to include a lot more 5-star reviews within a certain genre. Over time, I got a feel for that and could use it to my advantage.
But when you remove that individual context and only aggregate 5 star reviews across a massive population who all view the 5-star scale in their own way, you get that uselessness that you talked about. Some people may reserve 5-star ratings for truly extraordinary content, but Netflix doesn't weight those ratings any more highly than the 5-star ratings of someone who rates nearly everything as 5 stars. This makes the ratings that Netflix displays meaningless. But that's only because Netflix's current methodology makes them meaningless, not because they're inherently meaningless.
> Netflix doesn't weight those ratings any more highly than the 5-star ratings of someone who rates nearly everything as 5 stars.
Is that fact or speculation? It's a common recommendation technique to find the mean rating for a person, and use that to normalize. Normalized score = (actual_score - mean_score + 2.5) for a 5-point scale, for example.
I have no insight into whether or not Netflix does something similar, but I'd be surprised if they don't.
Doesn't normalisation assume that a person who rates everything five-star has low standards? It may just be that that person rates something only if they're really (un)happy.
Imagine two people watched 100 movies, and assessed them the same way, say that 20 are five-star, 20 are four-star, 20 are three-star, and so on. The first person rates all 100 movies, but the second one has a habit of writing a review only if he considers a movie excellent. Then your algorithm will wrongly discount the second person's ratings.
Is this true in the real world? Have you found different people to have different habits, like rating everything vs rating only good movies vs rating only bad movies?
First, few people will put in effort into understanding their friends' psychology and habits to use Netflix effectively. We shouldn't optimise for such users.
Second, I don't know whether individual ratings should be normalised as you say, but the final rating that's displayed should be normalised. For example, the app stores should have 20% of apps rated five stars, 20% four stars, and so on. This will make the ratings more objective and easy to understand. If the average rating is 4, then an an app rated 4 is merely average, which is counter-intuitive.
(Perhaps normalised within a top-level category, like Productivity and Games)
I like to use the French Laundry example for how meaningless the star ratings are. Was it the best meal of my life? Yes. Was having the best meal of my life worth $1000? Not even close. So, did I have a 5-star meal, or did I have a one-star money waste? And are my preferences at all meaningful to a random person who has no knowledge of my finances or desire for good tasting food?
Certainly the data you get will be of better quality that way, I just don't know that you'll get a huge quantity of it. Seems like for better or worse, Netflix is going for a higher quantity of ratings. (As well as the possible ulterior motive mentioned elsewhere in this thread, of making lame movies look not-so-bad - like helping out the dullards in class by grading it pass/fail.)
An exquisitely detailed system like Airbnb's costs the user more time & hassle. Which means, assuming you make it voluntary, that a lot of them skip it. Meanwhile a simple/easy system like this, turns a scalpel into a sledgehammer somewhat, but you get a lot of them. So there's a balance to be struck for your site's particular userbase.
I agree, that's a good point. I think it's a different cost/value proposition with Airbnb vs Netflix, since you paid for a, sometimes expensive, service and then when the service is rendered you can review it, whereas with Netflix, while you are paying for the service you're not paying directly for a movie/show. So, with Airbnb, since you invested a bit of money you feel like you should take the time and review, but with Netflix, if it's not too simple you may just skip the review.
I think that binary ratings are the purest form of rating. But you have to ask the right question. "Did you enjoy this Airbnb?" is not the right question, because you end up wanting to give a nuanced answer. But "Would you recommend this Airbnb to a friend?" is, I think, the right binary question because it forces you to distill all of your likes and dislikes into a single, unambiguous decision.
The nuance comes when you aggregate many people's binary decisions. A value near 1 tells you many people felt the good way outweighed the bad; a value near 0 tells you the opposite; and a value near 0.5 tells you that people are split. I think that's far more useful than trying to ask many individuals to provide you with a granular score.
That may be true from your point of view as a rater, but from mine as someone deciding whether to watch a movie, I want you to condense your thoughts down to a yes or no by assigning an appropriate weight to each factor. Was the acting so bad that it ruined the movie despite the good cinematography?
For example, I found Mirror Mask enjoyable and would recommend it. They dreamed up and brought to life a fantastical world full of magic and wonder. The visuals are excellent. The story is mediocre, so you should watch it with the right expectations, but watch it nevertheless.
The ability to distill something complex and multi-dimensional to a clear conclusion ("enjoyable and I recommend it" in the above paragraph) is a sign of clarity of thinking and expression. If you can't do that, I'll go by other people's ratings.
Remember that the goal isn't to accurately capture one person's opinion, but to help potential viewers decide whether to watch it.
>Users would rate documentaries with 5 stars, and silly movies with just 3 stars, but still watch silly movies more often than those high-rated documentaries
That's not incongruous to me. The stars are not about "enjoyment" factor, they are about perceived quality. I may have a go to cheap ice-cream and rate it 3-stars but rate a good affogato 5-stars and only have it once in a while.
They are diluting the meaning of quality and instead are opting for a saccharine "enjoyment" factor.
This binary choice does not sit well with me and I hope they abandon the idea soon.
When I watch netflix, I use the stars as a measure of if I would enjoy something. I don't think most people use stars as a measure of "quality" because when they browse movies, they are looking for something they enjoy, not something that's high quality that they might not enjoy.
This seems like pure difficulty for me. When I think of thumbs up / down metrics I think in terms of "would I endorse this?"
A movie like "Suicide Squad" was (to me at least) highly enjoyable, though I thought it objectively had many problems with plot and delivery. I would probably give it 3 stars, which is relatively neutral for me (or leave it altogether unrated). If it were thumbs up/down, I would probably have to leave it unrated completely.
Maybe that's an improvement, but the more discussion I see around this, the more I realize the futility of single-axis metrics for things like this, until and unless you're asking a single question "Did you enjoy this movie" is a much different question from "is this a good movie" -- and I don't really know which one I've been answering on Netflix all these years past.
I think you are an anomalous user. I'm not sure why you're OK with rating 1-5 stars but not a thumbsup or down. I think you're just upset because they are changing.
Lets say a movie previously would have shown 3 out of 5 stars. Where does that fit in with the binary thumbs up/down? I think they are just doing this because recently some actors movies are being attacked with 1 star ratings for being politically vocal.
I agree. I even want a couple more increments between 2,3, and 4. There are a lot of movies I see that I think are better or worse than a 3 rating but don't justify a 2 or a 4. I feel there's a lot of room for nuance around the middle area.
The current suggested rating is influenced by your history of rating, with weight given to how highly you rated it. There are shows that are suggested at 4 stars for my wife but at 2 stars for me.
This. And based on my history of rating, Netflix had been pretty good at guessing if something will be 3+ stars. If they think it is 3 stars, I consider watching. If they think it is 4+, I tend to really enjoy the suggestion. Though, nearly every Netflix original is suggested at 5 stars, lol.
I think a lot of people look at the stars as a measure of enjoyment when trying to find something to watch, but when actually rating something, as a measure of quality.
I use the stars to signal quality as a means of determining if I'd enjoy it. 5 stars? This is likely to be high-quality and enjoyable. 3 stars? This is going to be lower quality and I may not enjoy it. 1 star? This is likely shit quality, and I'll regret playing—or it's a cult classic and all the haters don't get it.
I use the stars to signal quality as a means of determining if I'd enjoy it. 5 stars? This is likely to be high-quality and enjoyable. 3 stars? This is going to be lower quality and I may not enjoy it. 1 star? This is likely shit quality, and I'll regret playing—or it's a cult classic and all the haters don't get it.
I've had this argument with my (tending towards elderly) parents while introducing them to Netflix. They don't understand rating for enjoyment, and keep rating for "quality". They'll watch a show, absolutely hate it, but give it a 4 or 5 star rating because "it was well produced and the camera work was good, but it isn't the sort of thing I personally like to watch".
I'm worried about the thumbs up/down, as Netflix has increasingly recommended shows I do not like this year, and I don't think the thumbs solves the "rating for perceived quality" issue. I've been using Movielens for recommendations a lot instead, and it seems more accurate for me... and it has enough stats to show that on their site, most rating is not binary, much closer to a normal curve for average movies or a linear graph on really good movies. (It lets you see the distribution curve on individual movies, and other metrics.)
I thought that Neflix made it pretty clear that the ratings were used to choose which content to show me. So I should only rate based on what I'd actually like to see.
Or edge cases like me will get that little boost to just give up on Netflix. I already use them less than other streaming services, and I could always just turn it on and off again.
Honestly one of the only reasons I keep it is because I have so many movies that I've rated and didn't want that time investment to go to waste. I guess now that isn't going to be a factor.
That was definitely part of it or me too; the rating system has always been a draw. It took time to make it actually respond the way I needed it to, and now it's just... sigh
My dream is to build an open source app that gathers all these personal ratings (from Netflix to Pandora to GoodReads to restaurants) and puts it back on where it belongs: the users hand!!
Unfortunately finding the time to work on this will be tricky..
Why on earth would you use Netflix as a rating service? First of all their library lacks a lot of great movies, or info about the movies they have. Netflix isn't IMDB.
If you include the DVD service they have practically everything. I've been rating movies I've watched in Netflix since 2005-ish. IMDB makes sense, but I chose Netflix and now I'm stuck with them. Data silos suck.
They're definitely not replenishing stocks of some DVD's, which I suppose makes sense given the trend. I would assume if they get a certain number of requests they probably do though?
Maybe? The thing is that the incentives don't really align to make this a real priority of theirs and they've been pretty clear that shipping physical DVDs is not their long-term business model. So long as a lot of people don't start canceling their DVD service because of unavailable disks, they don't have real reason to replenish.
Of course, given that they pretty much put every other rental place out of business, your only choice in a lot of cases is just to buy a disk if you want to watch something that isn't available streaming.
Almost all of the ratings I did were back before they even had a streaming service and the library was great. Plus it was helpful to get good recommendations. Now it doesn't really matter, but all of those ratings are locked in.
I've looked before halfheartedly for a way to export them but now that they cut the API off it doesn't seem like there's a great way to do that.
That is one reason I've always preferred to control my music directly. Music in particular is something I can't stand to have discontinuity of playlists and collections with.
I hear you. It's one of the few things that actually elicits a physical response me from... a kind of shiver of anxiety. I put time and effort and maybe even a little bit of myself into my music lists... it's personal!
I've rated thousands of movies and shows on Netflix, not just what I streamed or received from Netflix on disc but anything I've seen anywhere that I can recall and came across on their site. Having rated something is also a reminder that I've seen it. I didn't want to lose all that so I downloaded the pages [0]. I'll figure out how to scrape them and put them in a better format some other time.
I pulled mine from https://www.netflix.com/MoviesYouveSeen. I did notice a bunch that just said MOVIE but that was just a handful. My ratings page goes back to 2005 and I'm not subscribed to the DVD plan anymore.
What other services do you use? I paid for prime, so i've watched some stuff on prime video, but there's really not much on there. Netflix is pretty much my only option for most TV shows. Worth noting I'm in Canada.
If you enjoy Anime or Asian drama (asian drama being live action, not animated) Crunchyroll has a huge catalog. It fills the void that Netflix has between two big Netflix Originals.
Edit: Giving away one of my 48h Crunchyroll premium guest pass. No credit card required. I'm not working for Crunchyroll in any way, they give out guest pass monthly to premium users. First one to use this code wins: UWWJAQTDKYZ. Note that I will receive a notification telling me your username and perhaps your email address, if privacy is important to you.
I'm in the US here, so I'm not sure where out options differ. For me, I do (as a poster below noted) like anime so Crunchyroll is a good choice. Hulu is good for seasonal TV stuff, and then I actually use a smartDNS to get access to UK channels. It's probably a matter of taste though... I don't like most movies, and prefer longer-forms for telling stories.
I used to use a VPN to watch US Netflix, but they started to block known VPNs and I didn't feel like spending time on workarounds. On the other hand, downloading the latest episode of some TV series takes only a few mouse clicks. For sure easier than fighting geo-restrictions.
I still pay for Netflix, but doesn't really watch it much because of the tiny catalog available to me locally, I simply want to support them. Perhaps it will make me feel better if I would download any of their content. :)
Nothing to be sad about. Also curious, why do you want to support them ? Being a market leader, Netflix is a unique position to influence some of these regressive practices. More support is clearly not helping; it's only making it worse by creating monopolies that exert even more negative influence.
The biggest problem with Netflix is that it is not able to obtain rights for a lot of movies or series so you don't have a full catalog. So series and movies are fragmented between providers or totally unavailable.
Netflix is not much of a market leader until it contains at least 90% of the movies i want to watch, which can't happen without enough leverage from their part given by user support.
I do like some of their own content. I also like their general business model and I believe (well, I hope) that the stupid geo-restrictions are results of old-school licensing and not active business decisions made by Netflix.
But ofc, I wouldn't mind if they had some real competition, I don't want a monopoly.
>(well, I hope) that the stupid geo-restrictions are results of old-school licensing and not active business decisions made by Netflix.
Geo-restrictions have existed forever. What Netflix started was actively blocking entire ranges of IPs belonging to vpns and commercial ISPs (cloud providers like aws, dedicated servers like ovh etc.). This should fall under some corollary to net-neutrality where a service cannot discriminate between ISPs. Does such a corollary exist in countries that have some form of net neutrality (US, India, Netherlands etc.) ? Can someone with legal expertise comment ? From my standpoint, they can have geo-licensing, but should stick with some consistent way of showing content either based on billing info or IP. Blocking out vast portions of the internet because it doesn't suit them should clearly be some violation of laws like net neutrality.
Like many people I've had a Netflix account for a long time - I can't recall the last thing I watched on their streaming service. It has probably been four months since I've used it to watch a movie for example. Their library is increasingly complete trash outside of a few original content programs. I'd cancel it if it weren't so cheap, mostly I keep the subscription as an entertainment fall back (in the category of: well, maybe I'll watch something on Netflix, oh, nope). I suspect I'll just ditch Netflix in the near future and be done with it, as I'm pretty happy with Prime as essentially a freebie to my Amazon shopping.
I have to say, I quit Netflix a few years ago after a long membership, due to security concerns (it was a nightmare at the time). Recently I started my membership up again, and I'm really disappointed with their library. Every movie I've wanted to watch has turned out to not be available for streaming. They've got some good shows, but I'm getting tired of super heroes and I don't always want to watch television, sometimes I want movies.
I'm on the brink of cancelling my membership as it is, this certainly won't help, though it's obviously the proverbial straw rather than a big deal in its own right.
I agree with you. And hat tip for the nice analogy.
Perhaps they should implement a system analogous to facebook's Reactions (e.g. "disliked", "thought provoking", "easy watching", etc) as a way to differentiate between cheap/easy and high quality releases.
Yeah, if they went with up/downs but allowed genre up/downs I think this would correct some anxiety about this change. "Yes, I still want to watch anime. No, the last 4 shows you recommended to me were not good."
> The stars are not about "enjoyment" factor, they are about perceived quality
I ignore ratings because of that. They don't mean anything and I am guilty.
On Good Reads, I have given what is deemed as a trashy piece of writing 5 stars. At the same time, I have given Dune 1 star because the story was garbage.
So what is this 5 star rating? Some people say it is the experience, some say it is the writing style, others say it is the story telling, or the plot.
Lets also face it, when it comes to films, the news papers have their own agendas and personal commitments to other agencies. My film could be absolutely garbage, but since I dined with the Editor a few times, It gets 4/5 stars and then, it gets 4/5 stars because the rest of the population is told how to feel.
I would even say that this whole system of rating based on Stars or Likes can simply be removed. If a film is good, you will know about it. If a film has been watched 1000000 times more this weekend than another, base it on that if anything.
The problem as I understand it is that the "rating space" is multidimensional (a piece can be funny with great plot but bad writing compensated for by a great lead and awesome special effects although the set design was inconsistent) but all of this has to be projected into a 1-dimensional system.
And the rub is that not everybody uses the same projection, which means the utility of one person's rating for another unknown their projection system (the weight they give to each sub-component of the overall rating) is completely unknown.
Yes, the whole thing is meaningless but atleast views offer some glimpse. I would have given Barb Wire 6/5 as I was a teenager with a crush on Pamela. How can that relate to a rating of a film that you could use?
When I rate things it's usually an "overall" rating, rather than weigh one dimension disproportionately. Which is why I would almost never grant a 5-star rating to something. But I'll dish out 2, 3 and some 4s regularly.
That is only you, I can't use that information. Like you might be taking into consideration "Special effects were 3 on their own". Whereas my Mum would never even consider that as a thing which influences the rating.
I understand that. It's not perfect and I don't think the assumption is that it was but more that overall people as a group will smooth those rough edges. In other words the majority will compensate for the minority who use the ratings system differently.
For all the faults of the old system, the new one is even worse. Like Pandora, I'll probably just never rate things or mostly just give them a down vote if they don't crack a threshold of goodness.
Which is fine, because the recommendation engine should be using your history, not someone else's. They gave it a 5, you have it a 1. They will see recommendations for similar movies and you won't.
They should be optimizing "number of people who pay for Netflix". Instead they are optimizing "number of hours spent watching Netflix".
Those two metrics are correlated, but definitely not fully.
If I spend hundreds of hours a month watching meaningless action movies and TV on Netflix, I can cancel my subscription and save myself hundreds of hours and $10. I've got lots of other ways of filling my time with endless pablum.
If I spend 2 hours watching a thought-provoking documentary, I say to myself "yup, that justified spending $10 on Netflix this month."
> Users would rate documentaries with 5 stars, and silly movies with just 3 stars, but still watch silly movies more often than those high-rated documentaries.
Accounting for your average person, who do you estimate is more likely to cancel: The person watching hundreds of hours of meaningless action movies and TV, or the person subscribing but rarely/never watching anything? By average I mean not prone to suddenly realizing they want/feel sufficiently motivated to spend those hundreds of hours on something else, or posting on hacker news.
Speaking anecdotally, the one (temporary) time I canceled was when I found myself in the paying-but-not-using category.
Netflix have said for years that the star ratings they gather are not actually very good indicators for their recommendation engines compared to implicit signals such as - 'did the user watch the show / film to the end?' If they still believe this, it seems fairly irrelevant what they use.
> implicit signals such as - 'did the user watch the show / film to the end?'
I hope their version of that signal is better than what they're presenting in the UI. I have a few movies Netflix shows me as not finished, becaue I closed them just as the ending credits started.
Upon finishing the final season of Dexter, I clicked on an episode in the first season to look up something. I jumped to the 18 minute mark and watched about 10 seconds and closed the tab.
Now Netflix asks me every time if I want to continue watching Dexter from where it now thinks I left off in season one even though I spent a whole year watching all of the seasons.
So I have the same concern. If they have a more sophisticated model, it certainly doesn't get reflected in the UI...
The UI... They can't even make an 'f' shortcut key in the Win10 app (app, not browser) to go fullscreen.
And they force 21:9 movies to be played back inside the 16:9 portion of a 21:9 ultrawide monitor, with massive black borders as a result where there should be none.
I'm seriously unimpressed with Netflix as a company, for what they do with all their resources. Sometimes the MVP just isn't good enough.
Netflix have said for years that the star ratings they gather are not actually very good indicators for their recommendation engines compared to implicit signals such as - 'did the user watch the show / film to the end?' If they still believe this, it seems fairly irrelevant what they use.
> The stars are not about "enjoyment" factor, they are about perceived quality.
For you. But are you representative of the whole or even a majority? I personally use it to reflect whether I enjoyed it or considered it a waste of time.
I give some of my favorites - like Hackers - 5 stars but would never consider it a "quality" movie. It's awful and silly but that's the fun of it.
If you are sophisticated enough to appreciate "foreign documentaries", you should have little trouble discovering them by another means than Netflix rating algorithm. The latter is for the laziest, couch-potatoest audience, and it makes sense to optimize for that.
We've analysed product ratings in the past and found that the online ratings tend to have primarily bimodal distributions. People who rate online welf select and either love something and give very good ratings or they dislike and give poor ratings. Unlike surveys where people have to answer questions, only people with extreme reactions tend to answer these online rating questions and you don't get any meaningful distributions. So the only meaningful way to model say drivers from comments vs. ratings is to treat the ratings as binary and do a logistic type model.
Ultimately the key reasons to have these ratings in the first place is not to let people express their feelings, it is to get data for the recommendation modeling to predict better. Perhaps showing a simple like/don't like increases the number of people rating something which gives you more training data and thus better business outcome from your model without compromising on midel quality since you had to anyway model it bi-modally before.
I'm far from an expert on statistics, but wouldn't the fact that star ratings tend to fall on a bimodal distribution just mean that the ratings that don't fit the bimodal distribution have a higher signal/noise ratio than the ones that do? I'd think you could use a 2-4 rating as an indicator of a thoughtful reviewer. Additionally, it seems like you could also use the fact that a particular reviewer doesn't fit the overall bimodal distribution as a sign that this reviewer's reviews are similarly more thought out.
The fact that a reviewer uses 2 and 4 ratings might not be too meanngful. One reviewer might use 2s and 4s the way another uses 1s and 5s.
There may be reviewers who do not generally fit the bimodal distribution, but if they are the minority, why design your whole rating system around them? Better to optimize your system for the way most of your users behave.
yes scaled ratings do tend to have subjectivity which in practice we normalize by databasing surveys over time for categories x markets.
The reason online ratings tend to be bimodal is because the respondents are self selecting sample rather than randomized. Since product ratings are not typically compulsory, the extreme likers/dis-likers tend to fill the survey question.
When implemented as compulsory, it drives poor user experience and having to fill a survey to get something anyway introduces a bias (just fill something to get it over with). So no easy answer.
This is also the reason traditional survey firms haven't really gone out of business even with the wealth of online data from Facebook, Twitter and product reviews available, though better/easier access to actual behavioral signals are certainly replacing surveys in some areas.
I've built recommender systems at Etsy, among other places, and you get much better results if you make the data yes/no first. Shockingly MUCH better results. You'd think it would help to take into account, say how _many_ items of a shop a user likes, rather than just the fact that they liked some item, but it doesn't – it actively hurts. The only meaningful piece of information, it turns out is does this person like this thing or not.
That doesn't really surprise me. A lot of people imagine that highly granular, multi-dimensional rating systems must be best and they're often not. Of course, it depends a bit on what you're trying to distinguish but then it may not really matter to distinguish among the 75% that people are genuinely happy with.
I have no doubt you are right about the people who rate online (you definitely see it in restaurant ratings and comment sections) but when I rate movies on Netflix I very, very rarely ever rate something a 1 or a 5. I would imagine that 2/3 of what I rate has been a three (so average content for me) and that it takes something special to go up or down from there.
Just curious if I am an outlier with how I do my ratings.
I prefer the Goodreads rating system which compresses 1 and 2:
1 - Did not like it
2 - It was ok
3 - Liked it
4 - Really liked it
5 - It was amazing
The reason being is that most books won't end up being 1 or 2, and there really isn't much difference between them. But it sucks giving 2 stars to something that wasn't bad. Perhaps emoji arranged in a non-star-like pattern would be the best.
I've heard this a lot. When I look at reviews anywhere where there is a star rating and text review the only useful reviews tend to be 2-4 star reviews. 1 and 5 star reviews seem to have a tendency to be outliers or people who are really upset about something that most people won't care about. For example 1 star restaurant reviews that waited for someone to take their order and left because it took too long. I wonder if that could be applied to weighing the ratings accordingly. I have no data to test this unfortunately.
I don't work in the recommendations side of the house, but the way I understand things is that nowadays your star ratings (and now up/down ratings) are primarily to learn about you, what you liked and didn't like and therefore what you might like to see in the future. They're not really about creating a Rotten Tomatoes or IMDB style database of trying to rank all movies on some sort of objective "regardless of what you like, this is a high quality movie" sort of scale.
I have always wanted to get ratings from a selected group of viewers who has similar tastes as me. For example, I like SciFi movies that involves planetary travel, ie Star Trek or similar style movies. If I can get ratings from folks with similar interests as well, then the rating system would make sense for me. Otherwise, it is still a hit and miss for me.
That's exactly how their recommendation system works. If it just surfaced the highest-rated movies globally, everyone's recs would be identical. Collaborative filtering implicitly weighs the opinions of those similar to you more highly.
297 comments
[ 3.1 ms ] story [ 270 ms ] threadI'm about ready to drop them, just like they've dropped MASH, soon to be X-Files, and many many movies.
- Netflix's licences to stream shows are presumably for a limited period of time.
- There's only so much money in the pot, and they want to spend it on the renewals and new acquisitions they think mostly likely to retain existing subscribers and attract new ones.
- Sometimes the rights owner may not want to renew Netflix's licence: they may strike an exclusive deal with another platform, or may feel that having the show on Netflix is cannibalising their DVD sales.
I think Netflix though is making more off of their in-house produced content these days so I wouldn't say they're in danger, they just might look quite different in a few years.
Netflix is waay overhyped for what they actually have in their catalog.
https://www.amazon.com/Instant-Video/b?benefitId=xivetv&node...
You are not wrong
"US Netflix offers only 31 of IMDb's top 250 movies, study shows"
"David Wells, Netflix’s chief financial officer, was quoted by Variety stating that the company wants half of its content to be original productions over the next few years." [1]
RIP Netflix!
1. https://www.theguardian.com/film/2016/oct/14/us-netflix-imdb...
I find myself watching more of the stuff they've produced then the licensed content, so for me, it's been an improvement. If I run out of things to watch, maybe I'll care then, but at least I'm not tied to a cable contract, I can just switch to another streaming provider until they fix that issue. At this point though, I can't keep pace.
Why do so many not seem to understand that, in most cases, Netflix is likely not "dropping" anything? Netflix does not own the rights to the content and the owners can decide on the terms of the agreement on how long the content will be on Netflix.
People threatening to leave, or actually leaving, lowers Netflix's leverage to try to get and keep the content in their system.
This is why Netflix is creating their own content and bringing in content that's not normally within your area. So they can keep it up longer and have more control over their own destiny.
I started watching X-Files for the first time (yeah, never go around to it for the initial airing). Started watching in late February. And starting March 1st, is a warning on the upper left of my monitor saying "Show will be removed April 1".
From my viewpoint, I'm paying the same, and getting less. Not only less, but specifically something I'm trying to watch.
And yes, I tried watching that psychic alien abducted Netflix show. The sex scene at ep1 annoyed me, and had no interest still in ep3. Boring is putting it mildly.
Don't worry, the show goes downhill after the first couple of seasons, so it's probably for the best.
It's just as possible for the content owner to tell Amazon to stop selling their content if they wanted for some strange reason.
I have similar reasons to support HBO. However, I will not send money towards FOX/Comcast/Disney/Viacom because they still want to restrict their content and make it as hard as possible to watch it.
Of course, the danger is they could just start introducing advertising again by way of product placement in the story itself, but if it goes too far, I guess I'll cancel then.
It's a business model problem, and the entertainment model media providers offer is just not comparable to what is available on the internet.
Currently, I don't think the interest of Comcast's shareholders is the same as mine, they want to protect and grow their existing revenue streams, which depend on ads and outdated licensing contracts. My interest is to watch what I want, when I want, with as little friction as possible. Netflix and HBO provide that, none of the other big media companies do.
Also, Comcast has an interest in opposing net neutrality and providing a shitty internet experience so as to support their dying ad and TV business, so I also don't wish to support them for those reasons.
Meanwhile I rate everything 1-star vs 5-star.
So I'm not too convinced there's any nuance in star averages, much less a difference between "decent" and "fantastic". It might just be a more divisive movie that averages out to a 3-4.
Rating is a hard problem. No system works universally well. IMDb (Amazon property) for example uses 10 stars, Amazon uses five stars. Facebook use thumb up ("likes"). Games ratings are often in percent 1-100% (summarized by metacritics.com and others). School systems around the world use A, B, etc or numbers like 1-5 or 1-6 for grades.
You should also distinguish between aggregate ratings and individual ratings. Metacritic and other aggregators normalize to some scale, but not all the individual reviews use the normalized granularity. A common movie case is Metacritic having to normalize a 5-star, in half-star increments, review to a 100-point scale. I believe Metacritic also allows user reviews to use a 10-point scale, but present the average on a 100-point scale (the first ten integers but with one decimal point, technically allowing 101 ratings from 0.0 to 10.0). Aggregate ratings always have more detail than their constituent parts.
Meanwhile, I have to like all the things the same amount. This is just going to lead to me not rating the vast majority of mediocre content on the platform.
Why not let you choose as a user?
I bet many new users don't even know they can rate. I wonder if people aren't rating much because the UX sucks, not because it's a 5-star system.
What a disappointing simplification.
And anything with 5 stars has a 50/50 chance of being worth watching.
So, for me, any simplification is welcome.
It just seems to me it's a simpler way to rate something by a user, and at the same time classify it more adequately.
Star-rating can be too much detail, anyway: if you're comparing two shows, and one has a higher fraction of five-star ratings, but the other has a higher fraction of ratings that are four or above, which is better? Star ratings can be too much detail and cause confusion.
If you want more detail from each person, you can ask specific questions with a yes/no answer, like, "Were parts of it boring?" or "Was it violent?" That's probably better than star ratings.
The iOS App Store is a good example of this. An unbelievable number of people, for some reason, thing 1 star = great and 5 star = bad.
The other thing that annoys me with ratings in general, specifically places like amazon, is when people pan a perfectly good item because of some third party factor.
"Item was great, but it came in a beat up box from the post office, I'm giving it one star."
WTH does someone drop kicking your box have to do with the product?
Bad reviewers will always be part of the system. You have to expect it. Netflix has tons of reviews that go "I turned this off before the opening credit sequence finished, worst movie ever!!!! -- 1 star".
3.5: Good place, check it out
4.0: Great place, impress your friends
< 3.3: Double check your health insurance first
> 4.2: Fake
For example, you can look at the 50 Shades films or A Dog's Purpose or (especially) Gunday and have a pretty good idea that lots people are giving them bad ratings without having seen the films. (Their weighted average ratings are supposed to combat that, but they don't seem to be doing a very good job.)
(Obviously, "collect a dataset that philh finds interesting" should not be a business goal of Netflix. I'm not suggesting this is a reason to keep the star ratings. It's just a thing that I like about having ratings more detailed than yes-no.)
One good way to accomplish this is to pool users into cohorts of similar tastes, then suggest movies enjoyed by their cohort but not yet watched by a given user.
The question is whether doubling the frequency of rating a title is more valuable than a slightly more nuanced understanding of what each person likes. 'More valuable' meaning they can build you a better cohort from which to make title suggestions.
If you decide you want a more nuanced understanding of what a given reviewer likes, I think it's better to evaluate different dimensions (Was it funny? Was it violent? etc) than shades of grey* of the same one.
* pun not intended :)
I don't understand why engagement is the right metric here. If someone isn't sure how they feel about a movie, why is it a benefit to have them spew their half-formed thoughts into a like/dislike rating?
many has more value than
thougthgful one from few
Seriously. Lots of low fidelity measurement is sometimes better than a smaller amount of higher fidelity measurement. Especially when that higher fidelity measurement has errors.
The goal isn't making a database of the "best" and "worst" movies and TV shows. We already have the International Movie Database for that. This is just about finding out what you like, and finding other stuff like it.
That's pretty harsh.
I've watched plenty of movies that I enjoyed and would give a thumbs up but aren't sure whether it should be a 3 or 4 star movie. There's movies I love but consider too flawed to give 5 stars. And I find little difference between 1 and 2 stars other than the level of regret for wasting my time.
Overall I rarely bother rating things on Netflix because it just doesn't feel worth it. So they get nothing out of me, whereas with a thumbs up or down I am more likely to chime in.
You could say "well that's the same as not rating it!" but it's not. I'd like there to be a clear distinction between "I abstain from voting" and "I thought this content neither vomit-inducing nor life-changing."
I feel as though this devalues their recommendations to you.
Now that mediocre content will be scattered across the two categories I really care about; I guess IMDB/Rotten tomatoes will steer clear of this ridiculous binary rating systems.
0: https://www.rottentomatoes.com/tv/ash_vs_evil_dead/s02
I think Netflix's move might help this. It certainly lowers the cost rating.
Even most of my disappointments tend to be in the vein of "It was OK I guess but I don't see why people think this is so great."
They kind of already know this though. (How many times have I watched a particular movie on Netflix?)
I'm all for this!
But the signal isn't just for Netflix; it is also for users, who might sometimes be in the mood for something silly and sometimes in the mood for something good. Also, people might rather get more suggestions of good movies even if they are more likely to watch bad ones. (Of course, people might also just overrate documentaries.)
I'm going to rely on Netflix's recommendation engine to put things in front of me that I am going to enjoy.
Why not remove recommendations altogether? They clearly don't have a library to support it.
I'm reminded of Jerry Garcia's famous quote about the Grateful Dead: "We're like licorice. A lot of people don't like licorice - but the people who like licorice REALLY like licorice".
A 3.5 could either mean 50% of users give it a 3, and 50% give it a 4, or it could mean that, say, 30% of users give it a 5 and 70% give it a 3 or less. How do you know you aren't part of the 30%? Great art is often polarizing.
If they remove the axis of quality and merely opt for "will you watch this: yes/no", it will make it harder to select "good popcorn trash" versus "high-quality, high-enjoyment" to fit my mood.
Wouldn't it be more beneficial to have fewer people give a more detailed review to describe what they like and dislike about something? What use is "good" or "meh" compared to for example "I rate this product low because the shoes are very narrow for my normal feet" so you can actual relate it to what you actually are looking for? One person's "meh" could be another's "good"...
From the article:
"This makes sense – giving a five-star rating takes some thought, especially for something like a movie or TV show.
A binary “yes or no” option is much easier for viewers to commit to, [...]
[1] https://youtube.googleblog.com/2009/09/five-stars-dominate-r...
A binary option actually has three options. Did it prompt me to rate it as bad, or did it prompt me to rate it as good, or was it bland enough where I don't want to sway the rating of it at all?
If I rate a movie on IMDB as 5-6, and maybe even 7, the possibility of me recommending it to someone "depends". In that case I might as well not rate it at all.
Ratings are inherently tricky, because we all like different things. I don't think you lose anything by going to a like/dislike system. Recommendations should be based on aggregate maps and matching you with similar viewers that have a certain overlap in ratings. That's made easier with binary choices.
IMDB ratings are only partially informative to me anyways.
Now, I realize that's almost certainly not the level of individualization offered by their ranking algorithm (I'm convinced there's way more of a "groupthink" dynamic), but that's partly my point: if your ratings and recommendation system isn't capable of supporting individual differences, then your algorithm is broken.
If you have a complex problem (personalized recommendations absolutely are complicated) then you shouldn't fix a broken algorithm (Netflix recommendations are, at least for me, a bit meh) by simplifying the inputs (switching from stars to up/down). You should fix the broken algorithm, by creating a better algorithm.
I realize it isn't that simple, particularly for a publicly-traded company whose stock price would absolutely be influenced by a decision to scrap part of their core technology, but from my perspective as a user, this is going to hurt my experience with the product.
In any case, simplifying the ratings users can bestow makes a lot of sense, unfortunately. We like to imagine people carefully considering the merits and flaws like a professional reviewer, but the fact is that the vast majority of users only ever use two ratings anyway: if they like it, 5 stars; if they didn't, 1 star. There's maybe a third option where if they're ambivalent they just don't rate it.
So star ratings aren't actually very useful for evaluating products. This xkcd https://xkcd.com/1098/ made me realize that an Amazon product which has a 20% chance of exploding when you open the box, and otherwise works normally but unexceptionally, is gonna average out to a four-star rating for a product you really shouldn't buy. I always look at the one-star reviews before buying stuff now, no matter how few of them there are; knowing a thing's failure modes is much more useful than a bunch of praise.
But that's not even close to true. In addition to it technically being up/down/no rating, we've got how long people watch the show for, how the shows they watch form a pattern, how all the patterns of what people watch show global preference patterns, whether they rewatch a show, when they watch what sort of shows, what individual scenes are rewatched vs. skipped... Netflix is swimming in a sea of preference data, not sitting here trying to figure out "Gosh, um, if the user likes this movie 3 vs. 4 stars, uh... what does that mean?"
It makes perfect sense to me to optimize this one-data-stream-among-many to increase user participation and get more bits of information from more people engaging with the simpler system, rather than trying to squeeze bits out of the few people willing to use the star system and the even fewer willing to write useful reviews.
It isn't really even as shocking as it may seem at first. The star system has 6 states, "no rating", 1, 2, 3, 4, and 5 stars. That's 2.6 bits, with some simplifying assumptions [1]. The thumb system has up, down, and no rating; that's 1.6 bits [1]. To make up for the bits, you need only see ~40% increase in participation over the current star system... think that's going to happen?
[1]: The simplifying assumption is that all outcomes are equally likely, but that's not true. I don't have the numbers to run a more complete information theory analysis, but it's not hard to imagine the "no star rating" case is so common that it produces such a small fractional bit that a higher-participation-rate yea/nay/no rating (if you puth this UI in their face, "no rating" becomes much more meaningful, too) straight-up produces more bits of information on average, and is thus simply an improvement even before considering the superior UI experience. I rather suspect this is the case, some very sensible assumptions would suggest this, but I lack the ability to prove this; the "assume all outcomes are equally likely" is at least a concrete case I can discuss.
This move seems very inward-focused: Netflix is thinking about the information it wants to get from users, but I wonder if they surveyed people about how much they want to see percentage-likes instead of stars.
Of course, Netflix solves this by removing the quality content....
For some people, such as myself, it will decrease participation (making the "no rating" indistinguishable from "it was okay, but I don't love it and don't hate it, so I wouldn't know whether to upvote or downvote"). A 40% increase is also quite a significant change. I'd be astonished if they got that much more engagement out of it.
There's also quite a difference between having the data for their internal algorithms and what people want to see. Just like Facebook eventually added alternatives for just the "Like" button.
I don't mind having a percentage displayed rather than a star rating (I automatically convert everything in my mind anyway, even though the star ratings are more easily visible at a glance when scrolling through a list), but I mourn losing a lot of expressiveness in the ability to record what I really thought of the video that I watched.
Airbnb does this pretty much spot on, in that they ask for 1-5 stars for five categories, rather than just "did you enjoy this Airbnb?" I also think that Untappd does this wrong, because in general beers can't be described in just a 1-5 star rating without deeper explanation.
I think that moving from a pure star rating to a thumbs up and down rating is better overall, if only because it makes me, as a watcher, not have to think as much and therefore give a rating where I might not have before. If I want to go more in depth, I can explain more too.
My favorite example is Fallout 4. There are tons of negative reviews from people who put 600+ hours into the game. How the hell do you put 25 days of your life into a game you don't like? They claim it "got boring". Not many games will not be boring after 600 hours...
This is only true when those star ratings are taken out of context. Individual reviewers can have completely coherent star ratings that are very useful for making determinations.
The golden age of Netflix discoverability, for me, was back when they added social networking features. I connected my account with many friends and family and was really able to get a sense for everyone's taste in movies. When it came time to discover new movies/tv to watch, I'd open up the ratings history for someone who's tastes I felt aligned with the mood that I was in. Some of my friends would only rate high quality content with 5 stars, others would typically rate mass market, mindless entertainment with 5 stars and others tended to include a lot more 5-star reviews within a certain genre. Over time, I got a feel for that and could use it to my advantage.
But when you remove that individual context and only aggregate 5 star reviews across a massive population who all view the 5-star scale in their own way, you get that uselessness that you talked about. Some people may reserve 5-star ratings for truly extraordinary content, but Netflix doesn't weight those ratings any more highly than the 5-star ratings of someone who rates nearly everything as 5 stars. This makes the ratings that Netflix displays meaningless. But that's only because Netflix's current methodology makes them meaningless, not because they're inherently meaningless.
Is that fact or speculation? It's a common recommendation technique to find the mean rating for a person, and use that to normalize. Normalized score = (actual_score - mean_score + 2.5) for a 5-point scale, for example.
I have no insight into whether or not Netflix does something similar, but I'd be surprised if they don't.
Imagine two people watched 100 movies, and assessed them the same way, say that 20 are five-star, 20 are four-star, 20 are three-star, and so on. The first person rates all 100 movies, but the second one has a habit of writing a review only if he considers a movie excellent. Then your algorithm will wrongly discount the second person's ratings.
Is this true in the real world? Have you found different people to have different habits, like rating everything vs rating only good movies vs rating only bad movies?
Second, I don't know whether individual ratings should be normalised as you say, but the final rating that's displayed should be normalised. For example, the app stores should have 20% of apps rated five stars, 20% four stars, and so on. This will make the ratings more objective and easy to understand. If the average rating is 4, then an an app rated 4 is merely average, which is counter-intuitive.
(Perhaps normalised within a top-level category, like Productivity and Games)
An exquisitely detailed system like Airbnb's costs the user more time & hassle. Which means, assuming you make it voluntary, that a lot of them skip it. Meanwhile a simple/easy system like this, turns a scalpel into a sledgehammer somewhat, but you get a lot of them. So there's a balance to be struck for your site's particular userbase.
The nuance comes when you aggregate many people's binary decisions. A value near 1 tells you many people felt the good way outweighed the bad; a value near 0 tells you the opposite; and a value near 0.5 tells you that people are split. I think that's far more useful than trying to ask many individuals to provide you with a granular score.
For example, I found Mirror Mask enjoyable and would recommend it. They dreamed up and brought to life a fantastical world full of magic and wonder. The visuals are excellent. The story is mediocre, so you should watch it with the right expectations, but watch it nevertheless.
The ability to distill something complex and multi-dimensional to a clear conclusion ("enjoyable and I recommend it" in the above paragraph) is a sign of clarity of thinking and expression. If you can't do that, I'll go by other people's ratings.
Remember that the goal isn't to accurately capture one person's opinion, but to help potential viewers decide whether to watch it.
This saddens me.
>Users would rate documentaries with 5 stars, and silly movies with just 3 stars, but still watch silly movies more often than those high-rated documentaries
That's not incongruous to me. The stars are not about "enjoyment" factor, they are about perceived quality. I may have a go to cheap ice-cream and rate it 3-stars but rate a good affogato 5-stars and only have it once in a while.
They are diluting the meaning of quality and instead are opting for a saccharine "enjoyment" factor. This binary choice does not sit well with me and I hope they abandon the idea soon.
A movie like "Suicide Squad" was (to me at least) highly enjoyable, though I thought it objectively had many problems with plot and delivery. I would probably give it 3 stars, which is relatively neutral for me (or leave it altogether unrated). If it were thumbs up/down, I would probably have to leave it unrated completely.
Maybe that's an improvement, but the more discussion I see around this, the more I realize the futility of single-axis metrics for things like this, until and unless you're asking a single question "Did you enjoy this movie" is a much different question from "is this a good movie" -- and I don't really know which one I've been answering on Netflix all these years past.
3 stars = "not a bad movie, but wouldn't recommend"
4 starts = "decent movie, would recommend"
5 stars = "great movie, would definitely recommend"
A binary choice doesn't have any weight to it.
This is actually rude of you, bordering on ridicule. He explained very clearly why he is dissatisfied.
I'm worried about the thumbs up/down, as Netflix has increasingly recommended shows I do not like this year, and I don't think the thumbs solves the "rating for perceived quality" issue. I've been using Movielens for recommendations a lot instead, and it seems more accurate for me... and it has enough stats to show that on their site, most rating is not binary, much closer to a normal curve for average movies or a linear graph on really good movies. (It lets you see the distribution curve on individual movies, and other metrics.)
I thought that Neflix made it pretty clear that the ratings were used to choose which content to show me. So I should only rate based on what I'd actually like to see.
They are just trying to increase number of views for a poor quality library. This will mask how bad most of the content is and increase their numbers.
My dream is to build an open source app that gathers all these personal ratings (from Netflix to Pandora to GoodReads to restaurants) and puts it back on where it belongs: the users hand!!
Unfortunately finding the time to work on this will be tricky..
Anyone knows if that's a thing already?
Though quite a bit less than they used to. My observation is that more and more back catalog type of items are no longer available.
Or is that my wishful thinking?
Of course, given that they pretty much put every other rental place out of business, your only choice in a lot of cases is just to buy a disk if you want to watch something that isn't available streaming.
I've looked before halfheartedly for a way to export them but now that they cut the API off it doesn't seem like there's a great way to do that.
[0] https://www.reddit.com/r/netflix/comments/5zzc4g/batch_downl...
Edit: Giving away one of my 48h Crunchyroll premium guest pass. No credit card required. I'm not working for Crunchyroll in any way, they give out guest pass monthly to premium users. First one to use this code wins: UWWJAQTDKYZ. Note that I will receive a notification telling me your username and perhaps your email address, if privacy is important to you.
At that point, isn't just plain old piracy more convenient ?
I used to use a VPN to watch US Netflix, but they started to block known VPNs and I didn't feel like spending time on workarounds. On the other hand, downloading the latest episode of some TV series takes only a few mouse clicks. For sure easier than fighting geo-restrictions.
I still pay for Netflix, but doesn't really watch it much because of the tiny catalog available to me locally, I simply want to support them. Perhaps it will make me feel better if I would download any of their content. :)
>I simply want to support them.
Nothing to be sad about. Also curious, why do you want to support them ? Being a market leader, Netflix is a unique position to influence some of these regressive practices. More support is clearly not helping; it's only making it worse by creating monopolies that exert even more negative influence.
Netflix is not much of a market leader until it contains at least 90% of the movies i want to watch, which can't happen without enough leverage from their part given by user support.
But ofc, I wouldn't mind if they had some real competition, I don't want a monopoly.
Geo-restrictions have existed forever. What Netflix started was actively blocking entire ranges of IPs belonging to vpns and commercial ISPs (cloud providers like aws, dedicated servers like ovh etc.). This should fall under some corollary to net-neutrality where a service cannot discriminate between ISPs. Does such a corollary exist in countries that have some form of net neutrality (US, India, Netherlands etc.) ? Can someone with legal expertise comment ? From my standpoint, they can have geo-licensing, but should stick with some consistent way of showing content either based on billing info or IP. Blocking out vast portions of the internet because it doesn't suit them should clearly be some violation of laws like net neutrality.
I'm on the brink of cancelling my membership as it is, this certainly won't help, though it's obviously the proverbial straw rather than a big deal in its own right.
How do you compare ice-cream with cheesecake?
I ignore ratings because of that. They don't mean anything and I am guilty.
On Good Reads, I have given what is deemed as a trashy piece of writing 5 stars. At the same time, I have given Dune 1 star because the story was garbage.
So what is this 5 star rating? Some people say it is the experience, some say it is the writing style, others say it is the story telling, or the plot.
Lets also face it, when it comes to films, the news papers have their own agendas and personal commitments to other agencies. My film could be absolutely garbage, but since I dined with the Editor a few times, It gets 4/5 stars and then, it gets 4/5 stars because the rest of the population is told how to feel.
I would even say that this whole system of rating based on Stars or Likes can simply be removed. If a film is good, you will know about it. If a film has been watched 1000000 times more this weekend than another, base it on that if anything.
And the rub is that not everybody uses the same projection, which means the utility of one person's rating for another unknown their projection system (the weight they give to each sub-component of the overall rating) is completely unknown.
For all the faults of the old system, the new one is even worse. Like Pandora, I'll probably just never rate things or mostly just give them a down vote if they don't crack a threshold of goodness.
They should be optimizing "number of people who pay for Netflix". Instead they are optimizing "number of hours spent watching Netflix".
Those two metrics are correlated, but definitely not fully.
If I spend hundreds of hours a month watching meaningless action movies and TV on Netflix, I can cancel my subscription and save myself hundreds of hours and $10. I've got lots of other ways of filling my time with endless pablum.
If I spend 2 hours watching a thought-provoking documentary, I say to myself "yup, that justified spending $10 on Netflix this month."
> Users would rate documentaries with 5 stars, and silly movies with just 3 stars, but still watch silly movies more often than those high-rated documentaries.
Speaking anecdotally, the one (temporary) time I canceled was when I found myself in the paying-but-not-using category.
I hope their version of that signal is better than what they're presenting in the UI. I have a few movies Netflix shows me as not finished, becaue I closed them just as the ending credits started.
Upon finishing the final season of Dexter, I clicked on an episode in the first season to look up something. I jumped to the 18 minute mark and watched about 10 seconds and closed the tab.
Now Netflix asks me every time if I want to continue watching Dexter from where it now thinks I left off in season one even though I spent a whole year watching all of the seasons.
So I have the same concern. If they have a more sophisticated model, it certainly doesn't get reflected in the UI...
And they force 21:9 movies to be played back inside the 16:9 portion of a 21:9 ultrawide monitor, with massive black borders as a result where there should be none.
I'm seriously unimpressed with Netflix as a company, for what they do with all their resources. Sometimes the MVP just isn't good enough.
For you. But are you representative of the whole or even a majority? I personally use it to reflect whether I enjoyed it or considered it a waste of time.
I give some of my favorites - like Hackers - 5 stars but would never consider it a "quality" movie. It's awful and silly but that's the fun of it.
Ultimately the key reasons to have these ratings in the first place is not to let people express their feelings, it is to get data for the recommendation modeling to predict better. Perhaps showing a simple like/don't like increases the number of people rating something which gives you more training data and thus better business outcome from your model without compromising on midel quality since you had to anyway model it bi-modally before.
There may be reviewers who do not generally fit the bimodal distribution, but if they are the minority, why design your whole rating system around them? Better to optimize your system for the way most of your users behave.
The reason online ratings tend to be bimodal is because the respondents are self selecting sample rather than randomized. Since product ratings are not typically compulsory, the extreme likers/dis-likers tend to fill the survey question.
When implemented as compulsory, it drives poor user experience and having to fill a survey to get something anyway introduces a bias (just fill something to get it over with). So no easy answer.
This is also the reason traditional survey firms haven't really gone out of business even with the wealth of online data from Facebook, Twitter and product reviews available, though better/easier access to actual behavioral signals are certainly replacing surveys in some areas.
Just curious if I am an outlier with how I do my ratings.
1 - Terrible. Good Lord, who thought making this was a good idea?
2 - Poor. Would not watch again
3 - Fair. Neither for or against. Unlikely to watch again, but I don't grudge the time spent.
4 - Good. I enjoyed it. I would probably watch again.
5 - Exceptional. Probably in my top 10 now, and foresee enjoying watching for years to come.
3 and 4 are used a lot. 2 is used occasionally. 1 and 5 are rare.
1 - Did not like it
2 - It was ok
3 - Liked it
4 - Really liked it
5 - It was amazing
The reason being is that most books won't end up being 1 or 2, and there really isn't much difference between them. But it sucks giving 2 stars to something that wasn't bad. Perhaps emoji arranged in a non-star-like pattern would be the best.
1) How good was the movie? [1-5]
2) How likely are you to watch it again? [1-5]
I think it was too many questions to ask, but trying to separate out the enjoyment from the quality struck me as a good idea.