I wish they would take on Netflix with some better UI functionality. I can't even fast forward at hbogo.com without picking up the mouse. The only keyboard commands that work are Spacebar to pause and play and F to get full screen.
Another problem is that if I watch 2 episodes in a row and I bookmark the page of the 2nd episode to finish it later, the URL in the address bar is actually still the URL of the 1st episode.
Yeah because netflix loves those robots it serves. At least netflix doesn't have a name problem... Three different HBO products... yeah that's gonna work. /s
HBO Max is just their newly renamed HBO Go and Now unification. If you had either Go or Now before, you now have Max. It's not really that complicated.
Actually you are incorrect, HBO Go and Now still exist as separate products. It is complicated for people that already have trouble tracking where their shows are on several different services. Even if they intend to merge Now into Max completely which likely is the case they still are making it confusing as hell for the common user.
I'm not sure that confusion will really exist, or that it would be warranted if it did. At least for HBO Now, the existing Android app updated and is now named HBO Max. I just logged into Max using my Now credentials, and it seems to have all the content from Now, plus some extra stuff. I think Max is just a renamed superset of Go/Now (which were basically just two names for the same thing anyway). There seems to be no reason to use Now anymore, and Go seems to only exist as a separate site to appease cable companies that HBO isn't offering a competing streaming service vs what they're giving "for free" to HBO cable subscribers.
Lately it feels like Netflix's algorithm is "just show Netflix produced content". Maybe I'm being nostalgic but I really miss the old shooting star recommendation system where you rated content and got similar likes from other viewers.
I'm guessing the distribution of reviews was bimodel. Ideally, people should use a 5-star system in a way that it provides a normal distribution. Most content should fall between 2 or 3 stars.
But it doesn't actually work that way, 4 star uber drivers get kicked off the platform, 2 star Amazon products suck, 2-3 star movies suck. 4-star movies are okay.
As such, I think we can conclude that people don't actually know how to use a star rating system and we should just get rid of it.
I wonder if letter grades would be the way to go in relevant countries. Most people have internalized what a “C” means pretty well, even taking grade inflation in to account. Hell, that’s the whole reason 10 star rating systems are bad, as people treat 7 like it should be average and 5 is a failure.
I haven't been subscribed for about 6 or 7 months and the algorithm is mainly what drove me away. Outside of sports it's a service that seems to want to be every channel at once and that's mainly just not what I want. I'm not interested in game shows, reality shows, hgtv-type shows and it seems like there's no way to stop seeing this content no matter how many "thumbs down" I give each show. I'm not expecting to not see stuff I don't like but I want to at least give it a thumbs down and get it out of the interface.
HBO Max I'm on the fence about. I really like HBO but not sure I need or want everything that comes with the Max flavor.
There was a brief time when they had the absolute best recommendation algorithm on the planet. It was intentionally replaced by something utterly different which is geared to simply steer everyone to IP that they own. It is not remotely similar. I have absolutely hated every change they've made to their UI since they ditched the old ratings system, too (forces you to use the mouse and sets you up for fumbling what should be clear actions, so you spend more time browsing - which costs them almost nothing - and less time watching). My guess is, research indicated they'd make more money with these changes than by licensing content audiences actually prefer and building a sane interface.
As it turns out, the Netflix golden age was a short-lived anomaly if not outright farce. If you want decent curation of content and also to be able to watch it, you will have to pay through the nose or pirate. Same as it ever was. It's consumer hostile, but the system is working as its overseers intend. Whether it's really the best long term strategy is still an open question in my book.
The problem with a recommendation engine is that it breaks down when content options are limited. In Netflix's early days of streaming, they had a similar approach to content as they did with DVDs, i.e. capitalizing on the long tail and offering an enormous selection. That proved to be inefficient for streaming because there's a minimum bar at which rightsholders will sell licences, and it's far higher than just the price of a few DVDs. Thus, Netflix switched to a more limited acquisition model, and then began a more aggressive program of content creation to effectively gate its offer from competing services. The upside of this for consumers has been a shift by many talented creators into series-style content; the downside is that the market for independently-produced films has bottomed out considerably.
This is like when Apple said their Music platform would be more human curated rather than algorithms like competitors such as Spotify. But the reason why I love Spotify's recommendations and not Apple's is because Spotify focuses on algorithms and does them so well.
Same with Netflix, in my experience they truly know what I will like to watch, they seem to know what shows I'll love and what shows I won't love but will help pass the time, and seem to show it to me just when I want it.
I imagine it takes a lot of time and lots of past history and data on users to create those algorithms so when starting out the human curation method is required. And for companies like Apple, may be the long term strategy because the alogrithm method that YouTube and Spotify comes at the cost of potential tracking and privacy concerns.
I think it also depends on what you listen to. I've seen plenty of people complain about Spotify's Discover Weekly playlist and how it doesn't get it right, but I love it. If you listen to a wide range of genres, it will give you a weird mix, but otherwise its been my go-to for many years now.
> Same with Netflix, in my experience they truly know what I will like to watch...
Wow, I'd love to live in that world. Once Netflix abandoned their 5 star system their recommendations really tanked for me. It feels like it's recommending more based on outrage & "engagement" than enjoyment for me. Meanwhile MovieLens [1] consistently guesses what I would rate a movie within +/- half a star, I think I'm using their Warrior recommendation algorithm. The tagging features on MovieLens help - apparently I most enjoy "visually appealing surreal mysteries".
But I absolutely agree about Spotify. Spotify seems like its actually learning from my Liked tracks, curated playlists and behaviour, and picks up on when my tastes shift to wildly different genres for a while.
I agree with you regarding spotify (though in my opinion lastfm was better ages ago), but I completely disagree with Netflix. Even if it weights genres I might like, netflix always recommends me whatever is popular at the moment.
It might be because of the service balkanization, but in the past I was able to find hidden gems on netflix, the same afterwards with prime video, but nowdays not only I have to browse in the text catalogs you might find online to quickly search among all the stuff I don't like, to find something I might enjoy that would certainly be buried in netflix UI. And even worse, I find myself rewatching my favorites instead of being able to find something different than blockbusters in the platforms.
100% agreed re: Spotify, but Netflix's algorithm seems to think that if I liked Annihilation and Arrival then I'll like any young adult or action movie with scifi aesthetic.
Similarly, if I liked The Witch and Hereditary then top recommendations for me must be $5m budget cash grab teen slashers and haunted webcam gimmicks.
I think random selection would be more effective for me than the current Netflix algo(s).
In contrast to other commenters, I loved manually curated Apple Music playlists, I found so much great music through them in first 6 months
of using their service. You could really feel that they were created by people with real passion for underground and experimental music, on par with best user-curated Spotify playlists. When they switched to algorithm based recommendations its quality tanked and I moved back to Spotify because it has better overall user experience on desktop and social features. Spotify's algorithms has never, ever introduced me to interesting new artist, it only suggest to me things that I either already know and like, already know and dislike or new things that pretend to be good but there is no original though behind them. Fortunately, a lot of independent music critics and enthusiasts have their own playlists with good, deep cuts - but you need to find them outside of Spotify, because their discovery features are so bad. I miss early Apple Music so, so much. It reminded me of prime years of Last.fm.
Spotify also makes it seamless to tunnel down a rabbit hole and then back up. I've done this across genres from one seed song while still enjoying and occasionally sampling new music along the way.
Netflix bothers me because they have a marketing bend that creates your tunnels for you.
> But the reason why I love Spotify's recommendations and not Apple's is because Spotify focuses on algorithms and does them so well.
The article literally makes the point that Spotify's system is much more human curated than Netflix because it heavily relies on human curated lists that are shared between users.
I wasn't old enough to remember it, but when search engines tried to do this in the 90s, it became clear that people preferred machine recommendations. See yahoo directory, look smart.
This is different though, because of the potential exclusivity of the corpus. It would be like if Yahoo could hand pick websites and stop competing websites from retrieving them.
And it's not obvious that people want to discover art the same way they want to discover information (obviously not implying that Google can't be used to discover art).
HBO will never scale itself beyond it's own ability to comprehend itself, or, if that happens, it's quite a ways off. Considering HBO is throwing hundreds of thousands of dollars at each of these shows paying twenty bucks for manual curation on top of that doesn't sound unreasonable.
It feels like different problem sets might yield different results no?
As an avid music fan the ways I've sought out new material go as follows:
- Originally, recommendations via trusted outlets / forum members
- Not so long ago, Spotify algorithm recommendations
- Currently, curated recommendations via trusted outlets / social media posters
My anecdotal experience between Netflix/HBO follows a similar pattern:
I tend to first go to a provider like HBO when seeking out new content, assuming quality is higher than whatever Netflix is pushing on me these days.
In contrast, a roommate of mine almost exclusively uses Netflix. She is more of the put on whatever, maybe pay attention maybe not type of viewer.
I guess summed up the main difference FOR ME between the two is HBO: "I need something good!" and Netflix: "I need something new!"
People preferred it because Google - at the time - had found a scalable solution which gave (mostly) the best results. Why should I search for hours through directories if someone give me what I want after only entering a few words?
No one has found the "page rank of movies/series" yet, so I think HBO has a better chance here.
> No one has found the "page rank of movies/series" yet, so I think HBO has a better chance here.
I'm not so sure -- I think the problem is no one has deployed it.
As a consumer, what I want is: based on my viewing history and/or shows I've said I like, show me more shows/movies I'll like.
What I currently get is: shows/movies that are in a similar style and genre to what I've watched or liked. The algorithm doesn't seem to take into account "good" vs "bad" movies.
The result is if I just randomly pick some movie from Netflix's "Because you watched.." recommendation, there's a very high chance I'm going to end up watching something I don't like because it isn't good.
As a concrete example, I just went to Netflix movies. "Because you watched Avengers: Infinity War...". There are 6 movies on the first page, 2 of which I've seen and have high Rotten Tomato scores (they are both Indiana Jones movies, with 88% and 85%). The other 4 have RT scores of 31%, 27%, 12%, and 3%. (Don't get me wrong, there are movies I do like that have low RT scores, and there are movies with high RT scores that I don't like, but generally a bad movie is a bad movie and more often than not there's a correlation with RT scores.)
Even if I could filter by "RT score > 60%" that would cut down on a lot of garbage, even if it means I might miss a couple movies/shows I'd actually like. This would change my personal perception from "The vast majority of content on Netflix isn't really worth watching" to something more positive. Note I still subscribe despite this -- there's still ample worthwhile content. I just don't watch things based on Netflix's recommendations alone.
Certainly it is possible to come up with something better than that blunt filter. So the question is: why don't they?
As Netflix has moved away from carrying lots of other people's movies to largely producing their own in-house TV shows, it feels like they barely have enough content to need an algorithm to curate it. They just show the same new Netflix originals over and over in ever category.
I'm sure Netflix has piles of data that shows this is the best strategy. It just seems like a differentiated curation strategy doesn't matter that much in this battle. It will come down to who is producing the most popular original content most consistently.
It feels to me like the balkanization of streaming services was inevitable and Netflix getting into the content game was just a logical response to remain relevant. If the catalog is what garners subscribers then a streaming service without its own catalog is basically fucked in the long term.
It makes a lot of sense IMO. They can immediately make more money in the short term, and after their competing product is ready they can negotiate more favorable terms with Hulu or not renew their agreement, which will direct a portion of people that discovered their content through Hulu to their own platform.
This balkanization is a good thing. I'd far rather have the choice between many competing providers each offering a service at £8/month, than have the choice to take or leave a single expensive all-in-one subscription, like cable.
It's also good that market forces are driving content-creation.
If only the prices were lower to reflect that competition. There seems to be a tacit agreement that streamers will pay 6-12 USD a month per service. Netflix began at $8 with almost all popular content available (which could be streamed).
Can you even watch that much content? I would expect that you only keep 1-3 subscriptions at the same time, and juggle between the different platforms when you want to watch some exclusive content. Like, Netflix for 1-2 month to watch some exclusives, then you cancel until the next time something seems interesting enough.
I don't want to have to manage it on this level, the metagame kills the experience. I wish movies and tv had compulsory licensing like music, because the pick your music service and they have access to almost everything is much better for consumers.
Somewhat related: it's curious that Netflix are so coy about their catalogue. Without signing in, they refuse to show what they offer in your region. You have to make do with a third-party listing site like JustWatch.com
It would be good if they had the same content but compete with the UI (in such case Netflix would win, and HBO would loose, as they have the worst possible UI), streaming content etc.
Not as bad as some others I have tried. Disney Plus gets on my last nerve at times. Just try to rewind like you would in other apps with a bluetooth remote.
Most (all?) of that exclusive content only gets produced because of exclusivity, I’d think. I.e. we get a greater number of shows because of exclusivity.
The gaming industry has move (largely) away from exclusives and that has been a net improvement for content.
I suspect the pendulum will swing away from exclusive content for television shows. Producers might make more money up front from the deals, but the long game favors those who establish brands with broad market appeal, and the way to do that is to get your content in front of as many eyes as possible.
Huh? The most popular console (Nintendo) is almost entirely exclusive games. Playstation and XBox are less exclusive than they've been, but not by much (FF7 Remake was PS4 exclusive as well).
Ten+ years ago, it was common for best selling games of the year to be a console exclusives. Today is much more rare.
The Switch may be the most popular console, but it's install base is a small fraction of PS4+XBone+PC+Switch. So even Nintendo exclusives purchased by the majority of owners can't sell as many units as the latest Call of Duty. Heck, even the The Witcher 3 has sold more units than the best selling Switch game.
Sure, permanent exclusives are generally only first party. Bungie/halo was one but thats because MS at the time didn't have enough first party studios.
The exclusives fight in gaming is ramping up again with the cloud streaming services. Google's Stadia and Nvidia are racing to lock up content. Microsoft is about to launch out of beta as well and each service will compete on their library.
After critical mass of number of movie providers one will just go back to piracy.
There is only so much one can stand. If the market is spread to thin then pirating content will be less frustrating than getting it on streaming service (if you have to subscribe to 8 of them).
Throw in one of the streaming services that support 'live' cable TV and that hikes up the prices, plus adding things like HBO, Showtime, etc to a Hulu subscription, and you get there rather quickly.
Personally I try to keep the number of services I'm subscribed to to a sane minimum.
Competition is good, but each of the competitors are creating their own shows. This potentially requires you to buy multiple services to view all the content you want.
I would prefer the competition to not be content but on UI, speed, downtime, viewing suggestions, etc.
The best part of cable was that you could get any channel regardless of your provider. You could have Spectrum, AT&T, Direct TV or whoever and get any channel you wanted.
The problem was that you usually could not do al la carte and when you could it would not be cheap enough to justify losing all the other channels.
Imagine if you had to pay for Time Warner cable to get CNN, AT&T to get HBO, and Comcast to get NBC.
It’s just the opposite now. You use to have to pay for cable to get HBO, CNN and NBC (if you didn’t live in a good coverage area.) and you were stuck paying sports fees, broadcast fees, franchise fees, etc.
Now you can get HBO a la carte, NBC prime time either for free on their website or with no commercials or with Hulu for $12.
Really there is little on network TV that I care to watch anymore. Between Netflix ($5 with Tmobile), Hulu w/ no commercials, Amazon Prime, AppleTV+ (free for now), and Disney+ (Mostly for the kids), I have plenty to watch. I sign up for other services (Starz, CBS, DCUniverse) occasionally to binge and then cancel.
I didn't mean to suggest that you couldn't currently pay for whatever you want a la carte. It is of course better that you don't have to pay for cable but worse that you have to pay multiple content providers (Netflix, Hulu, CBS, HBO) to get the shows you want.
When you pay for Spectrum you get TBS, CW, Fox, NBC, CBS, ABC, etc. When you pay for AT&T you get the same channels. It doesn't matter your cable provider, you would have access to pretty much any channel you wanted.
With Netflix, Hulu, Amazon, HBO, etc you have to pay for all of the services to get the content you want. This is beneficial for people who only want shows on a few streaming services but bad for people who now have to pay 10 different companies.
Ideally all the content creators would work together and make their content available to all of the streaming companies at a reasonable price even if they had their own streaming product. Paying for Netflix would provide the same shows as paying for Hulu.
Each of the streaming companies would then allow you to pay a set price for all the content like a traditional cable company (and their current business model) or allow you to pay for the shows you want. If you only want 5 shows you shouldn't have to subscribe to Netflix, Hulu and Disney+. You could pay Netflix and get all 5 shows instead of paying 3 different companies. Since you don't care about the other shows you shouldn't have to pay for the streaming company to obtain / keep the rights of those shows or to pay for them to create content you don't care about.
This is a pipe dream and will likely never happen.
Yeah but how many of them are good? I can probably count the amount of Netflix original movies that I've heard discussed with one hand. And when it comes to stuff like Bird Box, the talk doesn't really even go with the quality.
I don't think it is as subjective as you might think. If it was you'd expect lists like the imdb top movies of all time to be constantly in flux. Instead, they're remarkably stable.
If you would have said one individual's taste is subjective or subject to more variance I might be able to agree, but with more data points it seems humans are able to more or less define good from bad objectively. Anything with 10,000 or more votes on imdb is likely within 5% of its final value with very few exceptions.
Personally I find anything over 7.4 on imdb watchable, over 7.8 quite good, over 8.4 incredible, and if it hits 8.9 or higher it's likely one of the best movies of all time.
So my definition of "good" is somewhere above a 7.4.
Duck Dynasty is widely popular with almost 9mm viewership. It has a rating of 6.3. Breaking Bad has 1.5mm viewership and an IMDB score of 9.5. If you're Netflix, do you make invest money into a show like Duck Dynasty or like Breaking bad.
Ya, I didn't cover them but there's certainly exceptions. Kids shows is a great example of a genre targeting a demographic which is likely under represented by the voting community - adults. Imdb is best viewed through the filter of "what do adults want to watch or think is good".
I remember watching duck dynasty, great kids show and I loved it, but I doubt it holds up for adults. Batman beyond on the other hand won an Emmy and still falls into the "kids show" genre but is very watchable for adults who can appreciate animated content.
So to answer your question, Netflix needs both as they represent very different audiences.
A few other exceptions that I didn't mention are comedies and horror. Ratings for these skew lower than average as they aren't meant to compete with Shawshank for greatest of all time, simply to entertain or scare. As such neither of these genres tend to attract serious directing, writing and acting talent and in turn, audiences tend to view them more as "fun" than serious.
You or I might love to watch pineapple express, hilarious comedy, but still vote it as a 6.9 because it doesn't hold a candle to Shawshank.
Duck dynasty isn’t a kids show. It’s a show that covers a family that makes duck calls. It’s the most popular non-fiction tv show ever created. I assume most people on hackernews haven’t seen an episode. I personally never watched it as it doesn’t appeal to me.
I remember watching duck dynasty, great kids show and I loved it, but I doubt it holds up for adults.
Duck Dynasty is a reality TV show, not a kids show.
I can't tell if you know that and are making a jab at them or not. If you are, then I think you're proving the point well, because you think it's clearly not a show for adults, while it was the most-watched nonfiction show in history. Clearly, tastes vary widely.
I think Netflix is going with the Blumhouse method of production[1]. They're giving a bunch of money to people and let them make stuff that (some) people will like.
Some of will be bad, most average and a tiny bit will make huge piles of money (bring new users).
The problem I have with this is that my available energy for finding the good stuff in a sea of mediocrity is pretty minimal. As a result I find myself not trying new stuff at all.
If most of the time when I tried something new it was good I'd probably do it more often.
Yeah, mustly rings true. There's actually very little I want to watch. Archer has become crap, and I've watched Rick and Morty and Arrested Development too many times already.
> I can probably count the amount of Netflix original movies that I've heard discussed with one hand.
This directly contradicts my experience.
I just checked my shows tracker and I have watched 32 shows (no movies included) that I would consider to be of some entertainment value and/or quality. I've discussed at least half of those with people, if not closer to 75%. I'd be happy to drop some recommendations if you give a genre or some preferences :)
That's very odd, because my experience and everyone I've ever seen commenting on it is the exact opposite: it seems like there's 80 new shows a week on Netflix and it's impossible to keep up. Every time I hear about a new show that's interesting there are 5 other ones up by the time I get around to watching it.
That's not been my experience. My recommendations (mostly Anime and Sci-Fi) is vastly different to my wife's (mostly American comedies with a strong emphasis on romcoms). The kids have a children's profile so theirs is vastly different again.
It's fair to say our individual viewing habits are vastly different to the others. Maybe if there was more overlap you might not notice any difference with the recommendations.
I am suspecting this strategy is a profit maximizing move. Instead of recommending 3rd party movie that cost high licence fees, Netflix is directing its users to its own content, where the cost is fixed. Netflix wants vast library to attract users but Netflix doesn't want them watch and only pay subscription.
Not to mention the utter crap they produce. Seriously, I can't say I have endured more than 3-5 good series or movies they have produced. I have started to watch a lot of them but instantly the low quality is a clear indication on what's to come.
Most of the stuff they produce are super political when they don't have to be, very bad writing or acting often with a great base story that could have been something great but filled with characters doing completely stupid things.
Some also have a really fucked up message due to the extreme woke-ness like "In the Shadow of the Moon". A movie which starts of pretty good and interesting and then quickly enters a downhill path to the extreme ideologies that seems to fuel companies like Netflix.
It makes sense to me because HBO doesn't really have the same type of catalog. They also don't have the data. In my experience, people subscribe to HBO because they have the content and people are already aware of their content. It makes curation way easier.
HBO Max is going to exclusively have the entire Friends collection and Ghibli collection. I know and am aware of what HBO has to offer.
On the flip side, Netflix is creating their own content. People have no idea what content is on Netflix. They have to try really hard to push content and generate engagement. I've never heard of Outer Banks, but they keep pushing it to me and so I saw the pilot and now I'm watching the series.
I watch one comedy show and now they keep throwing comedy shows at me so I saw and enjoyed another. Their need to create is different.
HBO Max has the entire WB + others back catalog, and represents some of the most productive studios out there. Rumor has it, they’re also going to be making exclusives.
Even knowing what they have across brands is nontrivial.
Why not? I know a lot of people who read and watch stuff that Bill Gates or Gwyneth Paltrow recommends. He's a celebrity. Other people have different celebrities. If you look up to a person, it's only natural to be interested in what they are interested in. People on HN think they are too good to care about celebrities, but we all know that's a total lie.
I would really dig lists from people I trust or follow. Get directors, actors, artists, editors, critics to really curate some lists and I'm in. Even better if they can add a couple of lines to tie the list within a theme or idea, not just a dump of series/films.
Only problem I see is it might end up being just marketing and they'll just use the celebrity name and dump the films they want to push.
Reminds me of how Yahoo! tried to differentiate itself from Google. There are companies that think they are in media, but really they are in search. I think the explosion of good quality content is turning the streaming services into search companies.
In my opinion Netflix tries too hard to make me watch more than I care to. In particular why can't I just by default enjoy the credits of a movie or series I just finished, instead I have to quickly use my remote so Netflix doesn't try show a trailer for something else, and there is no option to turn this off.
Relatedly their tvOS app gets laggy after just tens of seconds of browsing and it's been like this for over a year, how can they not fix it?
I don't know if netflix has enough content for curator aglorithms, at first it was recommending me some good older movies which was great but if netflix ever moves on to only originals and then only starts recommending those i will definitely cancel my subscription.
I doubt they will completely remove algorithms from the curation process. In my experience what is incredibly powerful is human curation ability multiplied by the leverage of algorithms.
The most straightforward example of this is for curation candidates to be proposed by algorithms, and then finalized by humans.
That approach works really well for things like creating digital maps from signals, and is employed pretty extensively by mapping companies as productivity multipliers.
Or even more familiar, it's what's behind things like smart select and magic wands in image editing tools.
It's pretty funny that even this headline shows the bizarre marketing problem WB created in choosing "HBO Max" as their streaming service name: HBO isn't taking on Netflix, the service is for all WB content.
Yep, and keeping the other services running at the same time is just stupid.
Its obvious HBO Now could be merged completely into Max since you get access to both if you have Now but the fact they still keep Go around still is a joke.
They should work on their search. I was trying to find the movie "Paul" and typing "Paul" in the search had the exact match as the fourth or fifth result in the upper right corner of the screen. It took me a while to find it.
One area where curation will fail is in "fancy bias". As in, the same bias that keeps crowd pleasing entertainment from winning awards.
People who are in the curation business (critics, reviewers, etc.) tend to favor things that make themselves look good to other people in the curation business. That is often opposite what the "unwashed masses" of people enjoy.
Prime example - Michael Bay movies. Michael Bay makes big, loud, entertaining movies with lots of explosions, bright lights, shiny objects, violence, and sex appeal. A "curator" is usually too snooty to recommend a movie like that.
An algorithm doesn't much care if a movie is artfully crafted, it only cares if people watch what is recommended. In the long run, an algo is more likely to give people what they want than a curator is.
If anything, curators over the long term seem to make a living telling people what they aren't supposed to like (or have access to).
I stopped really paying attention to Netflix's curation algorithms when I saw "The Santa Clause" categorized under "SciFi". I mean, I guess technically, but that's not really what I was looking for...
This is a hard problem. And it makes a great interview question too!
Recent PM interview question I've been using: You're the PM at Netflix handling the home screen. How do you determine how shows get promoted editorially vs algorithmically recommended? Walk through metrics/principles/trade-offs and how it impacts various parts of the biz.
This is a great question and I might steal this for future interviews. It's surprising how many of the people responding, despite being told optimizing for engagement is a cop-out, still jump to some kind of metric-based prescription. Or just jump right into throwing percentages for each content type (I mean, it's twitter, so sure, have a go).
My personal stab at this: this is really a strategic question. What are Netflix's long term goals? To be seen as HBO is, as a content-creator, or merely a content-provider? Both? What are the typical engagement rates for Netflix content vs purchased? What is the real, intangible (unmeasurable) objective we are maximizing for? Total hours watched? Or perceived value by consumer? Someone spending 25 hours a week with Netflix on still might perceive it as lower value than HBO that they spend 2 hours a week watching (and thus be more likely to cancel). Some forms of entertainment are easily substituted, as someone who uses TV for background noise wouldn't particularly mind to use Hulu for that purpose instead of Netflix. But a "must watch" show on HBO is just that, a must watch. Which customer do we want more of? More "background noise" customers implies a marginally higher infrastructure cost, while a more content focused strategy implies a higher cost for content production (and possibly a much more variable revenue stream, as people subscribe/cancel as their favorite show starts/stops airing). Maybe we want both, and we want to identify what kind of watcher a customer is, and then tailor their home screen to suit. Maybe one person has 90% Netflix Originals and the other has 10%.
If there is an answer here, it's to carefully weigh these strategic objectives and only then make changes to show promotion. Even then, it would be important to have some system of monitoring in place that allows you to confirm your changes are actually making measurable impact the objective in question (this may be very difficult, given an intangible goal of something like "increase Netflix mind-share". Probably a lot of marketing surveys and focus groups.) Straight-up engagement rate is just one of the many things to track here, and increasing it at any cost might not even be in Netflix's long term best interest.
You're hired! Great strategic answer that gets to the heart of the question--it's less about the solution and more about how you frame the problem and the underlying goals and assumptions.
Do you work in this space? It seems like you have a lot of context around OTT/streaming.
It introduces a nice concept of "marginal churn contribution" framed with bundling, but I think is also relevant to this discussion. Maybe more on the content sourcing/production side, but bleeds over into long-term goals (reducing churn/maximizing ltv/etc).
Thanks! I actually don't work in streaming/OTT, I manage a data science team for an auto insurance company. We end up thinking a lot about these kinds of questions from different business units that aren't quite sure what they want to be optimizing for, so I'm somewhat used to making the connection between high level strategy and measurable objectives. And as you might imagine churn/bundling/LTV are also very important concepts in the insurance space, which probably gave me a leg up in my response. The revenue model is at least superficially similar.
Thanks for the podcast link, it looks very relevant and will give it a listen. I see insurance is even explicitly addressed @ the 23 minute mark.
I'm not a PM nor hiring one but in my opinion you didn't answer the question. If the aim of the question is just to not answer the question and just raise more questions then it's a poor interview question in my opinion.
And from looking on Twitter, it's confirmed, imo, it's a bad question:
"As a lot of you have figured out instantly - this is really a iceberg question. Looks like a simple UX problem but is really about trying to understand Netflix as a system and as a business."
My kids used to laugh at Villager News, a dumb but occasionally funny skit show using 3D animation to copy Minecraft. Amazon Video now runs the show on its streaming service.
Basically YouTube became a method to make pilots for tv series.
Imagine all the weird but low cost documentaries i try and find on YouTube - Tods Workshop, Primitive Technology. There will be a thousand such people in thousands of niches.
Google could just flip on a content arm and hand each channel a producer and a decent camera and just own half the content. The other half is documentary / reportage rather than essays.
Honestly thinking of content as action shows or sci-fi 100 episodes is missing the point entirely. Content here on in is still high quality but aimed at a 100,000 people not 20 million.
> Amazon Video now runs the show on its streaming service.
In that scenario you're competing against YouTube, a place with abundant content with very different expectations for quality. People already go to YouTube for this content.
But the curation of lifting it out of youtube and into "mainstream" has a major discoverability effect - if youtube is just pure anarchy, selecting and promoting off youtube is ... feudalism?
HBO is betting on their content to be superior right now, but there are some obvious areas where the streaming experience isn't on par with Netflix. No 4K/HDR, one plan costing a premium that limits simultaneous streaming to only 3 screens, etc.
Hope that they catch up eventually else it's gonna be hard to compete solely based on content.
Netflix, Disney+, and Apple TV+ have also realized that adding more language tracks to their content is a _must_.
My family's viewership of Amazon Prime Video is severely limited because they refuse to add Spanish audio to their content. They have a few Spanish-language shows and movies but it's slim pickings.
HBO has no excuse on this, imho. I _know_ they have Spanish audio for every single one of their shows and movies - when we had cable TV we could switch language track in seconds, and this is not counting their Spanish-language channel that played in Spanish by default.
HBO's Spanish options are fantastic, I watch in a Spanish-speaking country and it's been a great service with good quality dubs and subtitles. We've run out of content though, I don't think we get all what the US gets.
I enjoy subtitles, my wife doesn't. I don't think I would enjoy subtitles when we're late-night binging until 2 or 3am.
Regardless, do Netflix, Apple and Disney have better engineers than HBO (and Amazon)? Do the latter two not care about offering Spanish (or non-English)? Do they have some contractual obligations to not offer the options online?
Turns out I don't care what the reason is. They don't offer extra language options, so I don't subscribe.
167 comments
[ 3.3 ms ] story [ 119 ms ] threadAnother problem is that if I watch 2 episodes in a row and I bookmark the page of the 2nd episode to finish it later, the URL in the address bar is actually still the URL of the 1st episode.
Yeah because netflix loves those robots it serves. At least netflix doesn't have a name problem... Three different HBO products... yeah that's gonna work. /s
But it doesn't actually work that way, 4 star uber drivers get kicked off the platform, 2 star Amazon products suck, 2-3 star movies suck. 4-star movies are okay.
As such, I think we can conclude that people don't actually know how to use a star rating system and we should just get rid of it.
HBO Max I'm on the fence about. I really like HBO but not sure I need or want everything that comes with the Max flavor.
Did everyone just forget about the Netflix Prize? [https://en.wikipedia.org/wiki/Netflix_Prize]
There was a brief time when they had the absolute best recommendation algorithm on the planet. It was intentionally replaced by something utterly different which is geared to simply steer everyone to IP that they own. It is not remotely similar. I have absolutely hated every change they've made to their UI since they ditched the old ratings system, too (forces you to use the mouse and sets you up for fumbling what should be clear actions, so you spend more time browsing - which costs them almost nothing - and less time watching). My guess is, research indicated they'd make more money with these changes than by licensing content audiences actually prefer and building a sane interface.
As it turns out, the Netflix golden age was a short-lived anomaly if not outright farce. If you want decent curation of content and also to be able to watch it, you will have to pay through the nose or pirate. Same as it ever was. It's consumer hostile, but the system is working as its overseers intend. Whether it's really the best long term strategy is still an open question in my book.
Same with Netflix, in my experience they truly know what I will like to watch, they seem to know what shows I'll love and what shows I won't love but will help pass the time, and seem to show it to me just when I want it.
I imagine it takes a lot of time and lots of past history and data on users to create those algorithms so when starting out the human curation method is required. And for companies like Apple, may be the long term strategy because the alogrithm method that YouTube and Spotify comes at the cost of potential tracking and privacy concerns.
Wow, I'd love to live in that world. Once Netflix abandoned their 5 star system their recommendations really tanked for me. It feels like it's recommending more based on outrage & "engagement" than enjoyment for me. Meanwhile MovieLens [1] consistently guesses what I would rate a movie within +/- half a star, I think I'm using their Warrior recommendation algorithm. The tagging features on MovieLens help - apparently I most enjoy "visually appealing surreal mysteries".
But I absolutely agree about Spotify. Spotify seems like its actually learning from my Liked tracks, curated playlists and behaviour, and picks up on when my tastes shift to wildly different genres for a while.
[1] https://movielens.org/home
It might be because of the service balkanization, but in the past I was able to find hidden gems on netflix, the same afterwards with prime video, but nowdays not only I have to browse in the text catalogs you might find online to quickly search among all the stuff I don't like, to find something I might enjoy that would certainly be buried in netflix UI. And even worse, I find myself rewatching my favorites instead of being able to find something different than blockbusters in the platforms.
Similarly, if I liked The Witch and Hereditary then top recommendations for me must be $5m budget cash grab teen slashers and haunted webcam gimmicks.
I think random selection would be more effective for me than the current Netflix algo(s).
Netflix bothers me because they have a marketing bend that creates your tunnels for you.
The article literally makes the point that Spotify's system is much more human curated than Netflix because it heavily relies on human curated lists that are shared between users.
HBO is operating on a much more tractable problem. Human curation may work better here.
As an avid music fan the ways I've sought out new material go as follows: - Originally, recommendations via trusted outlets / forum members - Not so long ago, Spotify algorithm recommendations - Currently, curated recommendations via trusted outlets / social media posters
My anecdotal experience between Netflix/HBO follows a similar pattern:
I tend to first go to a provider like HBO when seeking out new content, assuming quality is higher than whatever Netflix is pushing on me these days.
In contrast, a roommate of mine almost exclusively uses Netflix. She is more of the put on whatever, maybe pay attention maybe not type of viewer.
I guess summed up the main difference FOR ME between the two is HBO: "I need something good!" and Netflix: "I need something new!"
No one has found the "page rank of movies/series" yet, so I think HBO has a better chance here.
I'm not so sure -- I think the problem is no one has deployed it.
As a consumer, what I want is: based on my viewing history and/or shows I've said I like, show me more shows/movies I'll like.
What I currently get is: shows/movies that are in a similar style and genre to what I've watched or liked. The algorithm doesn't seem to take into account "good" vs "bad" movies.
The result is if I just randomly pick some movie from Netflix's "Because you watched.." recommendation, there's a very high chance I'm going to end up watching something I don't like because it isn't good.
As a concrete example, I just went to Netflix movies. "Because you watched Avengers: Infinity War...". There are 6 movies on the first page, 2 of which I've seen and have high Rotten Tomato scores (they are both Indiana Jones movies, with 88% and 85%). The other 4 have RT scores of 31%, 27%, 12%, and 3%. (Don't get me wrong, there are movies I do like that have low RT scores, and there are movies with high RT scores that I don't like, but generally a bad movie is a bad movie and more often than not there's a correlation with RT scores.)
Even if I could filter by "RT score > 60%" that would cut down on a lot of garbage, even if it means I might miss a couple movies/shows I'd actually like. This would change my personal perception from "The vast majority of content on Netflix isn't really worth watching" to something more positive. Note I still subscribe despite this -- there's still ample worthwhile content. I just don't watch things based on Netflix's recommendations alone.
Certainly it is possible to come up with something better than that blunt filter. So the question is: why don't they?
I'm sure Netflix has piles of data that shows this is the best strategy. It just seems like a differentiated curation strategy doesn't matter that much in this battle. It will come down to who is producing the most popular original content most consistently.
It's also good that market forces are driving content-creation.
Edit: cut extra word
Not when they compete with exclusive content.
It would be good if they had the same content but compete with the UI (in such case Netflix would win, and HBO would loose, as they have the worst possible UI), streaming content etc.
I suspect the pendulum will swing away from exclusive content for television shows. Producers might make more money up front from the deals, but the long game favors those who establish brands with broad market appeal, and the way to do that is to get your content in front of as many eyes as possible.
The Switch may be the most popular console, but it's install base is a small fraction of PS4+XBone+PC+Switch. So even Nintendo exclusives purchased by the majority of owners can't sell as many units as the latest Call of Duty. Heck, even the The Witcher 3 has sold more units than the best selling Switch game.
Ff7 remake is coming to pc, they just haven't figured out when.
There is only so much one can stand. If the market is spread to thin then pirating content will be less frustrating than getting it on streaming service (if you have to subscribe to 8 of them).
Personally I try to keep the number of services I'm subscribed to to a sane minimum.
I would prefer the competition to not be content but on UI, speed, downtime, viewing suggestions, etc.
The best part of cable was that you could get any channel regardless of your provider. You could have Spectrum, AT&T, Direct TV or whoever and get any channel you wanted.
The problem was that you usually could not do al la carte and when you could it would not be cheap enough to justify losing all the other channels.
Imagine if you had to pay for Time Warner cable to get CNN, AT&T to get HBO, and Comcast to get NBC.
Now you can get HBO a la carte, NBC prime time either for free on their website or with no commercials or with Hulu for $12.
Really there is little on network TV that I care to watch anymore. Between Netflix ($5 with Tmobile), Hulu w/ no commercials, Amazon Prime, AppleTV+ (free for now), and Disney+ (Mostly for the kids), I have plenty to watch. I sign up for other services (Starz, CBS, DCUniverse) occasionally to binge and then cancel.
When you pay for Spectrum you get TBS, CW, Fox, NBC, CBS, ABC, etc. When you pay for AT&T you get the same channels. It doesn't matter your cable provider, you would have access to pretty much any channel you wanted.
With Netflix, Hulu, Amazon, HBO, etc you have to pay for all of the services to get the content you want. This is beneficial for people who only want shows on a few streaming services but bad for people who now have to pay 10 different companies.
Ideally all the content creators would work together and make their content available to all of the streaming companies at a reasonable price even if they had their own streaming product. Paying for Netflix would provide the same shows as paying for Hulu.
Each of the streaming companies would then allow you to pay a set price for all the content like a traditional cable company (and their current business model) or allow you to pay for the shows you want. If you only want 5 shows you shouldn't have to subscribe to Netflix, Hulu and Disney+. You could pay Netflix and get all 5 shows instead of paying 3 different companies. Since you don't care about the other shows you shouldn't have to pay for the streaming company to obtain / keep the rights of those shows or to pay for them to create content you don't care about.
This is a pipe dream and will likely never happen.
If you would have said one individual's taste is subjective or subject to more variance I might be able to agree, but with more data points it seems humans are able to more or less define good from bad objectively. Anything with 10,000 or more votes on imdb is likely within 5% of its final value with very few exceptions.
Personally I find anything over 7.4 on imdb watchable, over 7.8 quite good, over 8.4 incredible, and if it hits 8.9 or higher it's likely one of the best movies of all time.
So my definition of "good" is somewhere above a 7.4.
Netflix has done decent drama along with some terrible reality shows (the one where men compete to impregnate one woman is notable https://www.theguardian.com/tv-and-radio/2020/may/21/labor-o... ).
I remember watching duck dynasty, great kids show and I loved it, but I doubt it holds up for adults. Batman beyond on the other hand won an Emmy and still falls into the "kids show" genre but is very watchable for adults who can appreciate animated content.
So to answer your question, Netflix needs both as they represent very different audiences.
A few other exceptions that I didn't mention are comedies and horror. Ratings for these skew lower than average as they aren't meant to compete with Shawshank for greatest of all time, simply to entertain or scare. As such neither of these genres tend to attract serious directing, writing and acting talent and in turn, audiences tend to view them more as "fun" than serious.
You or I might love to watch pineapple express, hilarious comedy, but still vote it as a 6.9 because it doesn't hold a candle to Shawshank.
https://en.m.wikipedia.org/wiki/Duck_Dynasty
Duck Dynasty is a reality TV show, not a kids show.
I can't tell if you know that and are making a jab at them or not. If you are, then I think you're proving the point well, because you think it's clearly not a show for adults, while it was the most-watched nonfiction show in history. Clearly, tastes vary widely.
Some of will be bad, most average and a tiny bit will make huge piles of money (bring new users).
https://en.wikipedia.org/wiki/Blumhouse_Productions
If most of the time when I tried something new it was good I'd probably do it more often.
As it stands I hardly use Netflix anymore.
Last movie I saw and liked at Netflix was The two popes. Netflix doesn't carry top series like
- Exit https://www.imdb.com/title/tt9612102/
- Le bureau des légendes https://www.imdb.com/title/tt4063800/
- Okkupert EDIT was/is available https://www.netflix.com/title/80092654
In the US & Canada Netflix has Babylon Berlin https://www.imdb.com/title/tt4378376/ but IMHO it is not up to above three.
Looking forward to be able to watch hbo-europe.com documentaries like
- The Other Side of Everything https://www.othersideofeverything.com/
- Lessons of Love https://www.youtube.com/watch?v=rM-wAlSFHZQ
For me they have that: Dark. It is top of the top, just like first seasons of Lost.
They also have Luter, Sherlock. Both of which are pretty good.
Couldn't agree more. Thanks for the tips. Watched all trailers, not my cup of tea.
This directly contradicts my experience.
I just checked my shows tracker and I have watched 32 shows (no movies included) that I would consider to be of some entertainment value and/or quality. I've discussed at least half of those with people, if not closer to 75%. I'd be happy to drop some recommendations if you give a genre or some preferences :)
It's fair to say our individual viewing habits are vastly different to the others. Maybe if there was more overlap you might not notice any difference with the recommendations.
Most of the stuff they produce are super political when they don't have to be, very bad writing or acting often with a great base story that could have been something great but filled with characters doing completely stupid things.
Some also have a really fucked up message due to the extreme woke-ness like "In the Shadow of the Moon". A movie which starts of pretty good and interesting and then quickly enters a downhill path to the extreme ideologies that seems to fuel companies like Netflix.
Try watching a bunch of Bollywood, K-drama or anime and you’ll see that the recommendations change drastically.
But yeah, they aren’t exactly trying to challenge their viewers to try something different.
Filter bubble is the term for when algorithms select the content you watch instead of you.
Filter bubble https://en.m.wikipedia.org/wiki/Filter_bubble
On the flip side, Netflix is creating their own content. People have no idea what content is on Netflix. They have to try really hard to push content and generate engagement. I've never heard of Outer Banks, but they keep pushing it to me and so I saw the pilot and now I'm watching the series. I watch one comedy show and now they keep throwing comedy shows at me so I saw and enjoyed another. Their need to create is different.
Even knowing what they have across brands is nontrivial.
ok so how do I know which human's taste is the same as mine. If only there was a recommendation engine for that.
HBO has human defined categories that makes sense.
Only problem I see is it might end up being just marketing and they'll just use the celebrity name and dump the films they want to push.
Here's a crowdsourced list:
https://letterboxd.com/peterstanley/list/1001-movies-you-mus...
I imported all of Roger Ebert's reviews:
https://letterboxd.com/re2/films/reviews/
His 381 "Great Movies":
https://letterboxd.com/re2/tag/great-movie/films/
A list of obscure films:
https://letterboxd.com/offensivename/list/official-top-50-na...
Listed tagged with "top ten":
https://letterboxd.com/tag/top-ten/lists/
This is super simple stuff.
HBO has IMDB.
I assume it's counterproductive to the goal of capturing eyeballs for as long as possible.
Relatedly their tvOS app gets laggy after just tens of seconds of browsing and it's been like this for over a year, how can they not fix it?
The most straightforward example of this is for curation candidates to be proposed by algorithms, and then finalized by humans.
That approach works really well for things like creating digital maps from signals, and is employed pretty extensively by mapping companies as productivity multipliers.
Or even more familiar, it's what's behind things like smart select and magic wands in image editing tools.
OTOH take 5 people selecting 15 movies every month https://cinetree.nl/over-ons: excellent selection, no algorithms. Or every week 1 or 2 top movies at https://www.vrt.be/vrtnu/a-z/#filter=categories%3Afilms. Or idfa.nl, etc, etc. None proposed by algorithms, only humans.
Similar to Youtube recommendations and how they are simply videos are most likely to watch through.
Its obvious HBO Now could be merged completely into Max since you get access to both if you have Now but the fact they still keep Go around still is a joke.
People who are in the curation business (critics, reviewers, etc.) tend to favor things that make themselves look good to other people in the curation business. That is often opposite what the "unwashed masses" of people enjoy.
Prime example - Michael Bay movies. Michael Bay makes big, loud, entertaining movies with lots of explosions, bright lights, shiny objects, violence, and sex appeal. A "curator" is usually too snooty to recommend a movie like that.
An algorithm doesn't much care if a movie is artfully crafted, it only cares if people watch what is recommended. In the long run, an algo is more likely to give people what they want than a curator is.
If anything, curators over the long term seem to make a living telling people what they aren't supposed to like (or have access to).
Recent PM interview question I've been using: You're the PM at Netflix handling the home screen. How do you determine how shows get promoted editorially vs algorithmically recommended? Walk through metrics/principles/trade-offs and how it impacts various parts of the biz.
https://twitter.com/sriramk/status/1222547047846297600
My personal stab at this: this is really a strategic question. What are Netflix's long term goals? To be seen as HBO is, as a content-creator, or merely a content-provider? Both? What are the typical engagement rates for Netflix content vs purchased? What is the real, intangible (unmeasurable) objective we are maximizing for? Total hours watched? Or perceived value by consumer? Someone spending 25 hours a week with Netflix on still might perceive it as lower value than HBO that they spend 2 hours a week watching (and thus be more likely to cancel). Some forms of entertainment are easily substituted, as someone who uses TV for background noise wouldn't particularly mind to use Hulu for that purpose instead of Netflix. But a "must watch" show on HBO is just that, a must watch. Which customer do we want more of? More "background noise" customers implies a marginally higher infrastructure cost, while a more content focused strategy implies a higher cost for content production (and possibly a much more variable revenue stream, as people subscribe/cancel as their favorite show starts/stops airing). Maybe we want both, and we want to identify what kind of watcher a customer is, and then tailor their home screen to suit. Maybe one person has 90% Netflix Originals and the other has 10%.
If there is an answer here, it's to carefully weigh these strategic objectives and only then make changes to show promotion. Even then, it would be important to have some system of monitoring in place that allows you to confirm your changes are actually making measurable impact the objective in question (this may be very difficult, given an intangible goal of something like "increase Netflix mind-share". Probably a lot of marketing surveys and focus groups.) Straight-up engagement rate is just one of the many things to track here, and increasing it at any cost might not even be in Netflix's long term best interest.
Do you work in this space? It seems like you have a lot of context around OTT/streaming.
Your answer also brought to mind this podcast episode I came across this morning: http://investorfieldguide.com/shishir-mehrotra-the-art-and-s...
It introduces a nice concept of "marginal churn contribution" framed with bundling, but I think is also relevant to this discussion. Maybe more on the content sourcing/production side, but bleeds over into long-term goals (reducing churn/maximizing ltv/etc).
Thanks for the podcast link, it looks very relevant and will give it a listen. I see insurance is even explicitly addressed @ the 23 minute mark.
And from looking on Twitter, it's confirmed, imo, it's a bad question:
"As a lot of you have figured out instantly - this is really a iceberg question. Looks like a simple UX problem but is really about trying to understand Netflix as a system and as a business."
Why doesn't that make it a great question? A PM should be trying to do exactly this, yes?
My kids used to laugh at Villager News, a dumb but occasionally funny skit show using 3D animation to copy Minecraft. Amazon Video now runs the show on its streaming service.
Basically YouTube became a method to make pilots for tv series.
Imagine all the weird but low cost documentaries i try and find on YouTube - Tods Workshop, Primitive Technology. There will be a thousand such people in thousands of niches.
Google could just flip on a content arm and hand each channel a producer and a decent camera and just own half the content. The other half is documentary / reportage rather than essays.
Honestly thinking of content as action shows or sci-fi 100 episodes is missing the point entirely. Content here on in is still high quality but aimed at a 100,000 people not 20 million.
In that scenario you're competing against YouTube, a place with abundant content with very different expectations for quality. People already go to YouTube for this content.
Hope that they catch up eventually else it's gonna be hard to compete solely based on content.
My family's viewership of Amazon Prime Video is severely limited because they refuse to add Spanish audio to their content. They have a few Spanish-language shows and movies but it's slim pickings.
HBO has no excuse on this, imho. I _know_ they have Spanish audio for every single one of their shows and movies - when we had cable TV we could switch language track in seconds, and this is not counting their Spanish-language channel that played in Spanish by default.
Regardless, do Netflix, Apple and Disney have better engineers than HBO (and Amazon)? Do the latter two not care about offering Spanish (or non-English)? Do they have some contractual obligations to not offer the options online?
Turns out I don't care what the reason is. They don't offer extra language options, so I don't subscribe.
It seems like to me that they use analytics to inform what kind of content to make but there are humans making decisions.