I've done this analysis for games. My theory was that the correlation was falling because the ratings of old games were dominated by players playing old games now and then rating them, and they were more likely to play games for which there is already a consensus that they are good (old bad games are mostly just forgotten). Whereas for current games the games being played are more random.
also consider the fact that you might have a 40 year old reviewing videogames when the target audience is like 4 years old. my kid and the typical video game reviewer probably have different opinions about the latest paw patrol game.
If you're 40 years old, you need to be able to consume any kind of media and then place yourself into the mindset of a younger person and judge it from that mindset - because you used to be that younger person.
Truly exceptional media is good enough to transcend the age barrier and is enjoyable for everyone, e.g., The Incredibles is enjoyable for young kids because it's a cartoon and it's flashy and exciting, it's enjoyable for young boys who want to be the "best" at a sport, it's enjoyable for girls transitioning to puberty because of Violet's storyline, it's enjoyable for men who have lost some of their steam and can relate to Mr. Incredible, and it's enjoyable for women because of the struggles Elastigirl has as a mother and a homemaker. Even Syndrome is shown as a sympathetic villain who is evil not because he is a bad person, but because he was dismissed and ignored by his idol. There are plenty of people who can relate to that.
Ghostbusters (1984) is another expertly crafted movie that is fun for kids, but well-written enough to be incredibly enjoyable for adults.
When my young cousins or my nephew corral me into playing a game with them, I'm not playing it from my 41-year old perspective, I'm playing it through their eyes.
If you can't do that, then don't review content. Content always has a specific audience. Really great content can weave together enough bits to at least satisfy every audience. Exceptional content manages to speak deeply to every audience. I still feel that's why The Incredibles is one of the best movies ever made. The struggles shown in that movie are relatable for every single person on the planet.
The funny thing is that games used to be FAR more punishing than they are today. Give an 8 year old (or hell, a 16 year old for that matter) Super Mario Bros. Watch their reaction when they get to World 1-3 or something and then lose all of their lives and realize that the game doesn't care, they have to start ALL the way from the beginning. Compare that to Elden Ring, which has difficult boss fights but otherwise isn't any harder than any level of one of those old games.
I recall as a small child ordering Castlevania from the Sears catalog and having to wait many weeks for it to arrive.
It would certainly be many more weeks before I would get any other game to play and there were a few other distractions.
>Compare that to Elden Ring, which has difficult boss fights but otherwise isn't any harder than any level of one of those old games.
I haven't played Elden, but for comparison's sake, I haven't had any trouble picking up SMB3 blind nor did I find any of the levels individually more difficult than most high rank monsters in Monster Hunter Rise. SMB1 definitely isn't a difficult game either bar a few specific levels, 8-3 being the most notorious.
The biggest thing I see people struggle with is patient, reactive and/or predictive gameplay. Any game with counter mechanics relying on tight windows showcases this: most people either fail to utilize them or simply don't bother. Let alone emergent counterplay. The other part, you can't just statstick your game through most older non-RPGs, where newer games provide you with many more methods to allow more failure.
I don't think Super Mario Bros was ever considered particularly difficult or punishing at the time. It was just designed for an audience that had less access to videogames. They'd rather repeat the same levels with the prospect of seeing a new one than breeze through the game they paid $25 for ($60 in today's money) in 2 hours.
It wasn't punishing for the time. That's not my point. My point is that being able to save games so that you don't have to replay hard content when you fail has made it so easy for the current generation of gamers that there are a lot who bemoan games that really aren't that hard comparatively. In Elden Ring, you only have to beat each boss once, and you have a save point right nearby to fight the boss without fighting almost anything else beforehand, so if you die, you're right back into it. In SMB, If you struggle on level 7-3 and then die on level 8-3, you start back at 1-1 and have to master them both again (provided you don't warp).
Most early console games had a heritage from arcades, which made more money if it was difficult but teased the chance of progress. So higher difficulty was more profitable, then when ported to console or when the same game designer designed for console it inhereted some of that. It was also more niche to be a gamer then, and those who did leaned more hardcore.
Plus, considerations for things like, can my 4-year-old decipher the controls to achieve what is achievable in the game without excessive frustration? Is the story entertaining enough to keep a kid engaged commensurate with it's cost? Is the game age appropriate? Does the kid talk about the game after having played it? Does it provide food for thought or conversation at all? What are the chances my child will remember this game fondly as they grow?
All of these things and more are the kinds of things that a 40 year old should consider in their review of a game, especially considering that a 16-25 year old game reviewer might miss on them in favor of other aesthetics.
Another part of that is that it seems like critics play a lot more games than the average consumer (or, at least, me) does and may get burnt out easier.
Most reviewers I watched heading into Horizon: Forbidden West said that it was just "more of the same". But I didn't play the most recent Far Cry or Assassin's Creed or whatever else they were comparing to. It was fresher for me playing though it than it was for them, constantly comparing.
I had a run of games a while back that were next to unplayable for the first 6 months while the bugs got ironed out and the patches got patched. Now I just wait 6 months or so (bonus - they're generally heavily discounted by then, haha) - if games critics are reviewing new releases but gamers reviews come in over the lifetime of the game, they're kind of reviewing two different products and it would make sense if critics' reviews were harsher.
On the other side of the coin, you can have situations like in 1989 when Jethro Tull beat Metallica for Best Hard Rock/Metal Performance at the Grammy's.
The Grammys have notoriously been considered out-of-touch for some time, even when you compare them to the other big awards shows (Tonys, Emmys, Oscars).
One of the interview questions I ask all UX Researchers is to open up rottentomatoes, sort recent films by rating and then look at the movies that have the lowest critic scores. Almost always you can see the audience score for those movies is much higher than the critic scores. I then ask why this is the case.
The overwhelming majority of answers involve critics being out of touch with the audience, critics being perhaps snobby, or watching movies in an elitist fashion whereas the audience just wants to sit back and relax, not think too much, just be entertained.
A very small minority of UX Researchers come to the correct conclusion, which is that most professional critics have to review every single movie, whereas the audience typically only goes to see movies that they are likely to enjoy, so that one should not be surprised that the audience score will often be significantly higher than the critic score.
People went to see these movies not because they wanted to per se but because critics said they were amazing. And because they went mostly due to critical reviews, anyone who didn't like it was even more likely than normal to be like "wait I feel tricked this was bad"
This might be a bit of a stretch, but it seems almost plausible.
- A film may have been marketed as one particular type of movie via trailers (e.g. as a horror film) but was actually a different type of movie, often a subversion of the genre it was marketed as (e.g. ended up as social commentary). So audiences feel deceived and report that in their ratings.
- Similarly, there are movies that are "applause lights" where critics are obliged to give positive ratings due to the social significance of a film (e.g. made by a woman / featuring a woman in a lead role). But the movie might just not be that good, and audiences report this in their ratings.
> social significance of a film (e.g. made by a woman / featuring a woman in a lead role)
for a less divisive example: movies about the film or theatre industry get high marks and fit your example while the audience is left scratching their heads
Excellent point. Critics have often remarked that movies about Hollywood tend to get Oscars just because of the subject matter, which Academy members like.
Also, a less cynical example: some movies are about something new: you might see people who live in a developing country, e.g., Somali fishermen in Captain Philips, or in unusual occupations (e.g., WWF wrestler in The Wrestler). Most audiences just want to relax and not learn something new, but critics love these kinds of movies.
Not GP, but looking at the relative difference graph[1] two hypotheses occur to me:
1) Most of the categories that are over-represented are ones that individuals watch out of obligation, rather than desire. A large portion of people watch documentaries (and biographies/historical films) because they feel they "should", because they feel like they should learn something.
Same for drama - these often include Oscar-bait that many people watch exclusively because it's nominated, not because they're actually interested in the movie itself.
Likewise, animation is (by volume) mostly synonymous with children's entertainment, so there are going to be a lot parents forced to watch what their kids are watching.
All of these line up with OP's hypothesis of obligation over choice.
2) War and westerns being overrepresented, well... film critics historically have a reputation of being Men of a Certain Social Demographic, and this is likely reflected even in modern film critic culture.
Never thought of that but it makes a lot of sense!
Edit: worth noting that this also creates an implicit prediction about the extent to which the advertising is an accurate representation of the movie. This could go in the opposite direction and audience scores would be lower than the critical assessment.
The comment literally states that it's one of the questions they ask, which is the exact opposite of a gotcha question.
And the specific question can lead to a conversation, which may "reveal" the answer (obviously, I don't know whether this is the correct answer), but at the very least it reveals whether the interviewee thinks about selection bias, among other concerns.
> The comment literally states that it's one of the questions they ask, which is the exact opposite of a gotcha question.
Huh? "Gotcha question" doesn't imply it's a single question that's not part of a series of questions. A "gotcha question" is simply a question designed such that it's very easy to respond with something that sounds bad or is incorrect.
Is it really easy not to come up with the desired explanation here? It seems like the most obvious possible answer, that people tend to watch the types of movies that they like. Actually it seems to provide almost no information unless their hiring pool is mostly candidates who utterly lack common sense.
The article claims that the difference between audiences and critics has been increasing over the last two decades. If the difference was only because audiences self-selected while critics had to review everything, why would it be increasing over time?
If anything, I'd expect that the audience would nown have more of a hit-and-miss experience than 20 years ago - with the abundance of streaming services at your fingertips, it's much easier to binge watch movies just because, whereas, say, having to rent a VHS tape had to bea more conscious and committal decision.
Actually, both statements boil down to the same with little modifications: (a) being out of touch with the audience is same as (b) the audience typically only going to see movies that they are likely to enjoy. (a) is broader than (b), can subsume (b).
If your narrative was true, we would always see critics scores lower than the audience (assuming statistically significant number of critics reviewing a given movie). But we see so many counterexamples with radically different ratings: Spy kids, Ad Astra, Noah, King Kong (2005), Babe, The Blair Witch Project, Chicken Run, and thousands of others.
Not to mention a number of films who stopped the audience scores, decrying "review bombing". Some of these films were fine, while some were objectively bad (-cough- Ghostbusters reboot -cough-). I have a feeling (though it is unsubstantiated rumor) that there are payoffs in the critics circles for certain movies.
It's possible that those movies failed to advertise to the correct audience. If they pulled in people who thought they were getting 1 experience, but ended up with a different experience, the reviews are going to be lower than if they hit the right audience.
Critics, as noted, see everything and so they wouldn't be affected by bad advertising.
I've definitely been wrong, in both directions, about movies based on their advertising.
I used to be a better professional film critic, mostly festival movies.
movie critics do watch the best movies. eg on a festival with 100-250 movies they watch from 30 to 50 typically. from these 30-50 there are about 20 good movies, the best movies of the year. from these regular moviegoers watch 1-2. normal people do not watch the best but the worst with the biggest ad budget.
of course critics don't watch every trash movie which starts every week. from 5-8 filmstarts per week, they watch 1-3. these weeks more, around 3, because now we have the best weeks of the year, with last year's Cannes movies arriving. and then again late fall, with the best Oscar movies being pushed. but eg this year's Oscars had only 2 good movies overall.
Fair point, it's categorically true that no critic watches every single movie released and I definitely overstated that. A brief search on Statista indicates that about 700 movies are released a year, the average American sees 2.3 movies a year (in theaters), and professional critics review over 200 movies a year, some even review 300.
I maintain that given this disparity (2.3 movies a year versus 200 a year) my point still stands, critics will tend to watch movies regardless of any predisposition towards them whereas an audience member will be much more discriminating about which 2.3 movies they will pay to see.
"A very small minority of UX Researchers come to the correct conclusion, which is that most professional critics have to review every single movie"
Could you provide your sources? This seems like it could be very incorrect, or only partially correct. Keep in mind rottentomatoes is owned by movie studios so they have a vested interest in changing the formula for disregarding audience ratings. I think they already disregard ratings below a certain threshold and other weird shit.
Then how do you explain that there are systematic differences for certain film genres? The article has a graphic on it. The differences are by no means uniform. Crowds love Action and Thriller, critiques History and Documentary. D'oh.
That is not to say there is no selection bias. But the population of critiques and film-going crowds are almost certainly different in education, lifestyle etc. That has certainly an effect on the observed discrepancies.
"The score, made up of audience ratings on a scale of 100, is calculated by taking the percentage of people who rated it at least 60 out of 100 (or 6 out 10) and multiplying it by 100%."
The critics score is calculated with a different formula. Do you discuss this with candidates?
Maybe research your interview questions a bit more before using them.
I absolutely do go over that, I also go over Metacritic and point out that this phenomenon applies to movies, video games, music, books, and almost all review systems that segregate professional reviews with customer reviews.
Calling that the "correct conclusion" is absurd. Whether or not a critic "wanted" to see a movie shouldn't really factor into the review at all. At the end of the day, the typical movie-goer is probably judging a film based off of an entirely different set of criteria (excitement / satisfaction) than a typical movie critic (overall artistic merit). Which isn't an exciting conclusion, but probably more correct.
I would personally rate many movies I've seen lower if you told me to judge it from the perspective of a movie critic as opposed to a general audience member (The Purge for example).
> audience typically only goes to see movies that they are likely to enjoy
That doesn't really explain the opposite phenomenon: movies that have higher critic rating than audience rating (e.g. movies like Mulan and Turning Red come to mind). IMHO, if you were to apply that rubric across all movies, the conclusion you should get is that critics and general audiences are simply different demographics and that it is normal for discrepancies to arise from sampling different demographics, not that general audiences are more biased towards what they like to watch. For example, parents wouldn't necessarily pick kids movies as their first choice of entertainment if age appropriate-ness wasn't a consideration (I think a lot of parents can relate to being sick of replaying Frozen, for example)
Also - a non-trivial percentage of ratings on Rotten Tomatoes & iMDB (and all review sites) come from accounts that have reviewed more movies than there is time in a human life to watch.
The amount fake user reviews can skew a rating is not to be taken lightly - and IIUC - Rotten Tomatoes & iMDB might be filtering out some/a lot of spam - but definitely not even close to all of it.
The same is true for GoodReads - the amount of teenage booktubers that have reviewed 10k+ books is awfully suspicious...
As someone that enjoys traveling and testing food places, the same applies to food critics.
Most of the time when I fall again into the trap of following their advices about some place, I come out disappointed, usually getting some creative dishes that still leave me wishing to go eat elsewhere.
I have discovered so many nice places just by randomly crashing into them, where one could watch from outside people were enjoying being there, regardless of how many stars the food critic gave it.
I think another acceptable explanation is that movies are marketed not to everyone but to certain people.
So say the latest horror flick that comes around is marketed to teens and 20-somethings. The over-30 crowd gets turned off by violence and gore. But that particular demographic loves it, and the more gore the more they like it.
There's a certain group of people that loved Human Centipede, e.g. But it got terrible reviews. Roger Ebert refused to award it stars, but gave it a few positive notes regardless.
And in it he even said:
I have long attempted to take a generic approach. In
other words, is a film true to its genre and does it
deliver what its audiences presumably expect? “The Human
Centipede” scores high on this scale. It is depraved and
disgusting enough to satisfy the most demanding midnight
movie fan. And it's not simply an exploitation film.
On the other hand, Ebert followed the critical consensus and gave Freddy Got Fingered (my personal pick for funniest movie of all time) a zero star rating. This is a gross-out comedy that succeeds spectacularly at what it set out to do, but very few critics were willing to judge it by the standards of the genre. The audience scores are very polarized (currently 56% positive on RT), but critic's reviews are only at 11%.
How does that challenge the argument that movies are marketed to certain groups of people? Gross out movies don't typically get marketed to soccer moms.
Wouldn't movie critics tend to be people to like movies in general? What about critics that stick to their preferred genre? They might start out pre-disposed to liking movies which could wipe out the effect of your assertion completely. On one hand, I think there is a good chance you are at least partially correct, but declaring your offhand intuition about movie criticism industry to be the "correct" answer is at best hubris.
You really don't think film critics being different from the general population has anything do with it? They're trained to see things in movies that most people don't. I get that you want people to say "selection bias" but that's not even close to the only significant factor at play here. You have two populations, which is also a classic statistical phenomenon. You even get close to this in your discussion of selection bias—they watch a lot more movies, and are thus more experienced—but you don't seem to want to allow this, which seems odd.
No, the correct conclusion is "I don't know". There is no real evidence one way or the other. Their unfounded speculation about human nature is just as plausible as yours. In the interview you can generate a hypothesis, not a conclusion.
This reminds me of the "why are manhole covers round" question, where people are judged by their ability to come up with clever wrong answers.
> most professional critics have to review every single movie
This isn't true. Quite far from it, in fact. Manohla Dargis at the NY Times, for example, averages only about 2 reviews per week. There are many more releases than that. Peter Travers at Rolling Stone does even fewer. Same with Justin Chang at the LA Times. Etc, etc.
IMO, the explanation is going to be multifactorial. Your "correct conclusion" probably explains some of the gap as critics are still obligated to review the prestige releases (even if they aren't necessarily interested in seeing them). But surely also does the divergence between the tastes of professional movie watchers and Joe Moviegoer explain the gap.
What's interesting is that in over-indexing on one bias, you've probably introduced another into your own hiring process. It does seem as though picking people who are careful, independent thinkers could still yield good hires. In the end, if you get a good candidate, who cares? I'd rather be right for the wrong reasons than just being wrong.
I reviewed films for the whole of the nineties for the BBC and local press in Manchester rarely missing any new releases. With this luxury and dream job though comes a responsibility and after a while it became obvious that I was suffering from Movie fatigue.
It was easy to dismiss much of Hollywood’s fare as formulaic and worthless because we had seen too much and seen better. Some of the movies I panned were perfectly good films and many were extremely successful and popular with audiences, but for me they were dull, and despite fabulous production values left me cold. Mark Kermode (Film Critic BBC) was one of my contemporaries and he said: “You have to see everything so that when something comes along that is good you will really notice it.” Of course, this exacerbated the fatigue and often meant that we were too quick to judge, and after a while, very little appealed because we had set ourselves far too high a standard to be realistic about the film’s quality.
If you look at the reviews in The Guardian, you will see more bad reviews that good. You might argue that a bad review is much better copy than a good review so there might be an editorial decision in there. To a certain extent Movie critics can be a poor parameter for quality and an injudicious contribution to its success or failure.
After ten years of reviewing films, I knew it was time to stop. I wasn’t enjoying it and I was probably doing some disservice to the film industry. Less is so much more. Harry Stafford – The Manchester Film School
"""
And, perhaps Manohla Dargis and Peter Travers watch far more movies than they write reviews for.
I wouldn't consider either of these to be the "correct" solution. You don't need to posit anything at all about how film criticism works. Occam's Razor suggests (ROT13):
Vg'f whfg erterffvba gb gur zrna pbzovarq jvgu erfgevpgvba bs enatr. Vs lbh unir gjb pbeeryngrq inevnoyrf K naq L, naq lbh cvpx gur zvavzhz A fnzcyrf sebz K, cebonoyl gur pbeerfcbaqvat A fnzcyrf sebz L jvyy unir uvture crepragvyr enax. Lbh pna pbasvez guvf ol fvzhyngvba. Gura pbzovar jvgu erfgevpgvba bs enatr: vg'f vzcbffvoyr gb tb orybj 0% fpber, fb ybj pevgvp fpberf "cvyr hc" ng mreb. Gura jura lbh ybbx ng gur pbeerfcbaqvat nhqvrapr fpber, nal aba-mreb inyhrf ng nyy jvyy envfr gur zrna. Ol flzzrgel, jr fubhyq rkcrpg gur zbivrf jvgu uvturfg pevgvp fpberf gb unir ybjre nhqvrapr fpberf, naq ivpr irefn va obgu pnfrf sbe ybj/uvtu nhqvrapr fpber.
A couple years ago, I could manage reading and writing ROT13 somewhat decently. I can still do it after some warm ups first. I can also do arbitrary Caesar cyphers in my head, but not fast enough to impress anyone. Memorizing the alphabet backwards helps, as does being able to visualize the numeric value of each letter. Embarrassingly, I'm actually rather slow at the simple mental arithmetic part.
Interesting article that shows via data that yes, critics are losing sync with audiences. Not that they were ever in sync? Although the data suggests that actually they are not that far off.
For me, I've always attributed a big difference of critics vs the general population is that critics have seen 10x the movies and so movies an audience finds interesting are often "been there, done that, and better" to a critic.
Other things I see which I'm not sure the cause and effect are movies that seem (to me) to be highly rated by critics for their message, direct or indirect, and not for whether they are actually good movies. An example to me (I know it's mistake to list one but I'm going to do it anyway), is Moonlight (2016). I'm not saying it's a bad movie but movie of the year? I saw it and other than remembering that it's about gay black men I remember absolutely nothing about the movie. It had zero impact for me. My feeling is it was chosen for its messages. That black men can be gay too. That black actors should be given more diversity in roles. That there should be more diverse movies with black leads. All of which I agree with. But the movie itself, while definitely a well made movie, had zero impact on me as a movie. I don't remember it. Conversely say, Women in the Dunes (1964), that movie will stick with me for the rest of my life.
Another random personal observation is it seems like "they" (the creators), have, at least for documentaries, figured out the formula for always making them crowd pleasers. They're the highest rated category both by my own experience looking up movies, and by the article. For me though, I've pretty much decided not to watch documentaries anymore because I see the formula and because I see they are super manipulative. Following the formula they can make you believe almost any conclusion about their topic.
There's an interesting concept with user reviews, in that they're inherently OF the people, whereas critical reviews are FOR the people.
In a way, critical reviews are like celebrity chef cookbooks in that they should be considered aspirational. What we HOPE to be watching to please our senses of austerity, versus the pulp trash we might actually prefer to be watching.
Similarly, sure, we might WISH we were making / eating boeuf en croute regularly, but if you go to a recipe site, the highest rated reviews is more likely to be chicken soup, or something more accessible / makeable on a weeknight.
It's always fun to review Plex and routinely see <95% critic ratings, 25% user ratings> or vice versa, but the deltas are more often closer than not.
I don't know if I have a point to make here other than to point out that there's utility in noting the distinctions between aspirational and practical, and that serious critics have always given off a vibe of divorcing themselves from the practical, preferring arthouse to action, and in both arenas, Julia Childs-esque figures that can bridge the gap between the aspirational and the practical are extremely rare.
Essentially that what we talk about liking and what the general population actually like are very different.
They interviewed this woman, who as an art student learned all about 'high art' and 'low art' and in a moment of loss of inspiration went back to using some low art she appreciated as inspirationn:
I don't know if this tells the entire story, but I think these are 2 of the bigger issues at play.
1) The corporatization of social media is a big factor. There's big money involved in controlling reviews. Critics and influencers devoted to certain fandoms inevitably get influenced, pressured, infiltrated by studios who expect favorable treatment towards their product or that entity loses access, gets attacked, etc. Try being a blogger who writes about something like Star Trek for example and consistently writing bad reviews. I'm sure you're going to lose any studio access for interviews and all kinds of good things if your criticism ever becomes a little too harsh.
2) Everything has become political, and the media class is totally out of touch from what average working people are looking for. A critic is going to want to rate some drama about a queer latinx non-binary vegan sex worker thats trying to come to grips with his atheism while interracial dating higher than the story deserves just to appear to have the "correct politics". Typical working people don't care as much about that, and just want to know if something is entertaining and worth their time and money.
> The corporatization of social media is a big factor. There's big money involved in controlling reviews. Critics and influencers devoted to certain fandoms inevitably get influenced, pressured, infiltrated by studios who expect favorable treatment towards their product or that entity loses access, gets attacked, etc.
Remember the online rehabilitation of the Star Wars prequels when Disney was spinning up their crap?
You bring up a good point and there's possibly an additional, simpler, effect: producers of big-budget films pay for good reviews, both by critics and by IMDb users.
My take with this movie is that the opposite of your #2 happened. Lots of regular folks who are concerned with global warming rated the movie highly, because they care about the political message it sends. Critics saw it as a pretty mediocre movie even though the message might be good.
I'm not saying you're wrong, but I do think it can go both ways.
EDIT: I looked again, and I realized the "Critic Consensus" and "Audience Says" sections on that page actually match my theory. For whatever that's worth.
Critic Consensus: Don't Look Up aims too high for its scattershot barbs to consistently land, but Adam McKay's star-studded satire hits its target of collective denial square on.
Audience Says: Although it can be heavy-handed with its messaging, Don't Look Up tackles important subjects with humor and heart.
> 1) The corporatization of social media is a big factor.
Yep. There was a time when I trusted internet/social media reviews because they weren't bought and paid for access media. But now, it's clear that they are also bought and paid for. They are now invited to press screenings and follow disney rules on release of spoiler reviews. Not only influencers but entire social media platforms are now arms of corporate PR.
> 2) Everything has become political
It's always been political. The only difference now is that ordinary people have a way of voicing their opinions. Hollywood, media, film critics, best sellers list, award shows, etc are all political propaganda. Always have been.
>A critic is going to want to rate some drama about a queer latinx non-binary vegan sex worker thats trying to come to grips with his atheism while interracial dating higher than the story deserves just to appear to have the "correct politics".
I like reading books which teach me new experiences, or ways of life I've never really thought about.
These types of movies are great because they expose me to subcultures and issues which aren't a part of my daily life. Even if I'm aware of them statistically understanding the emotional component to these kinds of stories is a worthwhile experience.
As long as the movie isn't simply bad, I'd give it a good review as well.
I think part of why this happens is that critics are part of the "creative" class of people. Creative people are more likely to be open to new ideas and explore, which is more likely to lead them to scenarios in their lives that look something like the non-binary vegan sex worker side of our world. And so, they find the dramatic look into the lives of marginal characters fascinating. I'm going to point to Silence.
To be a Christian in Japan during this part of the Edo Jidai is an especially marginal identity. It's not hard for a creative type who has seriously been on the margin at some point in their life to relate to that even if they are not Christian. It also doesn't hurt of course that the final message delivered is a pan-out view of his faith in its unconfessed nature which does indeed comport to a contemporary agenda summed up well in the movie's title; "Silence", as an imperative. "It is virtuous to make your sacrifices to Caesar/The Emprah, step on the fumie, and never speak about your religion in public because it causes other people pain and discomfort."
Interesting that the gap is with lower budget movies. It goes along with my gut feeling that mid budget movies are disappearing (I have no data to back that up), and it's as if the studios have optimized movie making into two types:
#1 The big budget crowd pleasing blockbusters. A few flop, but generally audiences like them and they don't do anything critics can complain about too much.
#2 Highly targeted low budget movies. There are small but consistent audiences with endless appetites for horror, action/thriller, etc films even at lower budgets. This seems to be where the biggest disconnect is, the genre fans highly rate their genre films even if it's direct to streaming fluff, while the critics see them as flawed, and broader audiences just don't watch them.
What killed the mid-budget movie was the rise of "peak TV" -- the B-list actors that defined mid-budget movies can land a meaty TV role that pays them well on a consistent basis. We're also seeing more and more A-list talent move to anthology or event series, where you can make a deep dive over 8/10/13 episodes instead of a 120-minute film.
Sometimes, I'd prefer some sort of limited series more than a movie. 'Maniac' was enjoyable for me because it had time to let things go slowly, despite taking course over what was probably 1 week in-universe.
I may also be different from the usual audience, in that I try to avoid binging anything. Hour-long episodes are great for my WFH lunch break, and to pad some time in the evenings if I so desire, but I prefer to commit to watching a movie if I know I have the time.
Molly's Game was a nice and long movie, but at over 2 hours long, I feel like it could have had potential at being a miniseries. But also, maybe not? Who knows how much meaningful material was left on the chopping block.
> Sometimes, I'd prefer some sort of limited series more than a movie. 'Maniac' was enjoyable for me because it had time to let things go slowly, despite taking course over what was probably 1 week in-universe.
Maniac is an excellent example. If it were a film, much of it would be cut down, and the movie likely would have had a singular focus on Jonah Hill's Owen character. The show, in having more time to breathe, realized relatively early on (by episode 3 or 4) that Emma Stone's Annie character had a far more compelling story to tell, and shifted its focus accordingly.
Matt Damon talks and laments about this. What killed the mid-budget movie was the collapse of DVD sales. Without DVD sales, there is no extended "second bite" that allows word of mouth to build up. So, instead, your marketing is $30-$50 million and you need to make that back.
Getting a second bite is also responsible for the recycled pablum with China pandering--if you say something that the Chinese government deems unacceptable you lose your shot at that market.
As for "episodic", I suspect that's less a "deep dive" and more ADHD background watching on a phone. "Encanto" was an absolute poster child for this--"HEY! LOOK UP NOW! HEEEEEEEEY!" "Okay, volume back down. You can go back to your text messaging for a while." "HEY! LOOK UP AGAIN! TIME TO PAY ATTENTION! HEEEEEEY!" ad nauseam.
General population has terrible taste, movie studios are now buying fake audience reviews to pump the ratings, they also buy critics but I assume they've always done that. Pretty hard to argue that critics are out of touch when the general population votes Avengers: Infinity War a top 10 movie of all time when it's released.
Infinity war if not in top 10, but is a very entertaining, well made action flick which deserves to be rated high. Probably one of the best action movies in the 2-3 decades.
Any mental model that assigns zero weight to the probability of being wrong is a wrong mental model.
That said, biases arising from endogeneity might have negative effects too. You can't conclude a parameter should have a different/zero sign just because of endogeneity, you have to go fix your model, and re-estimate the parameters.
> Any mental model that assigns zero weight to the probability of being wrong is a wrong mental model.
This is true when there is uncertainty. It also doesn't connect to "I only hire people who think like I do", which is the context of my response, so I don't follow your point.
Agreed on your other point that you need to instrument endogeneity. Another requirement is plausibility. Hookworm presence in Greenland should be uncorrelated to solar sunspots.
I don't know about in sync but I think there might have been more common ground in the past. Movies like The Godfather, Jaws, Goodfellas, Pulp Fiction or Shawshank Redemption sometimes feature highly on both critics and the public's favourite movie lists. I doubt many serious film critics are going to be putting any of the Marvel movies in their top 10 of all time yet there are large numbers of people out there who would.
Looking at the mechanisms alone I think its pretty unlikely that film critics have not shifted over time.
The standard film critic job in the 80s/90s was a local newspaper. That meant your audience was men/women of various ages and political beliefs who happened to live in the same area. If you were a successful critic you had to write something that appealed to all of them. This was actually a vital service that people used to determine if a movie was good.
Now the typical movie critic is online and has an audience which probably has a singular viewpoint. This isn't bad - but makes it unlikely the critics with "everyman" sensibilities would do well. People who consume an above average amount of film reviews will have their views represented more.
Chuck Klosterman (who was once a local newspaper movie reviewer and now writes to a certain type of audience online) has written about this.
Another possibilty is that audience scores are easier to gamify via the use of bots relative to critic scores (which I assume are limited to identified critics who work for major media outlets). One way for a movie's PR/advertising group to generate buzz is to flood rt with positive movie reviews for example.
I suppose you could detect this by correlating individual movie advertising budgets with rt audience scores across the board?
One possibly overlooked factor is the death of Roger Ebert, who managed to be both a man of the people and respected among critics (possibly because he was also an excellent writer).
No I did not come up with it myself. It's a common pitfall in statistics known as selection bias [1].
My problem is that if I ask a researcher about selection bias, they have no problem defining it for me and even giving me a textbook example of it. But when I present to a researcher a real life scenario that is highly susceptible to selection bias, they will almost all fail to identify it. It's a problem where many people are able to compartmentalize knowledge but fail to actually make use of that knowledge in practice.
As a run a quant firm I see this among so many researchers across many disciplines. They fully understand a concept when you ask them directly, but they fail to put those concepts to use outside of a very direct and artificial setting.
Do you agree that without proper research backing it up, you can’t say for sure that this discrepancy is due to selection bias, even if on the surface it looks like a nice fit?
If you had said that you’re looking for candidates to raise it as a possible explanation, instead of seemingly firmly asserting that it is the one and only correct answer, you’d probably have less replies arguing with you here (or perhaps if you do have such research readily available, to have shared it).
No I don't agree with you. I think my comment is a reasonable statement of the situation. Is it a 100% correct statement that is absolutely categorically true? No, but most people who are not looking to be pedantic and win an Internet argument won't go looking to nitpick the details of my statement.
What most level headed people will do is understand that my statement is about how you can not reliably compare a group of people who are predisposed to enjoy a movie to a group of people who review almost every single movie that's released, regardless of their predisposition towards it. People are welcome to argue that and honestly I'd expect nothing less, but it's a basic statistical principle that I expect every single researcher I hire to be well familiar with and if they're not, then I make the choice to not work with them.
>you’d probably have less replies arguing with you here...
There is also a selection bias among the people who comment on HN and they too are not representative of how most of the community feels towards a topic. The people who want to nitpick the specific wording of my statement are certainly entitled to do so and will likely find something to nitpick regardless of how I framed my statement, but it's not a particularly interesting discussion to follow up on.
Sure, though I think the other implication from the difference stated is that critics have sampled randomly or more often and can therefore recognize derivative work where a target audience that shares a limited interest in film will almost uniformly perceive novelty. I think realizing something is a poor knock off is the basis of snobbery.. so it is strange not to accept that answer.
That's a very strong assumption to make, I could easily assume that most casual movie viewers watch the same derivative knock-off action/horror/comedy films over and over and over again without issue. You think Adam Sandler is the highest earning actor/producer because his audience perceives a great deal of depth novelty in his work?
The answer that requires the fewest assumptions and is consistent with almost all basic statistical analysis is that a group of people predisposed to like a movie will rate that movie higher than a group of people who watch movies regardless of their predisposition towards it.
If you're using "selection bias" to refer to the fact that these two groups are selecting movies using different, biased methods, doesn't that imply that there's some way of sampling that would not be biased? But no matter what you do, an average critic and an average moviegoer are likely to disagree on many films. I would expect the explanations behind this to be more of a function of differences between the groups, which explains 1. why critics will occasionally enjoy a film more than the general population, and 2. why this disagreement can drift over time. Both of which are not accounted for by your analysis. I'm definitely not saying you're wrong - quite the opposite, just that there are other interesting things to discover here.
I suspect film might just be dead as an artistic medium that can be enjoyed by lots of people. I maybe go to a movie theatre once every couple of years these days, the rest of the time I watch things at home. It’s a hard ask to set aside two contiguous hours (or more!) to watch a film when there are so many good series or video games that demand less contiguous attention. People will rant about attention spans I guess but it’s really a matter of what quality of entertainment you can fit in with the rest of your life. If I watch a film in the evening that is pretty much the only thing I do with that evening; not so with other entertainment formats. The two hour film is also like the equivalent of a short story: there’s the mental barrier to entry of loading new characters and settings and world rules into your head, and so I find it more difficult to read anthologies of short stories than a novel of the same length.
Almost all my friends echo your sentiment that film is much harder to commit to than a TV show; and I can't disagree more. Watching a TV series means I'm committing to 8+ hours per season, across many days of my life. A film, even a 2 hour one, I watch in an evening and can enjoy taking it in in full, with all questions that it wants to answer itself being answered. A TV show will end with cliffhangers each episode just to make sure it demands your attention again tomorrow night.
That's not to say TV=bad, movies=good; just that IME respect your time way more and are easier to commit to most nights.
Both of you should give this lecture to people in the third world.
The act of sitting on a couch, watching something entertaining that thousands of people put work in to produce, is a "commitment". And in fact, the commitment is too much. Because for this same commitment, one could replace it with several smaller ones that entail the exact same thing, but with even less thinking.
If only there were a device that would just directly inject the memory of a movie into our brains, saves us from all this "effort".
It's also possible for critics to have systematically different preferences than movie-goers that don't deserve to be described pejoratively as "out of touch" or "snobby." In fact, that's precisely what I would expect and hope to see from people who are professional critics. I wouldn't want expect a list of best restaurants from a food critic to be identical to the list of top restaurant chains by revenue (1: McDonalds, 2: Starbucks, etc.).
> I wouldn't want expect a list of best restaurants from a food critic to be identical to the list of top restaurant chains by revenue
That's a very good point that the role of the critic is to provide their own expertise and opinions on the overall subject. But the critic still has to be within the ballpark of popular tastes if the expectation is a mass audience that really trusts their recommendations, right?
Many Americans might listen to a food critic that said that he thought some pasta dish at some Italian restaurant was the best, even if pasta is not currently everybody's favorite food. But if he he recommended boiled snails or braised calf brains, many people might ignore his judgement because it's a bit too far out of range of what they're comfortable with. At a certain distance from popular opinion, a mass audience might no longer trust that critic. Just a thought.
> But the critic still has to be within the ballpark of popular tastes if the expectation is a mass audience that really trusts their recommendations, right?
Well, no, because I'm pretty sure most film critics aren't trying to directly provide a "yes or no" recommendation to a general movie-going audience. I've read plenty of film criticism that was entertaining and informative despite me having very little shared taste with the writer. And I suspect if you asked film critics to straight up predict what the RottenTomatoes audience score for a film would be, the aggregate of film critics' predictions would tend to be closer to the RT audience score than the RT critic score itself.
> But the critic still has to be within the ballpark of popular tastes if the expectation is a mass audience that really trusts their recommendations, right?
Not really. A professional critic usually writes a longish review describing the item and why they liked or disliked it, as well as the conclusion. If they are good at description, that's useful to me even if I don't share their likes and dislikes. If they hate things I love, then I can look for their negative reviews as places to start. If they love things I hate, their seal of approval is a good sign for me to keep walking. I can trust their recommendations even if I don't follow them, if I think they are fair and honest and have consistent opinions. If their opinions blow with the wind, I probably won't trust their opinions, even if they agree with mine.
I found Anthony Bourdain trustworthy, but I do not like a lot of the dishes that he really did (no thanks to organ meat and blood sausage), so I wouldn't blindly follow his suggestions.
McDonalds is much cheaper than a fancy restaurant, so it should come as no surprise that it has worse quality. But with movies, people are chosing the lower rated movies even though it doesn't save money.
> It's also possible for critics to have systematically different preferences than movie-goers that don't deserve to be described pejoratively as "out of touch" or "snobby."
When the person writing a review has "systematically different preferences" from the people they're writing for, "out of touch" is exactly the right description for that ("snobby" - not necessarily).
Or put another way, maybe you’re not the intended audience if you’re reading a film critic’s work only to decide if you might like a film and you repeatedly find that your tastes are very different than that critic’s.
Have you ever considered that there might actually be something to "ooh lala, fancy pancy hard to understand arty farty" movies that you are merely unable to appreciate and that your failure to appreciate any virtue doesn't constitute proof of its nonexistence?
The whole point of pursuing the opinion of critics isn't averaging them to see what smart people like its in finding critics whose taste isn't wholly out of line with yours and using recommendations to cut through the crap and find things worth watching.
I used to love artsy stuff then I realized that often they are not any better or deeper they just appear to be or the depth and meaning is projected on them through interpretation which has its place I guess.
Artsy movies are great... sometimes, and unartsy movies are also great sometimes.
Sometimes I want to watch the Seventh Seal and sometimes I want to watch Die Hard - both movies have their place but a lot of critics tend to lean strongly into arthouse movies and sell them well above their actual appeal. I think one of the hardest parts of being a genuinely good critic is being able to leave your own tastes at the door when walking into a new performance - and those tastes are generally ones that have been honed by an education in classic works. Some movies use extremely bold cinematography choices to add a lot to their work (I'd point out Sin City and Schindler's List both of which made extremely good use of high contrast black and white (mostly) filming to add heaps to the story) but then you've got so-so art-house pieces that play into those familiar tropes without delivering anything of real value.
It takes a really good critic to watch a movie they really personally enjoyed and tell everyone that it's probably not for them - please do talk about what you liked to inform like minded people - but don't give a 5/5 just because it appealed to your specific tastes.
I think the most valuable thing a critic brings to the table is a wide breadth of experiences to compare new experiences to - you're watching all the crappy movies to pick out the good ones to recommend so that people don't waste their time on something flat.
I wouldn't base any statistics on IMDB ratings, they're garbage.
Just look at how many awful amateurish experiments there receive spectacular 10 star reviews within days of their release, very likely by cast and crew friends/relatives (or worse), then are buried by 1 star reviews when actual viewers eventually watch them.
Do you maybe have an example? I'm always astounded how well the IMDB score matches how likeable a film is (minus personal preferences of course). 5 is basically always really bad unless I'm a fan of a specific actor/director/genre. The comments are mixed, the comments below the YouTube trailers tend to be quite on point though to see if a film is worth watching
Begs the question as to how much Mr Plinkett changed the review landscape? Long form anal sis including the meta, the craft, and how the soup was made. Throw in every frames a painting as maybe another OG. Going beyond is it good or bad x/10 to something else entirely. 6 hour review of the 2 hour phantom menace. Or in efap, what it means to pan right or left. Flexing the knowledge of a critic and thereby proving the worth of film studies, often maligned as useless, allowing us to see through their eyes. Trusting the audience and giving them tools to be a critic themselves.
“You might not have noticed, but your brain did”
Now there’s mauler, critical drinker, er, to name a few. Going beyond entertainment, long form analysis of politics and more. Go beyond the press release and the status quo.
I’m conflating a lot of things here purely because it would be funny if mr plinkett could be shown to have revolutionised the world in ways the “pretentious snob” could only dream.
No, all of the awards are pretty much lifetime achievement awards that happen to coincide with some movie. 'Sure it's not his best work, but he deserves an Oscar.'
> the Academy largely votes the way the critics tell them to
This is absolutely not the case. For example "Oscar campaigns" have significant influence on the Academy, and they happen after the critics have given their verdict.
Most entertainment-oriented films. There's a big difference between "has a message" (which those usually do include) and "lecture" aka "beat you over the head with".
I think most blockbuster "entertainment" movies are MORE heavy handed in their preaching. The big differentiator for me is that they preach what is already assumed. It's not that they don't lecture you, it's that they lecture you in what you already agree with. In that sense entertainment-movies are more like sermons from your pastor. It's a reframing of a story you've already been told a million times, so your brain can just switch off and absorb the message uncritically.
Yes. The typical modern blockbuster constantly hits you over the head with its messages of "family" or "finding yourself" or "overcoming obstacles" of whatever like you're still in grade school. Compare that to something like No Country for Old Men - a movie with a much more nuanced set of ideas. Or even to older blockbusters like Die Hard which are more consistently focused on the action and keep the moralizing to a minimum.
i don't know if you've noticed, but people who do things that i understand are totally normal, and people who are different from me have dumb ideas. it's annoying when they talk.
Generally, popular films outside of documentaries don't lecture. It may be that you feel insecure and talked down to? Do you feel that way in normal human conversation?
Almost every sort of film, IME. I can hardly recall a film that 'lectured', if we mean the same thing, and it would be panned if it did.
For one thing, if the filmmaker can't convey things without lecturing, they are in completely the wrong job. Does a painter have to add call-out boxes with labels? Second, they almost never have a specific message, but depict the world - art holds a mirror to the world (very generally speaking).
411 comments
[ 3.4 ms ] story [ 339 ms ] threadIf you're 40 years old, you need to be able to consume any kind of media and then place yourself into the mindset of a younger person and judge it from that mindset - because you used to be that younger person.
Truly exceptional media is good enough to transcend the age barrier and is enjoyable for everyone, e.g., The Incredibles is enjoyable for young kids because it's a cartoon and it's flashy and exciting, it's enjoyable for young boys who want to be the "best" at a sport, it's enjoyable for girls transitioning to puberty because of Violet's storyline, it's enjoyable for men who have lost some of their steam and can relate to Mr. Incredible, and it's enjoyable for women because of the struggles Elastigirl has as a mother and a homemaker. Even Syndrome is shown as a sympathetic villain who is evil not because he is a bad person, but because he was dismissed and ignored by his idol. There are plenty of people who can relate to that.
Ghostbusters (1984) is another expertly crafted movie that is fun for kids, but well-written enough to be incredibly enjoyable for adults.
When my young cousins or my nephew corral me into playing a game with them, I'm not playing it from my 41-year old perspective, I'm playing it through their eyes.
If you can't do that, then don't review content. Content always has a specific audience. Really great content can weave together enough bits to at least satisfy every audience. Exceptional content manages to speak deeply to every audience. I still feel that's why The Incredibles is one of the best movies ever made. The struggles shown in that movie are relatable for every single person on the planet.
I haven't played Elden, but for comparison's sake, I haven't had any trouble picking up SMB3 blind nor did I find any of the levels individually more difficult than most high rank monsters in Monster Hunter Rise. SMB1 definitely isn't a difficult game either bar a few specific levels, 8-3 being the most notorious.
The biggest thing I see people struggle with is patient, reactive and/or predictive gameplay. Any game with counter mechanics relying on tight windows showcases this: most people either fail to utilize them or simply don't bother. Let alone emergent counterplay. The other part, you can't just statstick your game through most older non-RPGs, where newer games provide you with many more methods to allow more failure.
All of these things and more are the kinds of things that a 40 year old should consider in their review of a game, especially considering that a 16-25 year old game reviewer might miss on them in favor of other aesthetics.
When you get right down to it, it’s a movie about exterminators—not exactly fun-for-kids fare. And yet…
Most reviewers I watched heading into Horizon: Forbidden West said that it was just "more of the same". But I didn't play the most recent Far Cry or Assassin's Creed or whatever else they were comparing to. It was fresher for me playing though it than it was for them, constantly comparing.
I bet that critics in 1989 overwhelmingly preferred Metallica’s album to Jethro Tull’s, which at this point was kind of a has-been.
Honestly, gaining trust of audiences is also part of a critic. Else you're just a loud contrarian/populist.
The overwhelming majority of answers involve critics being out of touch with the audience, critics being perhaps snobby, or watching movies in an elitist fashion whereas the audience just wants to sit back and relax, not think too much, just be entertained.
A very small minority of UX Researchers come to the correct conclusion, which is that most professional critics have to review every single movie, whereas the audience typically only goes to see movies that they are likely to enjoy, so that one should not be surprised that the audience score will often be significantly higher than the critic score.
Those researchers are typically the ones I hire.
I assume "out of touch" but almost any explanation I think of and ridicule could be seen as a strawman argument, so I'll wait for your response.
People went to see these movies not because they wanted to per se but because critics said they were amazing. And because they went mostly due to critical reviews, anyone who didn't like it was even more likely than normal to be like "wait I feel tricked this was bad"
This might be a bit of a stretch, but it seems almost plausible.
- A film may have been marketed as one particular type of movie via trailers (e.g. as a horror film) but was actually a different type of movie, often a subversion of the genre it was marketed as (e.g. ended up as social commentary). So audiences feel deceived and report that in their ratings.
- Similarly, there are movies that are "applause lights" where critics are obliged to give positive ratings due to the social significance of a film (e.g. made by a woman / featuring a woman in a lead role). But the movie might just not be that good, and audiences report this in their ratings.
for a less divisive example: movies about the film or theatre industry get high marks and fit your example while the audience is left scratching their heads
Also, a less cynical example: some movies are about something new: you might see people who live in a developing country, e.g., Somali fishermen in Captain Philips, or in unusual occupations (e.g., WWF wrestler in The Wrestler). Most audiences just want to relax and not learn something new, but critics love these kinds of movies.
1) Most of the categories that are over-represented are ones that individuals watch out of obligation, rather than desire. A large portion of people watch documentaries (and biographies/historical films) because they feel they "should", because they feel like they should learn something.
Same for drama - these often include Oscar-bait that many people watch exclusively because it's nominated, not because they're actually interested in the movie itself.
Likewise, animation is (by volume) mostly synonymous with children's entertainment, so there are going to be a lot parents forced to watch what their kids are watching.
All of these line up with OP's hypothesis of obligation over choice.
2) War and westerns being overrepresented, well... film critics historically have a reputation of being Men of a Certain Social Demographic, and this is likely reflected even in modern film critic culture.
[1] https://stephenfollows.com/wp-content/uploads/2021/03/Differ...
Edit: worth noting that this also creates an implicit prediction about the extent to which the advertising is an accurate representation of the movie. This could go in the opposite direction and audience scores would be lower than the critical assessment.
And the specific question can lead to a conversation, which may "reveal" the answer (obviously, I don't know whether this is the correct answer), but at the very least it reveals whether the interviewee thinks about selection bias, among other concerns.
Huh? "Gotcha question" doesn't imply it's a single question that's not part of a series of questions. A "gotcha question" is simply a question designed such that it's very easy to respond with something that sounds bad or is incorrect.
Hiring pools tend to be filled with nonsense candidates
Critics, as noted, see everything and so they wouldn't be affected by bad advertising.
I've definitely been wrong, in both directions, about movies based on their advertising.
movie critics do watch the best movies. eg on a festival with 100-250 movies they watch from 30 to 50 typically. from these 30-50 there are about 20 good movies, the best movies of the year. from these regular moviegoers watch 1-2. normal people do not watch the best but the worst with the biggest ad budget.
of course critics don't watch every trash movie which starts every week. from 5-8 filmstarts per week, they watch 1-3. these weeks more, around 3, because now we have the best weeks of the year, with last year's Cannes movies arriving. and then again late fall, with the best Oscar movies being pushed. but eg this year's Oscars had only 2 good movies overall.
for comparison: my letterboxd https://letterboxd.com/rurban/films/diary/
I maintain that given this disparity (2.3 movies a year versus 200 a year) my point still stands, critics will tend to watch movies regardless of any predisposition towards them whereas an audience member will be much more discriminating about which 2.3 movies they will pay to see.
But there's no evidence that demonstrates either possibility (or if the reality is a 3rd, 4th or 5th possibility).
Could you provide your sources? This seems like it could be very incorrect, or only partially correct. Keep in mind rottentomatoes is owned by movie studios so they have a vested interest in changing the formula for disregarding audience ratings. I think they already disregard ratings below a certain threshold and other weird shit.
That is not to say there is no selection bias. But the population of critiques and film-going crowds are almost certainly different in education, lifestyle etc. That has certainly an effect on the observed discrepancies.
The critics score is calculated with a different formula. Do you discuss this with candidates?
Maybe research your interview questions a bit more before using them.
I would personally rate many movies I've seen lower if you told me to judge it from the perspective of a movie critic as opposed to a general audience member (The Purge for example).
That doesn't really explain the opposite phenomenon: movies that have higher critic rating than audience rating (e.g. movies like Mulan and Turning Red come to mind). IMHO, if you were to apply that rubric across all movies, the conclusion you should get is that critics and general audiences are simply different demographics and that it is normal for discrepancies to arise from sampling different demographics, not that general audiences are more biased towards what they like to watch. For example, parents wouldn't necessarily pick kids movies as their first choice of entertainment if age appropriate-ness wasn't a consideration (I think a lot of parents can relate to being sick of replaying Frozen, for example)
The amount fake user reviews can skew a rating is not to be taken lightly - and IIUC - Rotten Tomatoes & iMDB might be filtering out some/a lot of spam - but definitely not even close to all of it.
The same is true for GoodReads - the amount of teenage booktubers that have reviewed 10k+ books is awfully suspicious...
Most of the time when I fall again into the trap of following their advices about some place, I come out disappointed, usually getting some creative dishes that still leave me wishing to go eat elsewhere.
I have discovered so many nice places just by randomly crashing into them, where one could watch from outside people were enjoying being there, regardless of how many stars the food critic gave it.
So say the latest horror flick that comes around is marketed to teens and 20-somethings. The over-30 crowd gets turned off by violence and gore. But that particular demographic loves it, and the more gore the more they like it.
There's a certain group of people that loved Human Centipede, e.g. But it got terrible reviews. Roger Ebert refused to award it stars, but gave it a few positive notes regardless.
And in it he even said:
https://www.rogerebert.com/reviews/the-human-centipede-2010You interview people, ask them their opinions, and hire those that agree with your opinion.
The most excellent way I've seen to hire a group of people biased toward a singular way of thinking.
This reminds me of the "why are manhole covers round" question, where people are judged by their ability to come up with clever wrong answers.
This isn't true. Quite far from it, in fact. Manohla Dargis at the NY Times, for example, averages only about 2 reviews per week. There are many more releases than that. Peter Travers at Rolling Stone does even fewer. Same with Justin Chang at the LA Times. Etc, etc.
IMO, the explanation is going to be multifactorial. Your "correct conclusion" probably explains some of the gap as critics are still obligated to review the prestige releases (even if they aren't necessarily interested in seeing them). But surely also does the divergence between the tastes of professional movie watchers and Joe Moviegoer explain the gap.
What's interesting is that in over-indexing on one bias, you've probably introduced another into your own hiring process. It does seem as though picking people who are careful, independent thinkers could still yield good hires. In the end, if you get a good candidate, who cares? I'd rather be right for the wrong reasons than just being wrong.
I reviewed films for the whole of the nineties for the BBC and local press in Manchester rarely missing any new releases. With this luxury and dream job though comes a responsibility and after a while it became obvious that I was suffering from Movie fatigue.
It was easy to dismiss much of Hollywood’s fare as formulaic and worthless because we had seen too much and seen better. Some of the movies I panned were perfectly good films and many were extremely successful and popular with audiences, but for me they were dull, and despite fabulous production values left me cold. Mark Kermode (Film Critic BBC) was one of my contemporaries and he said: “You have to see everything so that when something comes along that is good you will really notice it.” Of course, this exacerbated the fatigue and often meant that we were too quick to judge, and after a while, very little appealed because we had set ourselves far too high a standard to be realistic about the film’s quality.
If you look at the reviews in The Guardian, you will see more bad reviews that good. You might argue that a bad review is much better copy than a good review so there might be an editorial decision in there. To a certain extent Movie critics can be a poor parameter for quality and an injudicious contribution to its success or failure.
After ten years of reviewing films, I knew it was time to stop. I wasn’t enjoying it and I was probably doing some disservice to the film industry. Less is so much more. Harry Stafford – The Manchester Film School
"""
And, perhaps Manohla Dargis and Peter Travers watch far more movies than they write reviews for.
Vg'f whfg erterffvba gb gur zrna pbzovarq jvgu erfgevpgvba bs enatr. Vs lbh unir gjb pbeeryngrq inevnoyrf K naq L, naq lbh cvpx gur zvavzhz A fnzcyrf sebz K, cebonoyl gur pbeerfcbaqvat A fnzcyrf sebz L jvyy unir uvture crepragvyr enax. Lbh pna pbasvez guvf ol fvzhyngvba. Gura pbzovar jvgu erfgevpgvba bs enatr: vg'f vzcbffvoyr gb tb orybj 0% fpber, fb ybj pevgvp fpberf "cvyr hc" ng mreb. Gura jura lbh ybbx ng gur pbeerfcbaqvat nhqvrapr fpber, nal aba-mreb inyhrf ng nyy jvyy envfr gur zrna. Ol flzzrgel, jr fubhyq rkcrpg gur zbivrf jvgu uvturfg pevgvp fpberf gb unir ybjre nhqvrapr fpberf, naq ivpr irefn va obgu pnfrf sbe ybj/uvtu nhqvrapr fpber.
For me, I've always attributed a big difference of critics vs the general population is that critics have seen 10x the movies and so movies an audience finds interesting are often "been there, done that, and better" to a critic.
Other things I see which I'm not sure the cause and effect are movies that seem (to me) to be highly rated by critics for their message, direct or indirect, and not for whether they are actually good movies. An example to me (I know it's mistake to list one but I'm going to do it anyway), is Moonlight (2016). I'm not saying it's a bad movie but movie of the year? I saw it and other than remembering that it's about gay black men I remember absolutely nothing about the movie. It had zero impact for me. My feeling is it was chosen for its messages. That black men can be gay too. That black actors should be given more diversity in roles. That there should be more diverse movies with black leads. All of which I agree with. But the movie itself, while definitely a well made movie, had zero impact on me as a movie. I don't remember it. Conversely say, Women in the Dunes (1964), that movie will stick with me for the rest of my life.
Another random personal observation is it seems like "they" (the creators), have, at least for documentaries, figured out the formula for always making them crowd pleasers. They're the highest rated category both by my own experience looking up movies, and by the article. For me though, I've pretty much decided not to watch documentaries anymore because I see the formula and because I see they are super manipulative. Following the formula they can make you believe almost any conclusion about their topic.
In a way, critical reviews are like celebrity chef cookbooks in that they should be considered aspirational. What we HOPE to be watching to please our senses of austerity, versus the pulp trash we might actually prefer to be watching.
Similarly, sure, we might WISH we were making / eating boeuf en croute regularly, but if you go to a recipe site, the highest rated reviews is more likely to be chicken soup, or something more accessible / makeable on a weeknight.
It's always fun to review Plex and routinely see <95% critic ratings, 25% user ratings> or vice versa, but the deltas are more often closer than not.
I don't know if I have a point to make here other than to point out that there's utility in noting the distinctions between aspirational and practical, and that serious critics have always given off a vibe of divorcing themselves from the practical, preferring arthouse to action, and in both arenas, Julia Childs-esque figures that can bridge the gap between the aspirational and the practical are extremely rare.
https://freakonomics.com/podcast/dont-worry-be-tacky/.
Essentially that what we talk about liking and what the general population actually like are very different.
They interviewed this woman, who as an art student learned all about 'high art' and 'low art' and in a moment of loss of inspiration went back to using some low art she appreciated as inspirationn:
https://news.artnet.com/art-world/flora-yukhnovich-2077868
1) The corporatization of social media is a big factor. There's big money involved in controlling reviews. Critics and influencers devoted to certain fandoms inevitably get influenced, pressured, infiltrated by studios who expect favorable treatment towards their product or that entity loses access, gets attacked, etc. Try being a blogger who writes about something like Star Trek for example and consistently writing bad reviews. I'm sure you're going to lose any studio access for interviews and all kinds of good things if your criticism ever becomes a little too harsh.
2) Everything has become political, and the media class is totally out of touch from what average working people are looking for. A critic is going to want to rate some drama about a queer latinx non-binary vegan sex worker thats trying to come to grips with his atheism while interracial dating higher than the story deserves just to appear to have the "correct politics". Typical working people don't care as much about that, and just want to know if something is entertaining and worth their time and money.
Remember the online rehabilitation of the Star Wars prequels when Disney was spinning up their crap?
As for "2,", come on, it's ALWAYS been political. It's mostly always been this way except for changes in which particular groups you've named.
https://www.rottentomatoes.com/m/dont_look_up_2021
55% Tomatometer, 78% audience score
My take with this movie is that the opposite of your #2 happened. Lots of regular folks who are concerned with global warming rated the movie highly, because they care about the political message it sends. Critics saw it as a pretty mediocre movie even though the message might be good.
I'm not saying you're wrong, but I do think it can go both ways.
EDIT: I looked again, and I realized the "Critic Consensus" and "Audience Says" sections on that page actually match my theory. For whatever that's worth.
Critic Consensus: Don't Look Up aims too high for its scattershot barbs to consistently land, but Adam McKay's star-studded satire hits its target of collective denial square on.
Audience Says: Although it can be heavy-handed with its messaging, Don't Look Up tackles important subjects with humor and heart.
Yep. There was a time when I trusted internet/social media reviews because they weren't bought and paid for access media. But now, it's clear that they are also bought and paid for. They are now invited to press screenings and follow disney rules on release of spoiler reviews. Not only influencers but entire social media platforms are now arms of corporate PR.
> 2) Everything has become political
It's always been political. The only difference now is that ordinary people have a way of voicing their opinions. Hollywood, media, film critics, best sellers list, award shows, etc are all political propaganda. Always have been.
I like reading books which teach me new experiences, or ways of life I've never really thought about.
These types of movies are great because they expose me to subcultures and issues which aren't a part of my daily life. Even if I'm aware of them statistically understanding the emotional component to these kinds of stories is a worthwhile experience.
As long as the movie isn't simply bad, I'd give it a good review as well.
https://www.rottentomatoes.com/m/silence_2017
To be a Christian in Japan during this part of the Edo Jidai is an especially marginal identity. It's not hard for a creative type who has seriously been on the margin at some point in their life to relate to that even if they are not Christian. It also doesn't hurt of course that the final message delivered is a pan-out view of his faith in its unconfessed nature which does indeed comport to a contemporary agenda summed up well in the movie's title; "Silence", as an imperative. "It is virtuous to make your sacrifices to Caesar/The Emprah, step on the fumie, and never speak about your religion in public because it causes other people pain and discomfort."
#1 The big budget crowd pleasing blockbusters. A few flop, but generally audiences like them and they don't do anything critics can complain about too much.
#2 Highly targeted low budget movies. There are small but consistent audiences with endless appetites for horror, action/thriller, etc films even at lower budgets. This seems to be where the biggest disconnect is, the genre fans highly rate their genre films even if it's direct to streaming fluff, while the critics see them as flawed, and broader audiences just don't watch them.
I may also be different from the usual audience, in that I try to avoid binging anything. Hour-long episodes are great for my WFH lunch break, and to pad some time in the evenings if I so desire, but I prefer to commit to watching a movie if I know I have the time.
Molly's Game was a nice and long movie, but at over 2 hours long, I feel like it could have had potential at being a miniseries. But also, maybe not? Who knows how much meaningful material was left on the chopping block.
Maniac is an excellent example. If it were a film, much of it would be cut down, and the movie likely would have had a singular focus on Jonah Hill's Owen character. The show, in having more time to breathe, realized relatively early on (by episode 3 or 4) that Emma Stone's Annie character had a far more compelling story to tell, and shifted its focus accordingly.
Getting a second bite is also responsible for the recycled pablum with China pandering--if you say something that the Chinese government deems unacceptable you lose your shot at that market.
As for "episodic", I suspect that's less a "deep dive" and more ADHD background watching on a phone. "Encanto" was an absolute poster child for this--"HEY! LOOK UP NOW! HEEEEEEEEY!" "Okay, volume back down. You can go back to your text messaging for a while." "HEY! LOOK UP AGAIN! TIME TO PAY ATTENTION! HEEEEEEY!" ad nauseam.
That’s just kinda how musicals are.
That said, biases arising from endogeneity might have negative effects too. You can't conclude a parameter should have a different/zero sign just because of endogeneity, you have to go fix your model, and re-estimate the parameters.
This is true when there is uncertainty. It also doesn't connect to "I only hire people who think like I do", which is the context of my response, so I don't follow your point.
Agreed on your other point that you need to instrument endogeneity. Another requirement is plausibility. Hookworm presence in Greenland should be uncorrelated to solar sunspots.
The standard film critic job in the 80s/90s was a local newspaper. That meant your audience was men/women of various ages and political beliefs who happened to live in the same area. If you were a successful critic you had to write something that appealed to all of them. This was actually a vital service that people used to determine if a movie was good.
Now the typical movie critic is online and has an audience which probably has a singular viewpoint. This isn't bad - but makes it unlikely the critics with "everyman" sensibilities would do well. People who consume an above average amount of film reviews will have their views represented more.
Chuck Klosterman (who was once a local newspaper movie reviewer and now writes to a certain type of audience online) has written about this.
I suppose you could detect this by correlating individual movie advertising budgets with rt audience scores across the board?
My problem is that if I ask a researcher about selection bias, they have no problem defining it for me and even giving me a textbook example of it. But when I present to a researcher a real life scenario that is highly susceptible to selection bias, they will almost all fail to identify it. It's a problem where many people are able to compartmentalize knowledge but fail to actually make use of that knowledge in practice.
As a run a quant firm I see this among so many researchers across many disciplines. They fully understand a concept when you ask them directly, but they fail to put those concepts to use outside of a very direct and artificial setting.
[1] https://en.wikipedia.org/wiki/Selection_bias
If you had said that you’re looking for candidates to raise it as a possible explanation, instead of seemingly firmly asserting that it is the one and only correct answer, you’d probably have less replies arguing with you here (or perhaps if you do have such research readily available, to have shared it).
What most level headed people will do is understand that my statement is about how you can not reliably compare a group of people who are predisposed to enjoy a movie to a group of people who review almost every single movie that's released, regardless of their predisposition towards it. People are welcome to argue that and honestly I'd expect nothing less, but it's a basic statistical principle that I expect every single researcher I hire to be well familiar with and if they're not, then I make the choice to not work with them.
>you’d probably have less replies arguing with you here...
There is also a selection bias among the people who comment on HN and they too are not representative of how most of the community feels towards a topic. The people who want to nitpick the specific wording of my statement are certainly entitled to do so and will likely find something to nitpick regardless of how I framed my statement, but it's not a particularly interesting discussion to follow up on.
The answer that requires the fewest assumptions and is consistent with almost all basic statistical analysis is that a group of people predisposed to like a movie will rate that movie higher than a group of people who watch movies regardless of their predisposition towards it.
That's not to say TV=bad, movies=good; just that IME respect your time way more and are easier to commit to most nights.
The act of sitting on a couch, watching something entertaining that thousands of people put work in to produce, is a "commitment". And in fact, the commitment is too much. Because for this same commitment, one could replace it with several smaller ones that entail the exact same thing, but with even less thinking.
If only there were a device that would just directly inject the memory of a movie into our brains, saves us from all this "effort".
That's a very good point that the role of the critic is to provide their own expertise and opinions on the overall subject. But the critic still has to be within the ballpark of popular tastes if the expectation is a mass audience that really trusts their recommendations, right?
Many Americans might listen to a food critic that said that he thought some pasta dish at some Italian restaurant was the best, even if pasta is not currently everybody's favorite food. But if he he recommended boiled snails or braised calf brains, many people might ignore his judgement because it's a bit too far out of range of what they're comfortable with. At a certain distance from popular opinion, a mass audience might no longer trust that critic. Just a thought.
Well, no, because I'm pretty sure most film critics aren't trying to directly provide a "yes or no" recommendation to a general movie-going audience. I've read plenty of film criticism that was entertaining and informative despite me having very little shared taste with the writer. And I suspect if you asked film critics to straight up predict what the RottenTomatoes audience score for a film would be, the aggregate of film critics' predictions would tend to be closer to the RT audience score than the RT critic score itself.
Not really. A professional critic usually writes a longish review describing the item and why they liked or disliked it, as well as the conclusion. If they are good at description, that's useful to me even if I don't share their likes and dislikes. If they hate things I love, then I can look for their negative reviews as places to start. If they love things I hate, their seal of approval is a good sign for me to keep walking. I can trust their recommendations even if I don't follow them, if I think they are fair and honest and have consistent opinions. If their opinions blow with the wind, I probably won't trust their opinions, even if they agree with mine.
I found Anthony Bourdain trustworthy, but I do not like a lot of the dishes that he really did (no thanks to organ meat and blood sausage), so I wouldn't blindly follow his suggestions.
When the person writing a review has "systematically different preferences" from the people they're writing for, "out of touch" is exactly the right description for that ("snobby" - not necessarily).
The whole point of pursuing the opinion of critics isn't averaging them to see what smart people like its in finding critics whose taste isn't wholly out of line with yours and using recommendations to cut through the crap and find things worth watching.
Sometimes I want to watch the Seventh Seal and sometimes I want to watch Die Hard - both movies have their place but a lot of critics tend to lean strongly into arthouse movies and sell them well above their actual appeal. I think one of the hardest parts of being a genuinely good critic is being able to leave your own tastes at the door when walking into a new performance - and those tastes are generally ones that have been honed by an education in classic works. Some movies use extremely bold cinematography choices to add a lot to their work (I'd point out Sin City and Schindler's List both of which made extremely good use of high contrast black and white (mostly) filming to add heaps to the story) but then you've got so-so art-house pieces that play into those familiar tropes without delivering anything of real value.
It takes a really good critic to watch a movie they really personally enjoyed and tell everyone that it's probably not for them - please do talk about what you liked to inform like minded people - but don't give a 5/5 just because it appealed to your specific tastes.
Then it's mostly Indian films which get ridiculous user ratings.
Examples in point (both exceptionally terrible, but with very fake high ratings):
* https://www.imdb.com/title/tt13668894 ("The Book of Boba Fett")
* https://www.imdb.com/title/tt9170108 ("Raised by Wolves")
“You might not have noticed, but your brain did”
Now there’s mauler, critical drinker, er, to name a few. Going beyond entertainment, long form analysis of politics and more. Go beyond the press release and the status quo.
I’m conflating a lot of things here purely because it would be funny if mr plinkett could be shown to have revolutionised the world in ways the “pretentious snob” could only dream.
Audiences want to be entertained.
Will Smith was a ridiculous, inauthentic disaster in King Richard, but won Best Actor anyway.
This is absolutely not the case. For example "Oscar campaigns" have significant influence on the Academy, and they happen after the critics have given their verdict.
the kind that confirm my existing ideology.
i don't know if you've noticed, but people who do things that i understand are totally normal, and people who are different from me have dumb ideas. it's annoying when they talk.
Not every moment has to be a teachable one.
Almost every sort of film, IME. I can hardly recall a film that 'lectured', if we mean the same thing, and it would be panned if it did.
For one thing, if the filmmaker can't convey things without lecturing, they are in completely the wrong job. Does a painter have to add call-out boxes with labels? Second, they almost never have a specific message, but depict the world - art holds a mirror to the world (very generally speaking).