128 comments

[ 8.7 ms ] story [ 132 ms ] thread
Very cool! I could have really used this a week ago, when the wife and kids were out of town and I had some free time to catch up on movies I've missed from the last 3.5 yrs.

What's the ranking criteria?

Also, being able to exclude based on categories (comedy, no animations, no family for example) would be a great feature.

Thanks.

The ranking criteria - basically, I combined ratings from IMDb and Rotten Tomatoes, with additional heuristic tweaks. The ranking method I use for is different than the popular IMDb method aka "True Bayesian estimate", which is not really a Bayesian estimate. I wrote more about how I approach ranking in my e-book http://arek-paterek.com/book/ , for people who are really interested in such technical details.

This was my first question too and I wouldn't necessarily consider myself incredibly technical.

I think the first question most people have when they see a "top _____" list, is "according to who?" Having a short description of the way the rankings are done along with a link to a more detailed description, would be great.

Nice job on shipping it!

This really doesn't seem to solve the main problem that most current ranking systems have - it doesn't compensate for recency. Newer critically-acclaimed movies like The Dark Knight rank higher than much older but just as well-liked movies like Citizen Kane.
Is that really a bug? Kane, while a technical masterpiece for it's time - shouldn't be up on some pedestal. As art it is, while powerful, also not flawless. If you had to pin me down to a black and white era drama, I much prefer, say, 12 Angry Men to Kane as _art_.
I'm not sure what this has to do with my point.
This is based on IMDB, right? Explains why the godawful "The Shawshank Redemption" and "Avengers" top the chart.
Honestly, this is the first time I've ever heard the words "godawful" and "Shawshank Redemption" in one sentence. We got ourselves a movie hipster, here.
Totally agree. Shawshank is one of the best movies I've ever seen.

Was the GP sarcastic or something, for using IMDB?

What exactly makes it so great? I found it rather pedestrian. An entertaining and competent cliché.
This is why we need curated lists like the AFI's top 100. I'm not saying they are the answer rather what I'm saying is truly reflective lists require critical thinking and evaluation.
Yes. I'm much more interested in the informed opinion of one person or a few experts than I am in the bland, diluted aggregate opinion of everyone.

The former at least has the potential to give me something other than safe, obvious choices.

Have you tried out Criticker [1]? Its recommendations are based on your previous scores and the scores of other members. The recommendations and the probable scores are very spot on.

[1] http://www.criticker.com/

Yes, mainly on IMDb. The Shawshank Redemption tops the IMDb, and The Avengers have a very high score 4.6/5 on Rotten Tomatoes.
I think you and I are in a pretty small minority. Maybe it's because I'd already read the novel - but I walked out of Shawshank Redemption thinking "meh" - and discovered I was very, very alone in that perspective.
neat idea, I would be interested in how the score is calculated.

I tried restricting it to TV series, and I noticed an interesting trend that anything older than 2000 is pretty far down the list.

This is probably just a factor of pre-Internet TV shows not being well represented by number of reviews.

I see the same bias on sites like Rotten Tomatoes where if you look at an excellent movie before 1995, it will have few reviews (unless there were a bunch of reviews collected for the DVD / BluRay releases)

I love the simplicity of the data! The alignment of the checkboxes/filters in the top left bothers me though. With everything else aligned so well, anyway you could do the same to those?
This is awesome! Only minor suggestion is to improve readability a little more. Font is way too small with too little space in between each entry.

Also, I got this error: http://cl.ly/image/2D2S1E020e0W. I think I clicked on Stanley Kubrick and then came back to the home page.

Thanks for the suggestions and for the bug report.
Who has the most movies in the top 5000? Does anybody have more than Marlon Brando's 19?
Counted 23 for Spielberg and 30 for Hitchcock.

Almost all of Kubrick's films are in the top 5000 except for one or two.

Hmm, I explicitly checked Kubrick; http://5000best.com/movies/Stanley_Kubrick shows just 12. But you're right Hitchcock is the one to beat with 33.
Yes, Kubrick only directed 13 movies in his life and 12 of them made it on the top 5000 list.
(comment deleted)
Well, he made only 11 real movies (Killer's Kiss doesn't count IMO), and all of them are listed. So, I think the parent was implying that he's probably the only director with 100% "hit ratio".
If you try to go down this list, your head will explode. Half of top movies are highly bogus comices (sorry guys, they are), the other half are highly epic dramas. You just should not mix them on a single page. This is so wrong.

As the whole idea of an universal list is.

Even if you view them by genre, it makes little sense. Like in sci-fi, Avengers is #1 (which I thought was a boring movie full of cliches), whereas the brilliant Watchmen is ranked #1780.

WTF?

And that's even assuming that Avengers should have been categorized as scifi.
Or Watchmen. If there was a separate category for superheroes, maybe....
I like this! I'm looking forward to the book section.
Can you add a column that shows if it's available on Netflix/Amazon/etc?

I'm always looking for a new way to find movies/series to watch and this list seems like a good one (I seem to agree with most of the ones I've seen near the top).

This. This would make the whole thing dramatically more actionable, efficient, and lovely.
The problem with these rankings (assuming they are from IMDB) is that they are more friendly to contemporary, Internet-age movies and TV. Anything older than about 30 years is significantly down further in the list.

As an example, I can't imagine a world where 'The Avengers' is better than 'Schindler's List', 'Godfather, Part 2', or 'Pulp Fiction' (just 3 examples).

A few stupid entries: #1721 'I Love Lucy' - easily considered by critics as the best TV comedy of all time yet barely in the top 20% of all tv and movies? I can't even find the MASH TV show, only the movie. Pretty weird considering that it's considered the 2nd best TV comedy of all time. #74 'Seinfeld' - considered the 3rd best TV comedy of all time. The internet is more friendly to this one. #15 'Game of Thrones (TV)' - fanboys strike again, or they just really like irrelevant nudity[1].

#1636 'The Constant Gardener' - it is an abomination that this is here, it's lower ranked than 'Super Troopers' (#1626), 'Iron Man 2' (#1610), and 'Soul Surfer' (#1599), just to name a few.

[1] http://gawker.com/5902076/snl-explains-the-nudity-in-game-of...

That's the mass taste. I am not trying to discuss with the mainstream taste, just made the tool. For those interested in getting a personalized movie ranking I created another tool "Movie Galaxy" http://arek-paterek.com/movie-galaxy/
> That's the mass taste.

I seriously doubt that. Just because it's what the available data show doesn't mean that that's the popular taste. If you surveyed 100 people who have seen both The Avengers and Godfather Part 2, I highly doubt a majority will regard the former as a better film than the latter. People are in different moods when they rate films, older films don't get rated as often, the scale with which an individual rates different films may be wildly non-normalized...

Website idea[0]: show two random movies, ask them if they've seen both, and if they have, ask the user which one's better without prescribing what "better" means. Repeat 1 trillion times. I'd be much more willing to trust differential data like this.

[0] - not going to call it a startup idea, because I have no idea how it would make money.

>"I seriously doubt that."

I think you'd be surprised. Though, I also think you'd be hard-pressed to find people who have watched both films (I've only seen one, and it isn't The Avengers). And there's a reason for that: most people don't want to watch a three-hour, epic drama from 1972. Ask people who haven't seen either film which they'd rather see, tonight, and I'll bet they choose The Avengers. Why? Because if you haven't seen GF2 by now, it probably doesn't interest you. So on some level, these rankings speak to the current Zeitgeist.

I can't understand the vitriol directed at these rankings. Sure, there is a definite bias towards modern films in the imdb rankings, but it is what it is. I don't think that it's any different than some crotchety old movie critic telling me which films I should like best.

The fact of the matter is, there is no objective measure of "best". Scan these comments. I can find examples to agree with and disagree with in nearly every single one. But they are all saying the same thing: "This list is wrong!" Based on that, I don't think you or I would have any more success curating a list of movies. This list is as good as any.

Anyway, great tool.

> Website idea: show two random movies, ask them if they've seen both, and if they have, ask the user which one's better without prescribing what "better" means. Repeat 1 trillion times.

http://flickchart.com

That's really neat. It's quite addictive to rank films that way. Annoyingly hard in some instances. The only thing I think is lacking is some way to get a reminder about the plot; when you're talking about movies seen more than a couple of decades ago ...

A "would like to see" button would be useful too.

Would the two movies be from the same genre?

For example, I would have a difficult time deciding which of the two - "The Secret in Their Eyes" or "Gladiator" - is better.

Having played with the site for the past 20 minutes now, the by far most difficult thing is choosing between two horrible movies. I really wish there was a "they're both equally horrible" button.
Out of curiosity, how'd you do feature selection for the clustering (on a high level) for Movie Galaxy?
The clustering was based on the regularized SVD / matrix factorization results. Details in my e-book.
And interestingly, "Research suggests that the best way to predict how much we will enjoy an experience is to see how much someone else enjoyed it." - “If money doesn’t make you happy, then you probably aren’t spending it right” by Elizabeth W. Dunn, Daniel T. Gilbert
Maybe put in an inflation adjustment factor. That would reflect how a movie rates compared to it's contemporaries.

You'd have to be careful not to cap a movies' rating just because scores in that era are more lenient though -- maybe pull movies toward an overall median based on their distance from a 2-3 year moving average.

So say we want the median score to be 50%. And the median rating in 2008 was 63%. Let's take two movies from 2008 -- one scored 98%, the highest score in several years, and one scored 63%. The 98% movie stays at 98%. The 63% movie score becomes 50%.

The tool has no point if the data source is flawed. It is like saying you made a weather forecast website but none of the predictions are accurate: It becomes worthless. If you make a tool that tool has to be able to extract quality data before anything else.
Yeah, avengers and dark night better than fight club? C'mon.
Firefly better than Citizen Kane?
Just think of it as each value being plus or minus 50. I'm sure the idea isn't to determine the best film of all time, it's to expose you to excellent movies and series you may never have seen.
I think you could at least make an argument that Firefly is comparable to Citizen Kane. It makes a lot more sense than saying that The Avengers is the third best movie of all time.
> I think you could at least make an argument that Firefly is comparable to Citizen Kane.

Er, how, exactly?

Citizen Kane is famous for a bunch of different reasons, including advancing the state of the art of cinema and being basically a flawless masterpiece. But probably the thing that stands out the most today is its psychological depth, in that to this day it's arguably still the best character study ever done.

Firefly doesn't have that sort of psychological depth, but it does have a lot of moral ambiguity, something that's virtually non-existant in CK. That's something that would have gone a lot deeper had the show gotten another two or three seasons, but they never got that chance. It also has some of the most clever writing of any TV show, up there with only Arrested Development really. Cleverness might not be as prized by critics as authentic character writing, but it's still at the top of its form nonetheless. The main downside of the show was that it was already starting to get a bit formulaic toward the end of season one when they canceled it. In any event, I don't think you could ever put Firefly above Citizen Kane, but certainly I think it pushed the boundaries of its genre in a way that is rare.

The Avengers, on the other hand, has virtually no redeeming value whatsoever except for Joss Whedon's clever writing during the second half and the amazing effects during the NYC fight scene.

Moral ambiguity is something that passes in and out of fashion, and it's something that Firefly never really delivered on. Where Firefly really shone was as a prototype for the naturalistic science fiction approach that Battlestar Galactica mastered.
Yeah, it's unfortunate that Firefly never got the chance to deliver there. That said, if you pit the first season of Firefly against just the first quarter of Citizen Kane, which is really more fair in some ways, I'd go with Firefly any day.
I don't think "the chance" is what it lacked. Firefly was relentlessly light and couldn't have sustained the darkness necessary for real moral ambiguity to work.
I'm not one of those "Firefly is t3h greatest show evar" people but I would still say that, yes, Firefly is better than Citizen Kane. But we would have to define what we mean by "better".

Citizen Kane gets a lot of rightful attention for being historic and groundbreaking in a number of ways, especially by people who have never seen it. Unfortunately, despite often being conflated, "groundbreaking" and "good" are not really the same thing. Now, don't get me wrong, Citizen Kane is a fine film and still worth watching today. But divorce it from its historic role and consider modern audiences. Think about what it would be like if released today. At best it would be critically-acclaimed and lauded by a small audience without ever seeing mainstream appeal, much as Firefly has.

Now, granted, I haven't gone a single shred toward showing that Firefly is actually any better than Citizen Kane though I have gone some way to putting them on roughly the same level. And Firefly has more explosions and space zombies making it more watchable for many people today.

Is Firefly really better than Citizen Kane? It will never be widely thought so by critics but I can definitely see an argument where it is better for modern audiences and I really think that's what matters at the end of the day.

You'll be much happier if you consider every top X list to be a list of stuff worth checking out rather than an absolute valuation of what is 'better'.
The problem with that is there is so much good stuff buried way too far down the list. Outside of the top 5% I could find a fair amount of movies and TV that I know would appeal to many individuals.
That's just it though while Doctor Strange Love (1964) was #32 and Up (2009) was #924 there both vary good movies. There was plenty of recent and fun movies vary low on the list, consider:

Bowfinger (1999), Solaris (2002), Honey, I Shrunk the Kids (1989), From the Village to the City (1974), Going Postal (2010), Legally Blonde (2001) that's that's 4913 to 4918.

PS: Granted, rating movies is a huge multi dimensional problem, just ask Netflix, but you don't need a lot of accuracy to find a good movie.

I'm generally a seller of the whole "social discovery" web application movement, but for things like movies, I have one or two friends whom I depend upon heavily for "social discovery" (I'll usually email them about what I'm looking for) in having great movies recommended to me personally :).
I very much agree with that approach. A curated monolithic ranking is hard to build up and maintain. Personal recommendations are truly hard to beat.
There is no enough time in life to check all the crap that made it to the top of that list. This seriously a problem with this tool, the data is so flawed that you cant expect anything good from it.
See also: Doctor Who (1963) (#2272)
Am I the only geek in the world who wasn't THAT impressed with Avengers? It was a fine superhero film, great ensemble cast, good script and all, but I didn't find it revolutionary. #3 "best" film ever? Come on!
I agree. I don't know how it has climbed up that far on the list!

Perhaps the film companies have finally found a way to alter the minds of movie-goers.

I do not even think it rates as fine! The script was awful, and Loki being beaten up by Hulk was just so wrong. Loki is joking about it but he ia supposed to be a God and way above the Hulk and so on. I could go on and on and nitpick forever on this movie, there were so many thins wrong with it, but the worst is that there was no tension, no sense of urgency, and no reason for the super heroes to team up together. The super heroes should have been going one by one after Loki and their ass kicked so that they realize they need to work together.

Anyway, Marvel really messed up what could have been awesome in the first place.

I thought Avengers was a joke. Wouldn't ever watch it again.
Avengers - http://www.imdb.com/title/tt0848228/

IMDB rating is 8.5 only for Avengers. That will place it outside the top 30 here.

If you look at the reviews at IMDB you will see something strange. Most of the reviews are either 1 or 10 stars. I feel like some kind of gaming of the IMDB ratings is happening.
That's not gaming. If a movie is rated 4 stars on IMDB but you think it should be 5, you won't rate it 5, you'll rate it 10 because that pulls the average closer to 5. Likewise, if it's rated 5 stars but you think it should be 4, you might rate it 1.
(comment deleted)
Case in point The Matrix at 13th. Kenneth Branagh's Hamlet: 1052.
I see an opportunity here. Is there already a recommendation service around that takes your rating profile into account? Apple has their 'Genius' system, however with respect to movies rentals I think this is lacking a bit: Movies I want to rent are usually movies I only want to see once, not gems like Godfather or Constant Gardener. A personal rating service would have to pose you targeted questions about your movie preferences and drill down into your personal taste.
Once you control for film age, the ratings makes a lot more sense. Films get worse ratings as they get older, so newer films naturally have better ratings.
There's another bias, where things are overrated because they're unusually good for their time. For instance, "Battleship Potemkin" is often considered one of the greatest films ever made because it was so innovative for its time, despite the fact that when you watch it now, it's easily a shitty propaganda film. It may have been a greater achievement, but as a film, it's somewhat lacking by modern standards. Likewise, MASH and Seinfeld were groundbreaking, but largely because of MASH and Seinfeld, we can and do make better television series today. I think the two biases largely cancel out.

This has some interesting consequences. For instance, in the restricted realm of science fiction TV, it's clear that Star Trek: The Next Generation is strictly better than the original Star Trek, just as the recent Battlestar Galactica is strictly better than TNG, because these series responded to and innovated on each other.

On balance, I still wouldn't say it's necessarily true that newer series are better than older series. It's still rare that a TV series tops The Prisoner, for instance.

" I can't even find the MASH TV show, only the movie. Pretty weird considering that it's considered the 2nd best TV comedy of all time. #74 'Seinfeld' - considered the 3rd best TV comedy of all time."

Are you really criticizing the subjectivity of this list based on the subjective rankings of your generation?

Also game of thrones is brilliant, imo.

I'm saying you are clearly seeing a generational and tech savvy bias to the rankings. For Avengers to be so high shows it's good movie, and it is. But the fanboyism of the comic-reading, tech savvy audience is skewing the metric way higher than it should be.

The same thing is happening with Game of Thrones. It's excellent but I've seen many, many TV dramas that are just as good if not better.

Comedy is a terrible example to criticize due to it tending to be very cross-generational. MASH, Seinfeld, and I Love Lucy are still shown on TV today. MASH and I Love Lucy on more obscure cable channels, but Seinfeld still is on in reruns for an hour before primetime TV on our CBS affiliate.

Very good implementation.

But please change the title to "500 best movies according to IMDB & RottenTomatoes".

I mean, look here: http://5000best.com/movies/Stanley_Kubrick

Clockwork Orange is 46th, 2001 is 211st, and Lolita (Kubrick's version), 1018th. Best movie that poor, not-so-much-talented Kubrick has apparently made is Strangelove, which is 32nd "best movie ever made".

It's very informative to know what IMDB thinks (for one thing, you know what movies to skip), but basing our "best movies" list on what they think is not all right.

I wonder what a better metric would be to account for this contemporary bias?
At first I thought this was an awful list, but then I filtered it by date up to 1995 and I felt that the list made a lot more sense. Is this an issue that it takes 15 to 20 years to get settle on what is truly worthy? Or is it an issue with the dawning of the Internet Age?

There are plenty of films made after 1995 that I think are highly deserving. But in 20 years from now, will we look and say Return of the King out ranks Star Wars? Or The Avengers is better than 12 Angry Men? (The Avengers was decent popcorn fun, but what was the plot?)

Good point, a 10+ year cooling off period would definitely help.
Which is how the National Film Registry here in the States operates https://en.wikipedia.org/wiki/National_Film_Registry

"The National Film Registry is the United States National Film Preservation Board's selection of films for preservation in the Library of Congress."

"The National Film Registry names to its list up to 25 'culturally, historically, or aesthetically significant films' each year, showcasing the range and diversity of American film heritage to increase awareness for its preservation.... To be eligible for inclusion, a film must be at least ten years old."

"At first I thought this was an awful list, but then I filtered it by date up to 1995 and I felt that the list made a lot more sense. Is this an issue that it takes 15 to 20 years to get settle on what is truly worthy? Or is it an issue with the dawning of the Internet Age?"

I remember seeing the same thing on IMDB years back, where contemporary movies will appear very high on the list and disappear.

If you read the reviews (and I think IMDB lets you filter by demographics, or used to) you'll see that it's mostly young kids and idiots voting summer blockbusters as "10/10 OMG best movie evar!".

Not sure if there is a solution as it's always been the same way, there's always some hot new art, book, technology, etc. that's "GROUNDBREAKING" "CLASSIC" "AMAZING" that fades away quickly and the items that stick around for posterity aren't necessarily the ones that you'd predict (eg. Moby Dick and The Great Gatsby were initially flops on publication).

The scoring system needs to be bell-curve weighted by the release date of the movie.
The HN article title says "5000 best movies" instead of "Fairly decent jumble of 5000 best movies with many of the best TV shows thrown in".

The HN title and site title are both mildly misleading.

I understand that your sources (RT & IMDB) may have these mixed and it will take some time figure out what makes sense.

So which taste the best? I started in the Netflix Prize, and for me a movie taste is a point in a 32-dimensional space or 64-dimensional space. Which point is better than the others?
Best movies(and TV shows) by decade (top 100 in parens):

  < 1910:       4 (0)
  1911-1920:    7 (0)
  1921-1930:   48 (0)
  1931-1940:  123 (4)
  1941-1950:  168 (8)
  1951-1960:  243 (10)
  1961-1970:  297 (5)
  1971-1980:  362 (13)
  1981-1990:  583 (7)
  1991-2000: 1030 (21)
  2001-2010: 1937 (27)
  2011-Now:   198 (5)
Maybe it's interesting technically, but what's the use of this site, if the movies are rated by retards? I mean, any ranking that puts Avengers in the first 1000 best movies is obviously deeply flawed.
The Avengers has the highest average rating on Rotten Tomatoes, and I can understand it. This movie has a very good script in my opinion, but de gustibus...
Oh come on. It's a fine piece of entertainment, but it's hardly the second-best film of all time. Using the raw data like that is an immediate statistical fail, you should be using a high-pass filter of some sort.
The Avengers has a 92% on RT. The Godfather, Part 2 has a 98%. The Avengers leads The Godfather 2 on this list.
How did you get this data? Did you screenscrape IMDB and Rotten Tomatoes?

If so, is there a legal issue with reusing this data? I actually don't know what the answer is, I'm curious if there is or not.

Well, Rotten Tomatoes offers an API, and IMDB offers all their data for download in text files.
Not sure about RT, but IMDB has some fairly strict rules about how their data can be used. http://www.imdb.com/help/show_article?conditions

IANAL but from my reading it sounds like non-commercial use is ok.

I'm also not a lawyer, but I think you'd have a very hard time litigating for user-contributed content that has no creative value and is publicly available for download.
Wondering why 5000? It looks like the scores are based on "IMDb and Rotten Tomatoes." Which means some opinionated data by the "crowds".

Regardless IMHO gladiator ranked at #57 is my favorite.

Sorry this is off-topic but does anyone know why there are no independent Netflix services? I'm thinking back when you could go to your neighborhood movie rental place...well, now imagine that selection is specific (ex. Western) or curated, and also imagine it is online, available for viewing. Is this just too hard to get the rights to? How did local movie rental places get the rights to rent out films?

People sell pirated films on the street here in Brazil and I once heard of a cultured 'pirate' who only sold pirated, high-brow DVDs. It got me thinking

Licensing terms are really hard and expensive to get, streaming services are expensive to set up/run and Internet brands are a "winner takes all" situation that make it hard for regional or niche brands to compete.

You can buy a physical media copy of a film licensed for rental quite easily through a standard-ish system (expensive, I think it's a few hundred bucks for the disc?) but there's no standard system for licensing streaming and as long as the studios are scared of the Internet there won't be one.

There's a few indie streaming services out there but they're small. Film critic Roger Ebert runs "The Ebert Club", an online subscription club with limited runs of fairly obscure older movies. I remember one for horror/arthouse stuff too, but can't find it now. LoveFilm is really big in the UK.

Also check out mubi.com
Most of the movies here are crap made for people who don't have any sense of art. If you think the ones with highest scores are best, you don't even know what a good movie is.

Check out this: http://www.youtube.com/watch?v=NFfTot3qYMc

This is what a good movie is. It reflects the life as it is. It's not for entertainment. It's for art and people looking for good movie.

Well, I don't know if I agree with some statements here... Art does not necessarily "reflect the life as it is". I think art is an artist's expression of his/her view of the world (or life). Some may see the same scene in bright colours, some in gray, if you see what I mean.

EDIT: However I do agree that majority of the "good" movies these days are just targeting the entertainment nerve of a potential consumer...

Giving more importance to the average rating in RT, rather than the tomatometer, would give a very different ranking. Avengers has more than a 10% lower average ranking than any other movie in the top 10.
i know thumbnails take up lots of space, but I don't remember titles.. i remember movie covers. Adding thumbnails would help immensely, it's an extra click away.. but still an extra click
Could you add in a filter for "not animation"?
(comment deleted)