Data sciencing IMDB ratings
Well, what does a data scientist do at his spare time? Watch movies, of course. But how a data scientist decides which movies to watch? By looking at the IMDB ratings, of course. But, how unbiased are IMDB ratings? Does a rating of 7.0 for an action movie from the 90th is the same as rating of 7.0 for a comedy just released? Well, there is only one way to check. Data Science. Happily for me, IMDB database can be downloaded from their site.
Let’s start from a basic data exploration. Overall there are 1,711,441 films in the database. However we drop all those with less than 1000 votes, those are just noise. Hence, we get to 107,189 films. Of those, we are interested in movies only, so now we have 48,960 movies for at least 1000 votes. Distribution by year is as follows
As one would expect, number of movies registered in IMDB each year grows consistently. The drop of 2020–2021 is clearly due to the COVID regulations, the drop in 2025–6 is most likely explained by the fact, that the database is updated with some delay and many of the new movies are not there yet. A bit more difficult to explain is the drop from 2022 to 2024. I’d guess, this is also due to the COVID. A movie takes time to make, and since fewer new productions were started at 2020/21, fewer new movies were released in 2023/24. If this is correct, we would expect the number to go up in the following years. Just wait and see.
Lets’ see now what is the distribution by genres. Each movie can have more than one.
Well, for Romance we have more than 8000 movies, for War is just about 1200. The producers clearly follow the rule “make love, not war”. But Drama they like event more.
Now to the interesting part. The rating. Do the average changes by year? By genre? By number of votes? For the last question the answer is clearly “Yes”. Number of votes and the rating have correlation of about 18% and, if we transform the number of votes to log scale, as we should for this type of feature, the correlation becomes even higher, about 23%. It makes sense, people like to watch good movies, therefore good movies have more views and more votes. Actually, I expected the correlation to be even higher, but it is what it is.
Get Boria Kapchits’s stories in your inbox
Join Medium for free to get updates from this writer.
What us the average rating per genre?
That’s a surprise. Seems that no one likes the Sci-Fi and Horror movies, while the Documentaries and Biographies are always blockbusters, one by one. It gets even more interesting when we check the average rate by year.
Can it be, that the more movies we have the worse they become? Well, probably in average, but this results is not very useful. It is also biased towards before-IMDB era movies. Clearly, the “bad” movies of the 90th did not make it to the IMDB rating, and therefore they do not drop the average. Moreover, the average for a year or a genre is very influenced by a long tail of movies that have just enough votes to make it above the 1000 threshold, but these movies are of no interest to us when we choose what to watch on the weekend.
So, let is make a different snapshot. We will take 100 best movies for each genre and each year, find the average rating and see what can we make of it.
OK, the average rating of the best movies is between 7.8 and 8.8, with standard deviation somewhere between 0.2 (for Documentary) to 0.35 (for Thriller).
By genre, the best Drama movies are rated higher than Romance, and Romance is higher than Fantasy and Horror. If I need to guess what the reason is, it seems that the genres most popular among the young audience are rated lower. May be that the young are more critical? Some researches claim the opposite. Anyway, the important point is that a Comedy with average score of 8.0 is 3 standard deviations below the top comedies average, while a Sci-Fi movies with an average score of 8.0 is 1/3 standard deviations above the average of the top Sci-Fi movies. It does not mean, of course that Dramas are objectively better than Animations. It is just that each genre has its own audience with different voting habits. Keep it in mind when choosing a movie of a specific genre and don’t be misguided by the rating.
What about the year? Actually, even before this research I had a feeling that in general the good old movies are rated lower than good modern ones, but now I can prove it.
Here, again, while the score of 8.0 is just an average good movie in 2018, it is high above the average for a movie from 2000. Well, I have no explanation for that. For one thing I’m sure anyway. The movies of the 90th were not worse than the modern. So, we just need to remember that comparing ratings over different years and decades.
Now, with all this knowledge, let’s pick a movie.
