A Breakdown of IGN’s Game Rating Distribution

If you’d like to take a look at the data referenced in this post and the script I wrote to gather them, feel free to head over to my Github.

I’ll admit it: I take stock in video game reviews. I don’t read them religiously, nor am I insulted when a site rates a favorite of mine a bit too low. I read reviews to find out which games I might like. With so many games and so little time, reviews and ratings are, for me, a much-needed filtering process.

But video game ratings come with problems. On a 10 point scale, what’s the difference between a 7.3 and a 7.5? Even a 7 and an 8? And why does it seem like reviewers hardly dish out scores from the bottom half of their scales? How can reviewers package their opinions, highly subjective and finicky things that they are, into definitive scores? Numbers feel a lot more objective than words. It might be this perceived objectivity that makes people protest the seemingly arbitrary scores that reviewers select. (I wouldn’t call these ratings arbitrary — they’re backed by opinion. Subjective? Yes.)

Present-day game rating aggregators add to these problems. Averaging across multiple sites, each with their own distributions, makes things messy. A score of 7 from one site very likely might be equivalent to a 5 from another. And what about those troublesome letter-grade ratings? Metacritic converts Cs to 50s — but last time I checked, a C mapped to a 70 percent or so in school. Metacritic claims that they generate their aggregate scores after running their data through a weighted, proprietary algorithm, and I’m sure they do. But I question their algorithm’s efficacy. Pick a few games on their website and do an old-fashioned average of the scores they list. The score you come out with will likely closely resemble their aggregate score.

But I digress.

I decided to do some investigation into the nature of game ratings. I’m currently in the process of building a dataset of game ratings from different sites. I started with IGN. After a bit of web scraping, I collected review data on the 75,005 games that IGN kept track of as of July 13, 2013. Of those, only 17,027 had ratings in IGN’s index (others were not rated, marked as “NR”). I coded a python script to collect the data and used BeautifulSoup for HTML parsing. I analyzed the data with an evaluation copy of Wizard, a suite of statistical tools.

Without further ado, here are some summary statistics of IGN’s rating distribution, peppered with some tidbits of information that I found interesting.

Summary statistics
Data retrieval date: July 13, 2013
Population size: 17,027 rated games

A distribution of IGN's 17,027 rated games. The distribution has a mean of 6.877 with a standard deviation of 1.762, and a median of 7.2 with an IQR of 2.1. All but one of the 51 0s are data collection errors that I didn't prune out.

A distribution of IGN’s 17,027 rated games. The distribution has a mean of 6.877 with a standard deviation of 1.762, and a median of 7.2 with an IQR of 2.1. All but one of the 51 0s are data collection errors that I didn’t prune out.

Mean score: Approximately 6.9
Standard deviation: Approximately 1.7
Skew: left

Interquartile range: 2.1
Lower quartile: 6.0
Median score: 7.2
Upper quartile 8.1

IGN’s index erroneously assigns a number of games 0s in their index (at least one game, though, actually did manage to earn a zero). I didn’t manually prune these entries from my data, so the above summary statistics are slightly off.

The highs and the lows
Only 314 games, or about two percent of rated games, received a score of 9.5 or above. IGN decorated 38 games with 10s. Classic games (the Zelda, Mario, and Pokemon franchises) dominated here. Some newer games did manage to break into the Hall of 10s, notably Naughty Dog’s third Uncharted entry and their recent The Last of Us. Rockstar’s GTA games and Red Dead DLC also carved out spots for themselves, as did a couple of Kojima’s Metal Gear Solid games. Some 10s you might not have heard of include Checkered Flag, Joust, and Shanghai (all Atari Lynx games), and Tornado Mania, a mobile game. Infinity Blade II was the sole representer of the iPhone.

2,913 games managed to be bad enough to nab a score of 5 or less, accounting for about 17 percent of total games. With a zero, Olympic Hockey Nagano ’98 boasts the lowest score of all games. Looney Tunes: Back in Action: Zany Race followed closely with a 0.5. Two other games, Extreme PaintBrawl and Action Girlz Racing, joined the less-than-one ranks.

Of the major home entertainment platforms (PC, Xbox 360, PS3, and the WiiU), the Wii had the lowest median score.

Of the major home entertainment platforms (PC, Xbox 360, PS3, and the WiiU), the Wii had the lowest median score.

Of the major modern consoles, the Nintendo Wii had the lowest median rating (a 6.8). The Wii U, Xbox 360, PS3, and PC have median scores of 7.5.

A closing remark
With a median of 7.2 and an IQR of 2.1 (i.e., 50 percent of scores lie between 6.0 and 8.1), it does look like IGN awards higher scores more often than not. This does not mean that they’re doing anything illicit (this might seem obvious, but you’d be surprised at some of the shoddy “journalism” out there that sensationally misinterprets data). Perhaps IGN thinks that most games just aren’t that bad.

I’ve got a hunch that other review sites’ distributions won’t match IGN’s, just as they likely won’t match each other. So I’ll gather more data — maybe I’ll be able to do something with them.

Update, July 23, 2013: Previously, this article linked to a website that I believed had misinterpreted ratings data. This post no longer links to that site.

Update, July 28, 2013: This post now links to my Github repository containing the source code and data referenced here.

debugmusic: Zelda Reorchestrated

While I may not have time to fulfill my nostalgic urges to revisit Hyrule by playing Skyward Sword, I can at least listen to old Zelda tunes at www.zreomusic.com. Zreo Music offers on-demand streaming of faithfully recreated and subtly mixed MIDI Zelda scores.

Between scholastic obligations and extracurricular activities, it’s been difficult for me to find the time to play video games these days. And even if my Winter Break has allowed for intermittent bursts of Skyrim and Arkham City, I still haven’t gotten the chance to plop down on my couch for a good hour or five and really immerse myself in The Legend of Zelda: Skyward Sword. The urge to do so became particularly compelling last night. During the two hours I spent at the Cupertino Coffee Society working on journalism, my ears were treated to throwback Zelda classics, including the Hyrule score and “Saria’s Theme”, in addition to Mario 64 tunes like “Inside the Castle Walls” (the owners of the café must be Nintendo fans!).

But I can’t rely on Coffee Society every time a wave of Hyurlean nostalgia washes over me; for that, there’s Zelda Reochestrated, an ambitious recreation and archive of Zelda musical themes that can be streamed on demand. The name, unfortunately, is a bit misleading: The tracks are MIDI’s, although the team has been working on a fully reorchestrated Twilight symphony, for what seems like (or perhaps really is) ages. Regardless, the scores do sound impressive and conjure, at least for me, childhood memories of braving the oceans in Wind Waker, or swimming through the depths of Lake Hylia. And so, while I may not have the time to play Skyward Sword and create new memories, reliving old ones through music seems like the next best thing.

debugmusic: Dire Dire Docks, a Mario Classic

When it comes to debugging your mind, listening to music is perhaps one of the most effective debuggers. The right music can submerge your mind in a cool, refreshing lyric pool and help you better take stock of the world around you. As such, I’ve decided to package my favorite music into blog posts through debugmusic, a soon-to-be recurring feature that highlights exemplary musical pieces from movies, video games, and really anything else.

In order to inaugurate debugmusic, I’d like to share a classic piece from my childhood: “Dire Dire Docks,” from Super Mario 64. As soon as that magical music starts playing, I’m transported back to my couch in Redwood City—I can see the 3-year-old me directing Mario through the watery depths of Jolly Roger Bay, only to discover that frightening, giant eel. Whether you’re a Mario fan or not, it’s hard not to appreciate the Dire Dire Docks theme. Take a listen:

read more »

Treyarch Fulfills its Duty with ‘Black Ops’

"Black Ops", although flawed, refines the "Call of Duty" formula

What started as a cherished gem of PC gaming has now exploded into an international phenomenon—Activision’s “Call of Duty” is easily the biggest title in modern gaming. With the release of each installment, fans’ expectations of the franchise increased almost exponentially.

Each title in the series is alternately created by game companies Infinity Ward and Treyarch. In the past, fans have not been too receptive of Treyarch’s attempts at crafting CoD games. Nonetheless, gamers’ enthusiasm for “Call of Duty: Black Ops,” developed by Activision-owned Treyarch, has mounted to unprecedented levels.

With its engrossing campaign, balanced multi-player, and attention to detail, Treyarch’s “Black Ops” is the best “Call of Duty” game on the market. read more »

NBA 2K11: A Legendary Game for a Legendary Athlete

Jordan makes a triumphant return in NBA 2K11, the best basketball simulator on the market.

Michael Jordan stands in a hallway, preparing himself for Game 1 of the 1991 NBA Finals. Energizing music booms in the background; the audience waits, restless with anticipation. Jordan turns his head and faces the camera.

“Are you ready?” he says.

Jordan jogs onto the court. The crowd roars thunderously as the commentator’s sonorous voice echoes over the speakers, and an electrifying energy pervades the entire stadium. With a seamless transition, the game begins and you’re playing as Jordan and the Bulls. read more »

My Tower of Video Games is Taller than I am

Just in case any of you were wondering about my gaming qualifications. The tower measured 7'4'' high. Note: my stack did not include my N64 nor GameBoy games due to safety issues.

My gaming hobby towers above nearly everything else in my life—quite literally. Last Friday, I assembled a 7’4” stack of my too-many-to-count video games. In order to possess such a large collection of video games, I had to start collecting them at an early age. read more »

Halo Reaches New Heights

These Spartans will accompany you on Reach's captivating campaign. Photo taken from Bungie.

It all started in 2001, when Bungie Studios’ “Halo: Combat Evolved” revolutionized the console first-person shooter market and created an enormous virtual community dedicated to all things Halo. But Bungie wasn’t content to stop there—they went on to release three more blockbuster games within the series. “Halo: Reach” is Bungie’s final Halo title; clearly, they wanted to go out with a bang. read more »

Half Xbox 360, Half PC: The God of Gaming Computers has Arrived

Origin PC has combined the best of both worlds with its Big O gaming rig, which is part liquid-cooled Xbox 360 and part PC, and costs up to $16,999.

Origin PC has defiantly rejected the notion that PC gaming is dead by releasing its monstrosity of a computer, the Big O.

The starting model of this saliva-inducing machine includes dual Nvida GTX480 graphics cards (liquid-cooled), 6GB of Corsair memory, dual 50GB SSDs, a 1500 watt power supply, a Rampage III Extreme motherboard (liquid-cooled), an i7 930 processor overclocked to 4.o GHz (liqui-cooled), and an Xbox 360 (liquid-cooled, of course).

Don’t get too excited just yet — this Big O computer will cost you $7,699. And that’s the cheapest model. A fully decked-out Big O costs nearly $17,000. read more »

Violent Video Games Head to Supreme Court

California argues that violent video games detrimentally affect minors.

Violent video games have often served as a scapegoat for criminal behavior in minors; therefore, California argues that minors ought not be able to buy such games. The validity of this reasoning will soon be decided by the ultimate authority: the Supreme Court.

On November 2, the Supreme Court will hear Schwarzenegger v. Entertainment Merchants, which will review a California law that would render it illegal to sell violent video games to minors. California’s law has already been struck down twice by courts, on the grounds that it violates the First Amendment.

Nonetheless, California remains persistent in limiting the sale of “violent video games.” The law states that a violent video game is one that depicts, “killing, maiming [or] dismemberment,…appeals to a deviant or morbid interest in minors,” and is “patently offensive…to the community.” California states that such games encourage violent behavior in children.

The proposed Californian law includes a host of messy complications and should not be passed.

read more »

Taliban Invade Video Game

Taliban in Medal of Honor. Click to enlarge.

Continuing with their trend of injecting controversy into the video game world, EA will allow players to assume the role of the Taliban in their upcoming game Medal of Honor, which is primarily set in Afghanistan. The contention revolves around the game’s multiplayer component, which pits US troops against the Taliban

As is the norm in the video game industry, people have begun spewing vitriolic venom towards the game’s inclusion of the Taliban. Karen Meredith, a mother who lost her son in Iraq, censured the game on Fox News. 

“A video game based on a current war…[doesn’t make] any sense at all. It’s disrespectful,” Meredith said. read more »