Metacritic: The destroyer of good games

Over the last couple of years, Metacritic has risen from an obscure measure of all things entertainment related into the industry benchmarking tool. Although it measures the metacritic score of pretty much all forms of media, I'm going to talk about something close to my own heart; Video games.

We just completely stripped out your article and took
your completely arbitrary score at the end. 

Once upon a time, the only measure that gamers had to know if games were any good were video gaming review magazines. We would buy the latest issue of Hyper, PC Powerplay or equivalent, and skip to the reviews section to try determine which games deserved our attention.

What we were looking at was a well-written article about the games mechanics, graphics, sound and story. We had a full explanation of the journalist's game experience, complete with highlights and pitfalls. After all of the different parts were read, we were given an arbitrary score. Some would be out of 5 or 10, others used a scale of 100 or a more obscure measure to give us an idea of what they personally felt about the game.

The thing with these articles however, was that while they were biased like any review conducted by any human, they were openly biased. Usually, you'd find yourself aligning with the tastes of an individual reviewer because over the course of time you'd noticed that their picks were very similar to your own interests.

You think this guy is going to able to provide objective opinions?

Gone are the days of game magazines due to the rise of the internet; A channel of freely available information. A whole mass of gaming review websites sprang up overnight. As this happened, a few of them rose above the pack due to their intimate ties with the industry itself; Gametrailers, IGN, Gamespot etc. People started gravitating toward these "authorities" on gaming reviews.

This shift was not a great deal different than gaming magazines to begin with. As consumers realised they had access to a whole bunch of review information without having to lay down their hard-earned dollars however, they began to compare and make aggregate scores for games using these main gaming websites as their source for "good" review content.

This changed everything.

They dismissed the article content itself, which is so important to understand both the intentions and context of the reviewers words, and instead only grabbed what they thought was the important take-away of the article: The end score. This would later become the be-all and end-all of a games worth. Gamers across the globe are standing in Gamestop or EB Games, deciding what game should earn the right to their purchase based entirely on their Metacritic score. This sucks, because so many great games fly under the radar, whilst powerhouse publishers backing billion dollar franchises steal all of the sales with their aggressive marketing and close ties to the important game review websites.

Metacritic saw this as an opportunity. All of these websites were churning out all of these articles with scores attached to them; How about compiling all of the scores to determine an average figure for the sentiment of the game? Seems good in theory, but it's not as straight forward as you think. The voting isn't equitable. They weight the scores based on how "quality" the reviewer is.

How is the quality of a game review determined? By their "quality and overall stature" (excerpt from Metacritic). Read: Popularity.

Any review coming out of IGN is going to have a great deal more relevance than a review from an actual gamer, say, I don't know... kungfucolin for example.

A reviewer with a GamerScore under 20,000 at this stage of the race is straight-up blasphemy. How could they possibly offer an expert opinion of a video game if they don't play the damn things?! Sure, anybody is entitled to do a review, but again: No one person should have a louder voice than another... But, if that's the way it's gotta work at least place more importance with somebody that has more gaming experience than simply how "prestigious" the company they happen to work for is.

How would you propose we fix this problem, CloakerJosh?

I'm bloody glad you asked, internet! Let me explain what I believe would be a better system. I don't pretend to have all of the answers, but hear me out.

One solution, is of course to completely remove review scores entirely, but not only is this impractical it simply won't stick. Metacritic already assigns scores to reviews that don't have one in summary. That's right, even if the reviewer themselves doesn't believe in scoring a game, Metacritic doesn't care. They'll do it anyway.

So, a scoring system is what is going to exist, regardless. But, we could change the way one is calculated.

Are you familiar with NPS?

NPS, or Net Promoter Score, is a measure that's being used globally by corporations around the world to measure the sentiment and engagement of their customer base. These measures are never perfect, of course, but they take a different angle than most scoring systems.

It works like this: A consumer, not a reviewer, could be posed this question:

On a scale from 0-10, how likely are you to recommend playing Game XYZ to friends or family?What this is doing is asking the consumer to think both analytically and emotionally.

They're analysing the game overall, and deciding whether or not the game is worth playing. It won't be judged solely on things like music, graphics, presentation, polish etc. but rather will simply boil it down to be as simple as can be, "Is this game worth playing?"

Then what happens is that the individual will engage their emotional centre by weighing up whether they would put their reputation on the line by recommending it to a friend.

Now, the way that the measure is resolved is like this:

• 9-10: Any one who answers 9 or 10 is considered a Promoter. They will actively tell their friends about how fun the game is.
• 7-8: 7s and 8s are Neutrals. They thought it was alright. They will answer that they played the game and thought it was okay if asked, but won't actively promote it or talk about it without prompting.
• 0-6: This demographic of people are Detractors. They will swing from negatively talking about a game if asked to actively trashing it in a social setting.

Okay, this is where the magic happens. You find the Percentage of people that are Promoters versus all of the scores. Say this is 4312 people out of a total of 10137 scorers, being 42.5% You then find the Percentage of Detractors of the total. Again, say we had 2144 people score between 0-6, being 21.1% of the total. To work out the Net Promoter Score, you simply subtract the percentage amount of Detractors from Promoters.

It's so simple.

It's so named because it's telling you the amount of Promoters you net once you subtract negative attention. This measure is removing the wishy-washy middle of scorers, and simply pitting the amount of people that decide it's worth playing, versus the amount of people who don't think it's worth playing. The score can range anywhere from -100% to +100%.

By doing this, we're giving a community-driven score that actually has context without words, and therefore is a fairer measure than simply ripping a number out of an article without context.

Again, it's not perfect and all arbitrary scoring measures will face similar issues to one another, but I think this would give you a pretty good indication of whether the game is worth playing or not without reading a review.

Then again, nothing can really beat a good review by a person who has similar tastes to yourself; If you're able to find a reviewer like this, then stick with them for as long as you can as they'll reliably deliver quality critique of a game to you as an individual... just not necessarily everyone else.