Game reviewing is still broken

Game reviewing is still broken

Game scores used to make sense, but then they took an arrow to the knee. Over-used memes aside, we're serious - what's up with game scoring and gamers?

 

Scoring a game is a funny old business. We’ve talked about it a bit before, as it’s a subject that’s close to our heart, and it’s one that we think is worth revisiting every now and then. It’s on our mind again now because of some recent comments we’ve seen around the ‘tubes, from both gamers and game journalists.

Basically, it looks like the way some reviewers rate games, and the way a lot of readers interpret those ratings, is getting far worse, rather than better. And it’s going to hurt the industry if it keeps up.

Readers, WTF?
Of course, we have to say up front that this is not all readers, so don’t start wishing fiery-cancer-AIDS upon me just yet. However, there are some folks out there, not necessarily on Atomic, that have inspired more than a tiny fear that we as a race are inherently doomed.

My case in point is our Skyrim review. This is not online yet, as it was a print exclusive and we didn’t want to harm sales for the issue. Suffice to say, we (or, more accurately, Nic Healey, the reviewer) really liked it; it got a mighty 95 per cent and a Hot Award. Looking back at that, even in light of Bethesda’s less than stellar patching efforts and the issue’s that PC gamers have the release, I still back that score. I love the game myself.

However, when that review did leak online (and man, that’s an epic story in itself!), I got an interesting insight into how the rest of the world looks at reviews like that. The Atomic Skyrim review pretty much went viral overnight, as it was leaked a week before it was meant to, and you’d think a lot of folks would be kind of impressed with a score like that.

Wrong.

There was a mix of frankly fucked up thoughts on the score, that pretty much ran the gamut from “Only 95?! I’m cancelling my pre-order!” to “95?! These guys must be getting paid! I’m cancelling my pre-order!”.

Now, that first attitude is bad enough, because it’s fed by, and feeds back into, the way a lot of reviewers really over-use the upper-end of the scale. There’s not a lot of outfits these days that use the lower half of any scoring system, and it could very easily be argued that even the lower three-quarters of any given scaling mechanism is sadly ignored. However, in these days of tighter and tighter household budgets, perhaps gamers are wanting only the best; that said, surely anything in the nineties, or similar, is bloody good!

However, that’s undermined in turn by the insistent belief that publishers pay reviewers for good scores. I can’t say for certain, of course, but I have never heard of anything like this. I’m certain this doesn’t happen in Australia, and pretty sure the rest of the world doesn’t operate like this either; there seems, however, than a certain conspiracy-minded segment of the gaming community would rather believe we’re all corrupt than perhaps consider the idea that some reviewers simply have a different opinion. It reminds me a lot of 9/11 Truthers, actually; I sometimes think that the ‘game reviewers are on the take’ crowd feel somehow more cool that they’ve worked out ‘how it really is’. Which is, of course, kinda sad.

That said, there are ways for publishers to enforce their wishes. Withdrawing advertising is certainly a strong threat, and the now infamous GameSpot incident is a pretty good case in point, though it’s not really conclusive. Nonetheless, the threat does get exercised; the trick is having a healthy enough publication, with wide revenue sources, to be able to wear one publisher (or hardware vendor) getting the shits, and having the spine to just suck it up and stick to your guns as a brand and as a reviewer.

And that’s not easy. I like to think Atomic writes without fear or favour, but I can also say that in some cases it has cost us – that’s why I tend to take accusations of collusion and bribery rather... personally.

There’s another area where readers put a lot of pressure on writers, I think. Too often, in review comments, you’ll see stuff like ‘this is wrong’, ‘this review is stupid’, or the classic ‘I can’t continue to read this review if you dislike game X’. Hell, I’ve been insulted and worse over giving a game negative review! There’s even the subtle putdown, ‘well, this is just reviewer Y’s opinion, so not to be taken seriously’ kind of comment (yes, I’m looking at you, Pappes).

Well, of course it’s all bloody opinion! The idea that there is some kind of hard, solid and measurable metric by which a game can be rated is, well, retarded. There are games out there that I know are perfectly fine, but I would not be caught dead playing, nor would I recommend to others; and yet, I know other people out there love them. This does not mean that those folks are wrong, or right, or whatever – it means that games are complex, inherently subjective experiences. Of course my take on a game is going to different to *yours*. Expecting otherwise is untenable.

However, you can expect a reviewer to state why they don’t like a thing, to explain what the issues are with a game why it may rate lower or higher than other reviews or even reader expectations. It’s certainly what I try to do.

Reviewers and writers, WTF?
Sometimes, however, we reviewers are in fact part of the problem. In fact, my main inspiration for ranting in this much depth comes from a recent Kotaku story, an otherwise entertaining piece on the nature of games that come in threes. Some of it is written with tongue in cheek, some of it even supports some of my own arguments made above, but I do think the author’s actually being honest when he says this:

“the difference between a 94 and an 88 is staggering. A “94″ says “pretty decent video game for people who don’t deserve the absolute best”. An “88″ says “wait for the bargain bin” or “maybe purchase used if you don’t know what to get your grandpa for Old People’s Day”.”

What the hell?!

Now, the guy does claim to not be a real critic, but he writes reviews on his blog, and he’s writing under the tag ‘the real world’ on one of the biggest video gaming sites in the solar system. So I think seeing the suggestion that the review scale only exists between 70 and 100 is kinda damaging, and damaging to all of us invested in this industry.

Sadly, though, none of this is going to change overnight; all this is is me getting stuff of my chest in the middle of a very frustrating production cycle on the magazine. But if you as readers can take a little bit of it to heart, and try not to fall into the belief that we, as reviewers, are all out to pull the wool over your eyes, and to maybe push reviewers to adopt a more useful scale of judgement then maybe something can come of it.

See more about:  game  |  reviews  |  scores  |  scoring  |  theory  |  feature
 
 

Readers of this article also read...

Gigabyte & INTEL OC event 

Gigabyte & INTEL OC event

 
Arma 3 at Atomic HQ 

Arma 3 at Atomic HQ

 
PAX Australia - a celebration of awesome geekery! 

PAX Australia - a celebration of awesome geekery!

 
Exclusive First Look: Gigabyte's Z87X-UD3H 

Exclusive First Look: Gigabyte's Z87X-UD3H

 
Unboxed: Thermaltake Chaser A41 

Unboxed: Thermaltake Chaser A41

 

Latest Comments

Latest Poll

What PC component are you planning to upgrade in the next six months










Ads by Google

From our Partners

PC & Tech Authority Downloads