Jump to content
Awoo.

Number Ratings in Reviews


RedFox99

Recommended Posts

This is a topic that I've been thinking of a little bit recently after reading a status post on the forum. I remember it was when someone said that gaming companies pretty much bullied gaming magazines to raise the average score for an average game from around 5/10 to 7/10 and the discussions to follow talked about how we use ratings from reviews.

Now with that in mind, I'm beginning to wonder if we should reconsider the follow of using ratings when evaluating rather a game is good or not, especially since everybody's idea of a passing score should be different. So, I wanted ask for your guys' input on this topic.

Link to comment
Share on other sites

I find it a little unbelievable since one would think game companies with any clout and capital would just buy reviews instead of putting pressure on press outlets to call certain scores average (whether they make sense as actual numerical averages or not).

I always associated the public perception, at least in America, of the average score being the way it is is because of how our school grading systems are set up. Scores at 59% or below are considered failing (F), 70-79% are average (C). To be honest, I don't really think too heavily on the scores since I'm more curious about why they're scored the way they are. As impressive as Sonic Forces' 35/40 Famitsu score sounds, there's a reason people wrote that off when it seemed like the reviews had little to do with how the game actually functioned.

  • Thumbs Up 3
Link to comment
Share on other sites

I don't think ratings alone should be used when evaluating if a game is good. As you mention, different people use different scales. For example, 5/10 to one reviewer might be "The game was decently enjoyable but had serious flaws" whereas to another it might be "This game isn't unplayable and functions well enough, but its not fun." And that says nothing of what the reviewer thinks should be done-- even somebody in the latter camp for the aforementioned 5/10 may not hate the concepts behind the game or think every part isn't fun. This can go the other way around too, where somebody in the former camp for the aforementioned 5/10 may have enjoyed the game but not really want anything in the future to resemble it. Context is always important.

And I also think it has the side effect of invalidating or lowering the significance of reviews from reviewers who don't use a number system, as they often get pushed aside along the way. The rise of Metacritic and other aggregate review sites have only highlighted this issue as professional reviews without something that can be translated to a score just straight up don't get listed-- so for people who only look at the Metacritic, they may as well not exist. Even something that can be translated into a numerical score such as a letter grade may not be represented with the proper weighting. A C can be average like it is in the US school system, or very good like it is in the UK school system, but if the site is operated from the US, well, how do you think the site will be programmed to translate a C to? It'll probably be to fit the US's perception of a C. Some sites have gotten better about mentioning if a reviewer has a tendency to be harsher or lighter on games, but even that's based purely on number ratings instead of what the reviewer is actually saying. The solution from my perspective would be to have a statistician-curator go through the reviews thoroughly and weight their scores accordingly, but even that's not perfect and its expensive and time-consuming-- not to mention that we'd of course want to know who the curator is so that we can confirm that they're trustworthy and competent (or not). 

I actually wound up getting involved in a discussion here about this topic. Its pretty interesting, though unfortunately it ended with a mod having to step in.

The perception amongst non-professional reviewers is often that anything below 69 is horrible, so @Zaysho is right there. But that doesn't mean that the reviewer actually thought the game was horrible when they gave that score. (And well, I don't really think the American grading system is all that good but that's a whole 'nother topic right there) Part of combatting that perception is by challenging it, and showing what different reviewers mean by their scores when people are throwing around review scores to make a point without considering the context.

As for game companies paying reviewers for good reviews-- well, @RedFox99 and @Zaysho, you're actually both right in some way. Game companies do pressure reviewers to give higher scores, most typically by threatening to pull the company's advertisements from the review site and order no more until further notice. But these advertisements, especially from large companies, effectively fund the review site. So it could be described as indirectly paying people to give higher scores while punishing people who don't. Which... kinda defeats why gamers who have lots of experience with various games but also were outside the industry came to be more trusted to begin with. Before the days of IGN and such, you only got reviews from dedicated magazines, which were in turn run by companies that made or published the games. Said companies saw zero reason not to shamelessly plug and lie about the quality of their product in these reviews, so a game on the quality level of E.T. for the Atari could easily get glowing reviews across the board if the company found the right bullshit artist(s) for the job. Some publishers such as Atari also would not credit the developers at all (and the easter eggs that hid the developers' names were only accessible from the game proper, so if you didn't own the game you had to take your friend's word for it until you could prove otherwise), so it was often hard to tell if the developers were trustworthy enough that the glowing review could be believably true or not. But if the companies are able to fudge the numbers anyway... well, what's the point of leaving their magazine scores in the dust if they'll just find another way to get the scores they want? At least the content of the outsider gamer's review is mostly unchanged here, but the score itself ought to be taken with a grain of salt with this in mind.

  • Thumbs Up 3
Link to comment
Share on other sites

4 hours ago, RedFox99 said:

I remember it was when someone said that gaming companies pretty much bullied gaming magazines to raise the average score for an average game from around 5/10 to 7/10 and the discussions to follow talked about how we use ratings from reviews.

 

42 minutes ago, Mad Convoy said:

Game companies do pressure reviewers to give higher scores, most typically by threatening to pull the company's advertisements from the review site and order no more until further notice. But these advertisements, especially from large companies, effectively fund the review site.

I might have misunderstood RedFox's post after reading Mad Convoy's, so I was hoping I could get clarification on this. Was the point about companies pressuring outlets to raise the scores of games (so, make something sound better than it is), or that they somehow forced a big change in what constitutes an average score? I can understand the former but the latter just doesn't make sense to me since it seems like a cultural sort of thing, so I'm not sure if there's something in the OP's wording I just didn't get.

  • Thumbs Up 1
Link to comment
Share on other sites

14 minutes ago, Zaysho said:

 

I might have misunderstood RedFox's post after reading Mad Convoy's, so I was hoping I could get clarification on this. Was the point about companies pressuring outlets to raise the scores of games (so, make something sound better than it is), or that they somehow forced a big change in what constitutes an average score? I can understand the former but the latter just doesn't make sense to me since it seems like a cultural sort of thing, so I'm not sure if there's something in the OP's wording I just didn't get.

I know the status the OP refers to, where I was the one who brought this up. The point is of the former, as many video game magazines long ago needed advertisements to stay afloat so to be able to afford production. Alongside those advertisements were reviews on the different games of the time. And back then, average games were given a 5/10, 50/100, 50%, whatever.

One day, a company hated this score that was given by a magazine, and threatened to pull their advertisements unless they upped the score. Under the threat of companies pulling out their advertisements unless they got the scores they wanted, the rating for "average" games began getting an odd amount higher. Eventually, it became the standard.

7/10 games are average. Anything below that is a steaming pile of shit.

Nowadays, the method has changed. Review sites don't necessarily need advertisements to stay afloat, but they need visitors. So companies give them early previews, exclusive interviews, and even money to pump up the score, while the review sites get visitors from their exclusive content. And if the companies don't get the score they paid for, all of the exclusive things the review site gets is gone.

 

Reviews have come down to a number, while the content of the review is skipped. And sometimes, you want to read the review itself to get an idea of if the game is good and you want it instead of a number saying you should get it. But the average person wants easily consumable content, and game companies, aware of that, want to plaster their "goodwill" on this easily consumable comtent.

  • Thumbs Up 3
Link to comment
Share on other sites

Long stroy short staring at the number should not replace the full opinion.

The fact that most "worst games ever" have 40% rating, makes me wonder why we even have lower numbers. For games that send your personal data to government?

I find number rating useful in two situations 1 when reviewer is bad at expressing his opinion 2 when he praised two products more-less equally and I wonder witch of them he liked more. With that said, first one means review is probably of little worth, while second is a trap. What do you like more, playing good RPG or good shooter? Eating pizza or ice cream? Some experience are hard to compare.

  • Thumbs Up 2
Link to comment
Share on other sites

While Jim doesn’t do numbered reviews anymore, which I find myself appreciating as it gets rid of a lot of problems, this video he made I think does a great job highlighting some of said problems and why numbered reviews and the perception and reaction from consumers regarding them can be ridiculous. This doesn’t even bring up his other “controversial” personal reviews that differed from the crowd regarding games like Assassins Creed 2, Sonic Colors, and of course Breath of the Wild. It’s no wonder he just simply dropped the number system straight up. People just seemed to look at that and not the words 

  • Thumbs Up 3
Link to comment
Share on other sites

When I was young as in 20 years ago when magazines such as the Official PlayStation Magazine, N64 Magazine and NOM were around, I used the ratings systems in magazines as a rough guide and anything about 6/10 or 60% meant that the game was good and the higher the score, the better the game was. I also did read the summaries and much of the time the actual text as well because not every game was to my tastes or had some issues. It sort of make sense to a young person because of the price of games at the time and not every game got a demo back then nevermind now where you are lucky to even see one. It happens to be that many of the games that were purchased are now considered classics by people. Not every game followed this path though like I bought Cruis'n USA because I liked the arcade game despite the game itself got low scores. Most of the time, the low scores were reserved for broken games or just bad games however it also affected a few series too. The Mega Man series was negatively reviewed at the time.

Now though the number system doesn't really make sense. What's the difference between a 8 or a 9? Exactly and that's subjective. It doesn't tell you much about the game itself. All it is saying that buy the game because of the score, not because of the game itself. It's like here's a few screenshots, a video or two, go buy it! What if you don't like the game? The movie industry has the same problem with this system with its stars and Rotten Tomatoes score causing some criticism. Books too to a smaller scale but that has a completely different issue that's just as corrupt (famous writer opinion). Even if you removed the scoring system, there are still problems with reviews. For the past 10 years or so that the reviews are getting less and less detailed in general, many reviews that used to be throwaway in magazines are getting closer to the standard and places like Metacritic for example are encouraging one line or extremely short reviews. Taking one random example, Eurogamer used to have about 3 pages worth of a review during the 6th generation and might have compared another version but now its just one page and many review sites don't even have second opinions either.

The major problem with the rating system is that it focuses on popularity both good and bad rather than overall, there are also many games that either don't get a review or very niche places take the challenge. It means that many of what are considered underrated games, games people missed out because they weren't reviewed and people weren't willing to try them.

Even then the problem with reviews, rating system or no rating system is the element of bias from the reviewer no matter whether its a magazine, website, Steam or on Youtube. Like something? Positive review. Don't like something? Negative review. Someone who likes one genre might not like another genre. A fan is most likely either going to praise the series no matter what or go into some detail some more. Seriously many of the reviews on Steam are jokes. Also another thing regarding reviews is that apart from perhaps the odd PC review or somewhere like Digital Foundry, they don't go into the technical details very much. A game might be described as groundbreaking but its not a pleasure to play if it is constantly crashing, slowdown/bad framerate drops, wiping progress or annoying bugs. Battlefield 4 is a good example regarding this where reviews glanced the big issues that the game got, some that still aren't fixed to this day or the LEGO games in general.

Reviews in general just need a big shake up, not just removing the score system but changing the way that they are reviewed.

  • Thumbs Up 4
Link to comment
Share on other sites

I've never understood decimal review scores. Rating a game on a 20 point scale (IE. 8.5, 9.0 etc.) seems ridiculous but a few years back IGN have started using a 100 point scale. How the fuck can you give a game a score of 8.7 or 5.2 or 9.6? Their scores aren't even an aggregation of scores by multiple reviewers, it's just one reviewer plucking these bollocks numbers out of mid air.

  • Thumbs Up 3
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...

Important Information

You must read and accept our Terms of Use and Privacy Policy to continue using this website. We have placed cookies on your device to help make this website better. You can adjust your cookie settings, otherwise we'll assume you're okay to continue.