• Log In | Sign Up

  • News
  • Reviews
  • Games Database
  • Game Discovery
  • Search
  • New Releases
  • Forums

Adventure Gamers - Forums

Welcome to Adventure Gamers. Please Sign In or Join Now to post.

You are here: HomeForum Home → Other → Site & Forum Feedback → Thread


   

AG’s Reviews Rating (DEBATE!)

Avatar

Total Posts: 7620

Joined 2011-10-21

PM

Some of you are giving far too much weight to a rating on a review site. It’s just a subjective score, it’s going to be different if someone else reviews the same title.
I also don’t get why you would go and compare titles with the same rating. If the games aren’t similar (i.e. not in the same subgenre), and if the reviewer isn’t the same, and if the games aren’t from the same time period, then there’s just no comparing. And even if they are “comparable”, the rating is just a subjective indication.

Just look at AG’s top 100: The Dig is on there, while the review only gave it a moderate 3 stars, and in comparison, Time Hollow isn’t on the top 100, even though the (raving) review gave it 4 stars. Were the reviews wrong? No, they were just made by different people with different tastes. Is the Top 100 wrong? No, it was just a consensus list made by a group of reviewers. Is one game better than the other? Well, if you’d only look at the rating, you’d say “yes, Time Hollow is better”, if you’d only look at the top 100, you’d say “yes, The Dig is better”. The only true measure, however, is playing both yourself and then deciding for yourself.
It sounds to me like some of you are taking ratings given by reviewers as gospel. Ever considered that some reviewers seldomly give low ratings and thus average at 3.5 stars, while others rarely give high ratings and average at 2 stars? I’d say this site is doing a fine job at trying to keep the averages at about 2.75 stars, and that’s thanks to the editing by Jackal.


Also:

natalia -

The other part of this discussion I find interesting is the thought of relying on a score to make a decision about a game. I love reading book and movie reviews and would never rely on a score alone to sway my purchase decision. While I do tend to gravitate toward reviewers that I know have tastes similar to mine, I also read the ones that are fantastic writers and manage to reveal truths about the work that I wouldn’t get anywhere else, making me seek the reviews out even after I’ve read the book or seen the movie.

^ This.

If you want to know whether or not you’re going to like a game or not, you should really put some effort into finding a reviewer whose tastes are similar to yours (and even then you’re bound to disagree on a lot of points - we’re all individuals, and reviews are inherently subjective), or at least into reading some balanced reviews.


And I fully agree with this:

Kurufinwe - 11 July 2012 12:24 PM

I like the star ratings a lot more than percentages, but people really need to stop thinking of them as scores that can be used to rank games on some sort of universal scale that cannot exist. I think the only way to think about them is as recommendations:

* 2 star and below means the game should be avoided.
* 2.5 and 3 stars means the reviewer thinks the game does not really succeed at doing what it’s trying to do, but if you’re really passionate about this style of games and you lower your expectations, you might enjoy it.
* 3.5 stars means the game tries to do something and acquits itself competently; if it’s your style of games and you have the time and money, you should check it out.
* 4 stars means the game is really, really good at what it does; if it’s your style of games, you really don’t want to miss it.
* 4.5 and 5 stars means the game is exceptional; even if you’d think it’s not your style of games, you should give it a chance anyway, because it’s Just That Good.

And I think that’s enough. Sure, sometimes you might need to make up your mind between two 3.5-star games, but I don’t think knowing one is 68% and the other 71% would help much. At that point, it boils down to personal preference and deciding for yourself which game appeals to you more based on style, gameplay, etc. (i.e. RTFR!)

     

Now playing: Blade Runner (post-CPT) | The Witcher: Enhance Edition (on hold) | Danganronpa: Trigger Happy Havoc (on hold) | Professor Layton and the Azran Legacy (3DS)
Recently finished: Whispers of a Machine (CPT) - 4/5 | Beneath a Steel Sky (CPT) - 3/5 | 3 in Three - 3.5/5 | Puzzle Gallery: At the Carnival - 2.5/5 | The Fool’s Errand (replay) - 3.5/5 | The Dig (replay) - 4.5/5 | Return of the Obra Dinn (CPT) - 4/5 | Beavis and Butt-Head in Virtual Stupidity - 3.5/5 | League of Light: The Game (CCPT) - 3/5 | realMyst: Masterpiece Edition - 2.5/5 | Contradiction - 3/5 | Tex Murphy: Mean Streets - 2/5 | The Last Express - 3.5/5 | South Park: The Fractured But Whole - 4/5 | Indiana Jones and the Fate of Atlantis (replay, CPT) - 5/5

Avatar

Total Posts: 1178

Joined 2012-02-17

PM

TimovieMan - 11 July 2012 12:53 PM

I’d say this site is doing a fine job at trying to keep the averages at about 2.75 stars, and that’s thanks to the editing by Jackal.

You were doing so well, until you got to this.  Grin I appreciate the vote of confidence, but lest this launch another wave of “editor decides the scores!” objections, I’ll just reiterate that the only thing I do (well, besides all the editing) is to demand that the score and text match, and that the rating accurately reflects our scoring criteria based on what is written.  (i.e. if someone praises a game extensively and then still only gives it a 3, or raises really significant issues and gives it a 4, that’s out of sync with our own criteria)  That ensures that one reviewer’s idea of 3 stars is the same as another’s, even if they don’t agree on what games deserve it.

But I can’t and don’t exert any control over actual averages.  Sometimes a month with Resonance, Botanicula, and Walking Dead is just going to be a damn good month.  Smile  And some months we just get a bunch of turds. 

Kurufinwe - 11 July 2012 12:24 PM

* 4 stars means the game is really, really good at what it does; if it’s your style of games, you really don’t want to miss it.

Excellent summation, except I’d take one “really” out of this one.  The really reallies are the HIGH fours, and the single reallies are the low fours.  Tongue

     

Editor-in-Chief, some obscure little site called Adventure Gamers

Avatar

Total Posts: 643

Joined 2006-09-24

PM

Has AG considered a check-plus, check, check-minus grading system?

Wink

     
Avatar

Total Posts: 7620

Joined 2011-10-21

PM

Jackal - 11 July 2012 01:33 PM

But I can’t and don’t exert any control over actual averages.  Sometimes a month with Resonance, Botanicula, and Walking Dead is just going to be a damn good month.  Smile  And some months we just get a bunch of turds.

I was thinking more in the lines of “you’re trying to convince reviewers that generally give too many stars to lower their averages a bit, and vice versa”.
Not that you’re looking at the overall averages with a magnifying glass…

     

Now playing: Blade Runner (post-CPT) | The Witcher: Enhance Edition (on hold) | Danganronpa: Trigger Happy Havoc (on hold) | Professor Layton and the Azran Legacy (3DS)
Recently finished: Whispers of a Machine (CPT) - 4/5 | Beneath a Steel Sky (CPT) - 3/5 | 3 in Three - 3.5/5 | Puzzle Gallery: At the Carnival - 2.5/5 | The Fool’s Errand (replay) - 3.5/5 | The Dig (replay) - 4.5/5 | Return of the Obra Dinn (CPT) - 4/5 | Beavis and Butt-Head in Virtual Stupidity - 3.5/5 | League of Light: The Game (CCPT) - 3/5 | realMyst: Masterpiece Edition - 2.5/5 | Contradiction - 3/5 | Tex Murphy: Mean Streets - 2/5 | The Last Express - 3.5/5 | South Park: The Fractured But Whole - 4/5 | Indiana Jones and the Fate of Atlantis (replay, CPT) - 5/5

Avatar

Total Posts: 1178

Joined 2012-02-17

PM

TimovieMan - 11 July 2012 02:52 PM

I was thinking more in the lines of “you’re trying to convince reviewers that generally give too many stars to lower their averages a bit, and vice versa”.

But I don’t do that either.  That would still be imposing my will on a reviewer, asserting that they’re consistently over- (or under)rating games over time.  All I can do is make sure they justify their arguments each and every time. 

What I have done on occasion is tell a reviewer “you’re saying X about a game, and many others are saying Y.  How do you account for that discrepancy?”  But that’s merely food for thought, never an overt attempt to change their mind.  Better I ask first than risk the reviewer having second thoughts later.  But so long as they’re sure in their assessments, they’re good to go. 

Admittedly, if someone consistently turned in 4’s and 4.5’s for rather obviously mediocre games, or 2’s and 2.5’s for widely acclaimed ones, I’d have cause to raise some questions.  But honestly that’s never come up.  Since we do stress emphasizing both positives and negatives (in whatever proportions they exist), that kind of forces reviewers to be more analytical, and I’ve always been impressed by the commitment of AG writers, year in and year out, to do just that.

     

Editor-in-Chief, some obscure little site called Adventure Gamers

Avatar

Total Posts: 7747

Joined 2012-01-02

PM

OK , this is getting so intriguing..
i want to put 2 points that would explain my Fears,

and please dont mind me talking to myself more than directing this to anyone.

Mr.Jack is doing a Great Job, yes i/we all can see his dedication , so why i have some fears when it comes to the Review’s Rating,ok i will not do the same mistake again and sound like if i knew better than Mr.Jack and the people runnning the site here , because he/they surly thought of all (this and that) or even took care of it already.. but here i am just expressing my fears.

i think its because the relaunch and having the feeling that if i do not explain my fears now i will have to wait maybe 3-5 years more to tell them when another relaunch would take place. i sound greedy indeed Smile

  i will stick 2 two points (2 fears) that i want to gather them within this issue of ratings.
the 1st, is kinda impossible (i guess) to get a feedback that can remove my fear about:
how to make sure the the reviewer is (kindly)
neutral
(we are all human after all) and even if we accepted his/her pesonal taste appearing through the review how would we be sure that his/her taste is up to be good a example that we all can accept with all the different tastes we have?? .

A reviewer who loves only 3rd person games( hypothetically speaking )
but still has big share in playing 1st person would be comforting , but at least we all dont want a MI Lover reviewer, reviewing Aura, or A Myst fan ,reviewing AoM…

the 2nd i guess is most imporatnt :
i need to believe/touch that the stars explains/describes the review (keeping in mind its neturel), i am afraid that looking at the Rating’s Stars would be like taking my parent’s orders the way “we know better trust us son” couldnt these stars explain a little more than just obliging me to read the whole review to understand them.

i will suggest a formula ,but far away from calculation and percentages ... and numbers,
its just (imo) gather all the elements of good Adventure which the
player needs to make sure of how they had been represented in the game through
reading the review and looking up the ratings.

1- graphics, designs ,backgrounds ,artworks ....
2- animation, cut scenes, character and surroundings ,movements ...
3- story,and is it serving the game , originality ...
4- gameplaying, interface, controls, puzzle crafting, interacting with soundings,and the living things in the game,
5- Sound ,voice acting, dialogues and how they all fit ,music, sound effects ...
6- puzzles, logical or not, easy or hard , fit the story or they are just there and pop-up like they just pushed into the game with no relevance ...
7- financials!!, (production) price and how its fair maybe to length of the game,and its production .
8- response (if it was released in another language or country before) , copies sold maybe in the 1st week like the case with Movies’s US-Box office ,  just the commercial feedback ...

i have no idea & and 8 can be represented through the review but as i said i just telling my fears not exactly what i want !

all those elements   matters (imo) are very important for me a have a good picture of how the game would look like, seeing a couple of the game shots does not serve me well, maybe they are taking from the best scenes or just just not enough ,can not trust them .
at last i want to make my opinion and vision/picture about the game that will about spend 20-40 $ in it.

     
Avatar

Total Posts: 1178

Joined 2012-02-17

PM

I always ask for a volunteer to review each game, so it’s extremely rare for a certain type of game to be given to a reviewer who doesn’t particularly like that style. 

I’m really not sure I understand your other suggestion.  Those are exactly the sorts of categories our reviews already cover.  Well, not “response”, which has no business in a review, nor the “financials”, which stops being relevant as soon as the first price drop.  But everything else is in there now.  If you’re asking for a bullet list of how a game does in all these categories on top of the review, I don’t see that ever happening.  It’s more than enough work just writing the review without having to break it down into point form for people who can’t be bothered reading. 

If you’re talking about actually scoring the game in each of those categories, that just makes things worse.  People are complaining about ONE subjective score now; having multiple subjective scores won’t help.  Nor is each category equally important across different games.  Voice acting and dialogue are far more relevant in some games than others.  Easy puzzles aren’t as big a problem in a story-driven game as they would be in a puzzle-based game. And so on. There’s no universal standard; it’s all dependent on context.  And as I said, it’s ultimately all subjective.  One person may like pixel art, another may not.  What’s hard for one person may be easy for another.  There are so many variables in a game, and putting a number on each is more trouble than it’s worth.

And if you didn’t mean either of those two things, then you’ll need to explain it better.  Tongue

     

Editor-in-Chief, some obscure little site called Adventure Gamers

Avatar

Total Posts: 7747

Joined 2012-01-02

PM

i was meaning the 2nd; scoring the game in each of those categories, but as you said “There’s no universal standard”, these 1st 6 matters i mentioned wouldn’t/Shouldn’t be all considered with the same value or weight .

i am surly have NO IDEA if i to do this what the mechanism it can be taken to score the game upon those elements, but if we keep out the scoring and rating and put them aside , maybe those elements can only be described in a box same as the game’s box for prescriptive and Date ...etc.

so you can have a Review, rating and the major Adventuring elements described within that box with some very a short sentences (satisfying)  for each one. 

Edit: if that what you already meant with (a bullet list) then i guess you answered this suggestion Smile
it seems no way to change your mind or even make you rethink/consider a subject into planning Mr.Jack, and your answers are always ready and convincing Smile

     
Avatar

Total Posts: 643

Joined 2006-09-24

PM

The ratings system and review process are quite fine to me.

     
Avatar

Total Posts: 278

Joined 2008-07-11

PM

Jackal - 08 July 2012 12:27 PM

Not to undermine the reviewer (who to this day still maintains it’s not a particularly praiseworthy game), but simply as a matter of fact, that was written long before our editorial standards were fully established.  (And well before my time.)

With that in mind, have you considered alternate reviews for games? If a review is just one person’s opinion, then where’s the problem in having multiple opinions on a game? Or do you feel that multiple reviews could dilute the editorial voice of Adventure Gamers, or possibly confuse readers?

I think I prefer the idea of simply redoing older reviews that don’t adhere to current editorial standards. The original writers should understand that you’re doing what’s best for the site and the readers.

Personally, I’ve only ever strongly disagreed with one AG review (that I remember) and that was Runaway: A Road Adventure. I bought the game from The Adventure Shop on Adventure Gamers’ recommendation (very positive write-up, 4.5 stars) and to my surprise…it was terrible. So laughably bad that I just couldn’t believe someone would recommend spending money on it. I later read that there was a serious adventure game drought at the time, and a huge amount of hype had grown around the launch of Runaway, so maybe that was partially the cause.

     
Avatar

Total Posts: 320

Joined 2003-09-16

PM

Just my 2 cents.
I don’t put much stock in any ratings unless I know the particular tastes of the reviewer. I want a description of the game - how it plays, types of puzzles, 3rd person or 1st, dialogue, etc. I know what I want from a game and what I don’t want. It’s as easy as that.
Of the recent (and there’s not many) games that I’ve played, the ones that I thought exceptional are listed, along with their ratings:
Rhem 1   2/5
Rhem 2   2/5
Rhem 3   3.5/5
Rhem 4   3.5/5
Outcry   3/5
Slip Space   2/5
Machinarium   4.5/5
Botanicula   4.5/5

     
Avatar

Total Posts: 4011

Joined 2011-04-01

PM

colpet - 12 July 2012 08:15 AM

Of the recent (and there’s not many) games that I’ve played, the ones that I thought exceptional are listed, along with their ratings:
Rhem 1   2/5
Rhem 2   2/5
Rhem 3   3.5/5
Rhem 4   2/5

Rhem 4 got 3.5/5
Most people who’ve played the Rhem games mention 1 or 2 as the best one. Reading the review for Rhem 1, I can’t understand why it only got 2/5 if you take away the “I don’t enjoy hard puzzles in a minimal environment” viewpoint of the reviewer. He even says “It is obvious that Mr. Mueller’s talent and focus lie in puzzle theory and composition” like that’s a bad thing. The Rhem 2 review is slightly better, focusing on some negative technical aspects.

     
Avatar

Total Posts: 105

Joined 2003-11-15

PM

Oscar - 12 July 2012 09:16 AM

Reading the review for Rhem 1, I can’t understand why it only got 2/5 if you take away the “I don’t enjoy hard puzzles in a minimal environment” viewpoint of the reviewer. He even says “It is obvious that Mr. Mueller’s talent and focus lie in puzzle theory and composition” like that’s a bad thing. The Rhem 2 review is slightly better, focusing on some negative technical aspects.

I haven’t played Rhem (nor am I interested in doing so) so I’m not going to argue specifics on that review, but what you write seem to me to show the problem with ratings in general rather than any problem with this site. The text makes it clear that it’s your kind of game even though the reviewer doesn’t like it much. Isn’t that pretty much just what a good review should be like?

     

You can play my game Frasse and the Peas of Kejick for free! (AG review here.)

Avatar

Total Posts: 7747

Joined 2012-01-02

PM

Trumgottist - 12 July 2012 10:04 AM

The text makes it clear that it’s your kind of game even though the reviewer doesn’t like it much. Isn’t that pretty much just what a good review should be like?

will , upon what you said , i say ” i wish the reviewer taste would stay within his conscious and never leaves or jumps into the Review itself”

Sorry Oscar and Trumgottist for interrupting Smile

     
Avatar

Total Posts: 1178

Joined 2012-02-17

PM

Advie - 12 July 2012 12:54 AM

it seems no way to change your mind or even make you rethink/consider a subject into planning Mr.Jack, and your answers are always ready and convincing Smile

That’s not true at all.  But I/we have have been at this for quite a while now, so lots of ideas have already been considered in some form or another.  Some are rejected on principle, and others just because there’s only so much time and energy to go around.  I’d say your bullet list suggestion (and yes, that is what I was referring to) belongs in the second camp.  All things being equal, with lots of time and personnel to deal with, I can see value in doing that.  But stretched as close to breaking as we (always) are, I just think it’s more trouble than it’s worth. 

orient - 12 July 2012 04:22 AM

With that in mind, have you considered alternate reviews for games? If a review is just one person’s opinion, then where’s the problem in having multiple opinions on a game? Or do you feel that multiple reviews could dilute the editorial voice of Adventure Gamers, or possibly confuse readers?

It’s more a practical matter than anything.  It’s tough enough finding one reviewer for every new game that comes out, let alone two (and since we only get one review copy, if that, a second person would have to pay for the privilege of doing the extra work).  And once you open up that “second opinion” can of worms, where do you stop?  Does one game deserve it and not others?  Doing even one would be a precedent-setting nightmare.

That’s why we’ve got the user review option now, plus comments for alternate opinions.  It may not be as “official”, but it essentially serves the same purpose. 

We have replaced some of the older reviews, and I’m not opposed to doing more of that.  But again, it’s a matter of time and manpower.  Along with all the new games to cover, there are a whole lot of older games we never reviewed that I’d still like to get to (and never seem to).  On a priority scale, re-reviews would come after both of those.

     

Editor-in-Chief, some obscure little site called Adventure Gamers

You are here: HomeForum Home → Other → Site & Forum Feedback → Thread

Welcome to the Adventure Gamers forums!