• Log In | Sign Up

  • News
  • Reviews
  • Games Database
  • Game Discovery
  • Search
  • New Releases
  • Forums

Adventure Gamers - Forums

Welcome to Adventure Gamers. Please Sign In or Join Now to post.

You are here: HomeForum Home → Other → Site & Forum Feedback → Thread


   

AG’s Reviews Rating (DEBATE!)

Avatar

Total Posts: 7749

Joined 2012-01-02

PM

Since i have been Lurking aroud the site and especially lately i surly noticed that there is always Debates from (Us) members around any new game’s Rating.

i guess to make a positive/productive Debate one has to Point out (imho) three Different issues:

1-what we see of the ratings of the Games here at AG are always referred to as OUR’S (from the AG’s Staff) ,...though the reviews and their ratings made by individuals and not under the supervision of a single Mod/source not even Jackal (whom his Job as i can tell with all the respect for him and his daily efforts, is to watch over us not the AG’s Staff) which smashes the whole Idea about that there is an AG’s Standard for rating the games.

so that points to an obvious issue that states ; “there is no theory or a certain vision for Rating here”, and that is why there always a big hassle (from members) around each one.

(who rated Captain Margane with 3.5 was someone else than the one who rated The Dark Eye with same Rating.
when you look at the rating’s/Stars difference between (i.e) Riven and Yesterday to find out it is only half Star, you get the feeling that there is something not quite right.

2-when try connect the review with its rating’s i find (sometimes) it a bit confusing , with such a big ratings one shouldn’t read about lots of cons mentioned through the review ,Like the Case with Yesterday the reviewer was telling “requiring practice”.. “frustration with rushing through activity”...“Objects have no descriptions other than their names”,.. “an annoying hot spot revealer requiring two or three attempts to scan an entire screen” and “unvoiced descriptions”! ..all that and the game takes 4 out of 5 Stars, it just gave me an impression that the reviewer was not quiet sure of the quality of the game or the rating s(he) gave.

3- the new feature of the User’s review Average should be taken more seriously , i see what about an Average for the users Ratings and The Reviewer’s .
and why the rating should be fixed forever ,why there wouldn’t a system of Rating that reveals it after/later than the Review itself,keeping in mind that all those ratings are what the A-Z games section is Mostly helpful/useful for. lets say it a record that stays to whenever, and when you got more 1500 Adventures rated, you wouldn’t want them to be looked at someday later/in the future ,and they smell of big differences in the overall standards .
plus a big number of players/Members depend on scrolling through AG games list/reviews and choose the highest rated games they missed to purchase . so its something that should be handled VERY carefully.

all those points i had posted before here or there but i thought it was time to gather them all into a Topic
(i loved the Thread  definition more than the Topic as it was the case in the Archived AG’s Forums )...

P.S: I started this thread in a intention to pull all the hassles about each game rating’s here, in order to Debate freely about them.

and about me i will be leaving to Moscow tomorrow with my wife which means i would not be able (surly) to interact with AG for a while, so posters here also can relax from my intense posts for a while as well. Cool

Long live AG!  Smile

     

Total Posts: 98

Joined 2004-02-16

PM

Advie - 07 July 2012 04:58 PM

though the reviews and their ratings made by individuals and not under the supervision of a single Mod/source not even Jackal (whom his Job as i can tell with all the respect for him and his daily efforts, is watch over us not the AG’s Staff) which smashes the whole Idea about that there is an AG’s Standard for rating the games.

You’re wrong there. Jack does supervise each and every review (and preview) and makes us reviewers explain exactly why we think a game deserves the rating we give it. If we can’t convince him, we need to rewrite the review and/or rethink the rating until he thinks the pros and cons we describe match the rating we give.

There most certainly is a standard and Jack makes very sure that five stars are really only given to instant classics and 3.5 stars to solid games that lack some polish or ambition etc.

     

Madre de Dios, es el Pollo Diablo!

Avatar

Total Posts: 974

Joined 2007-02-23

PM

The only review on AG I have a direct problem with is the official one for Broken Sword 2, which is so far off base it’s ridiculous. Other than that - no matter if I agree with the reviewer or not - I always find the reviews to be interesting reads with justifiable scores.

     
Avatar

Total Posts: 7749

Joined 2012-01-02

PM

PolloDiablo - 07 July 2012 05:14 PM
Advie - 07 July 2012 04:58 PM

though the reviews and their ratings made by individuals and not under the supervision of a single Mod/source not even Jackal (whom his Job as i can tell with all the respect for him and his daily efforts, is watch over us not the AG’s Staff) which smashes the whole Idea about that there is an AG’s Standard for rating the games.

You’re wrong there. Jack does supervise each and every review (and preview) and makes us reviewers explain exactly why we think a game deserves the rating we give it.

i am glad i am wrong (somehow) at this point,though it still doesn’t seem to solve that repeatedly hassle/issue of individual ratings/standards

but thanks .

     
Avatar

Total Posts: 3933

Joined 2011-03-14

PM

You are not the only one that has noticed a certain amount of inconsistencies in the reviews and ratings lately.

1) It is inevitably that personal taste is going to have a big influence on what kind of review a game is going to get.
As long as different games are reviewed by different people, it simply cant be avoided!
Nor do i want it to be avoided, if the reviewer loved a game despite many flaws, or hated a game despite its objective qualities, then i want the review to reflect that.
(I also want it to reflect the objective qualities and flaws, but that should be self evident)

I might completely disagree with any given review, but that is what the comments section is for Smile

2) This is where i see the main problem.
There are some reviews that has gotten a much higher rating, than not only the pro/con box can justify, but also compared to how positive/negative the whole review was.
Perhaps some reviewers are simply more generous when it comes to handing out stars?
Regardless of the cause, it is something that needs to be addressed.

3) I don’t think it is a good idea to mix the reviewers and users rating into an average, but the user ratings could have a more prominent position, both at top besides the reviewers rating in the actual review, but also in the different lists.
As Advie pointed out, many users browse quickly through the lists, to find high rated games they haven’t played before, a large difference in user and reviewer rating here, would at least alert them that the rating is disputed.

Despite my above comment, and any angry commnets i might have posted to some reviews, i generally thing the AG staff is doing a good job Thumbs Up

     

You have to play the game, to find out why you are playing the game! - eXistenZ

Avatar

Total Posts: 4011

Joined 2011-04-01

PM

Iznogood - 08 July 2012 05:32 AM

Despite my above comment, and any angry commnets i might have posted to some reviews, i generally thing the AG staff is doing a good job Thumbs Up

I think so too. If you remember that reviews are meant to help you decide whether or not you will like the game, scores aren’t too much of a problem. I’m not going to skip over a 2-star review of a game that catches my eye but I’ll read it to see what problems the reviewer had with it, and I’ll read a highly scored game review even if I know it’s something that I’ll probably not enjoy.

Even so, I still love to have a good chuckle when I see Yesterday scoring higher than Dark Fall or that Book of Unwritten Tales scored higher than Myst.  Content

     
Avatar

Total Posts: 1178

Joined 2012-02-17

PM

Kasper F. Nielsen - 07 July 2012 05:18 PM

The only review on AG I have a direct problem with is the official one for Broken Sword 2, which is so far off base it’s ridiculous. Other than that - no matter if I agree with the reviewer or not - I always find the reviews to be interesting reads with justifiable scores.

Not to undermine the reviewer (who to this day still maintains it’s not a particularly praiseworthy game), but simply as a matter of fact, that was written long before our editorial standards were fully established.  (And well before my time.) 

Advie - 07 July 2012 05:49 PM

i am glad i am wrong (somehow) at this point,though it still doesn’t seem to solve that repeatedly hassle/issue of individual ratings/standards

Nothing will ever solve that issue.  That’s strictly the nature of the beast in a task that is ultimately subjective, spread out across a number of different people.  I can only make people justify their arguments, not dictate a common experience. 

Iznogood - 08 July 2012 05:32 AM

(I also want it to reflect the objective qualities and flaws, but that should be self evident)

...

There are some reviews that has gotten a much higher rating, than not only the pro/con box can justify, but also compared to how positive/negative the whole review was.
Perhaps some reviewers are simply more generous when it comes to handing out stars?
Regardless of the cause, it is something that needs to be addressed.

You’ve actually answered your own question here.  It’s precisely because of the need to “reflect the objective qualities and flaws” that even a positive review can have many criticisms. Even though flaws may not bother a reviewer personally, or really impact their experience with the game particularly negatively, their mere existence means we need to mention them (they could certainly bother other people more).

Obviously not all “cons” are weighted equally (and this goes to Advie’s point about Yesterday).  Some are deal-breakers, some really hamper one’s enjoyment of a game, and some are just minor annoyances that are easily overlooked by all the good stuff.  (Overlooked while playing, but not while reviewing. Wink) If a game nails its story and puzzles but has lots of little polish issues, that’s why what could have been a classic is marked all the way down to 4 or 3.5.  It’s still a pretty darn good game, but it’s our duty to point out all the warts that prevent it from being so. 

It’s a rather unfortunate reality that it takes longer to explain problems than it does to offer praise.  That can seem misleading, but it’s all about the context. 

Oscar - 08 July 2012 10:55 AM

Even so, I still love to have a good chuckle when I see Yesterday scoring higher than Dark Fall or that Book of Unwritten Tales scored higher than Myst.  Content

That’s really comparing apples and oranges, though.  They may all be adventures, but they’re so different that even if they were all reviewed by the same person, the scores might still vary quite drastically from your own (or mine, or anyone else’s).

     

Editor-in-Chief, some obscure little site called Adventure Gamers

Avatar

Total Posts: 4573

Joined 2012-03-24

PM

It must be such a difficult job as a review writer to present a balanced opinion, take all aspects into consideration & give a fair rating.

Personally I don’t take too much notice of the rating but pay a lot of attention to the details of the content of a game. 

The fact is that an A rated game such as BoUT (on a lot of sites) which I thought was a lovely game which deserved that rating wasn’t for me as enjoyable as 15 Days which unfortuntely invited a far lower rating due to it’s technical shortcomings!

It doesn’t matter how good a game is & how perfect it is, if it’s not to your taste….

     
Avatar

Total Posts: 2106

Joined 2007-01-04

PM

Most of the time I believe that the review is spot on. I bought a used PSP for sixty bucks and a copy of “Shattered memories” for 9 bucks - and it is a four star top 100 adventure game - amazing. The game got a great review, The part in the review on the evolution of the main character’s psyche sold me on the game.

The score for “999” I though was a little low, I loved that game but overall great reviews like the one Shattered Memories got are the norm.

Neutral

     

I enjoy playing adventure games on handheld systems- PS VITA, Nintendo DS and ipad mini.

Total Posts: 3

Joined 2004-02-05

PM

PolloDiablo - 07 July 2012 05:14 PM
Advie - 07 July 2012 04:58 PM

though the reviews and their ratings made by individuals and not under the supervision of a single Mod/source not even Jackal (whom his Job as i can tell with all the respect for him and his daily efforts, is watch over us not the AG’s Staff) which smashes the whole Idea about that there is an AG’s Standard for rating the games.

You’re wrong there. Jack does supervise each and every review (and preview) and makes us reviewers explain exactly why we think a game deserves the rating we give it. If we can’t convince him, we need to rewrite the review and/or rethink the rating until he thinks the pros and cons we describe match the rating we give.

There most certainly is a standard and Jack makes very sure that five stars are really only given to instant classics and 3.5 stars to solid games that lack some polish or ambition etc.


Well, I don’t want to be offensive or anything, but this certainly does not seem like good practice. Or, if I didn’t get it the right way, please do correct me.

The fact that every single review, even if written by very different reviewers with very different points of view, has to go through the filter of a single person is definitely wrong. What makes Jackal more capable of saying what deserves a 3.5 and what deserves a 5? This is utterly wrong and bad policy. One thing is to revise every article to ensure it meets a minimum of quality and rigor; to make reviews undergo the personal tastes of an editor is a very different one.

I for one, tend to digress with most of Jackal’s opinions (which is nothing against Jackal himself), and knowing that he supervises all reviews in this site to the point of rejecting texts if he’s not “convinced” explains a lot about the editorial line of Adventuregamers in the last few years.

Also, this a consequence of the stupid tendency of giving the numerical score any importance. It’s just a quick way to say “I liked it, I didn’t like it”. It never should be before the text, to the point of rewriting it to match a score.

 

     
Avatar

Total Posts: 49

Joined 2004-11-26

PM

Well, I don’t want to be offensive or anything, but this certainly does not seem like good practice. Or, if I didn’t get it the right way, please do correct me.

The fact that every single review, even if written by very different reviewers with very different points of view, has to go through the filter of a single person is definitely wrong. What makes Jackal more capable of saying what deserves a 3.5 and what deserves a 5? This is utterly wrong and bad policy. One thing is to revise every article to ensure it meets a minimum of quality and rigor; to make reviews undergo the personal tastes of an editor is a very different one.

I for one, tend to digress with most of Jackal’s opinions (which is nothing against Jackal himself), and knowing that he supervises all reviews in this site to the point of rejecting texts if he’s not “convinced” explains a lot about the editorial line of Adventuregamers in the last few years.

Also, this a consequence of the stupid tendency of giving the numerical score any importance. It’s just a quick way to say “I liked it, I didn’t like it”. It never should be before the text, to the point of rewriting it to match a score.

That’s not Jackal’s job as editor. He is not dictating scores. He is editing the article to make sure the narrative supports the score the reviewer gave and to make sure the writer is covering important aspects of the game for the reader. He has nothing to do with score choice. I probably didn’t explain it well but what you are thinking is happening is definitely not.

     

Resident science freak

Total Posts: 3

Joined 2004-02-05

PM

Your answer is not making it any better. So the basis for correction is the score? It is ridiculous anyway. Do you first assign a score, then write a review to support it? It should be the other way around!

The score is superfluous. If the text is well written, if the points are well argumented, then the score is irrelevant, it’s just a summary.

Obviously if you see a game that receives praises all over the text and then gets a 1, then I would also suggest the reviewer to correct an obvious mistake. But that’s common sense, not editorial line. But again, this should not be an ORDER, it should be an ADVICE. You only reject a text if it’s badly written, or if it does not prove its points.

These “you gave BS2 a 2 and then you went on to give Runaway a 4.5!! that’s nonsense!!” rants should be automatically answered with “shut up and read the reviews, not the stars”. Scores are just not comparable. They’re just not.

That’s my opinion anyway.

     

Total Posts: 98

Joined 2004-02-16

PM

Perhaps I didn’t explain well enough, either. It wasn’t my intention to make it sound like Jack determines the score, nor that the score is the most important thing, at all.

Jack reads the review, determines what score would logically follow from that, and compares it with the score the reviewer has actually given. If the two are in balance, no problem at all and apart from some typo-fixing or re-ordering of paragraphs so they flow better, no changes are made and the review is published as is. Only if they are miles apart he makes the reviewer explain (to him) why they chose that particular score, and he uses those arguments to make suggestions on how to enhance the review. Jack never says “this game doesn’t deserve 3 stars” but he says “your review doesn’t convince me the game deserves 3 stars”. The reviewer should then either rewrite (parts of) the review to establish more clearly what is so good or bad about the game to warrant the score, or change the score to match the body of the review. He really makes us think hard what score would be fair and why, and how to bring that across to the reader. In the end, it is still the opinion of the reviewer, but far better worded.

     

Madre de Dios, es el Pollo Diablo!

Avatar

Total Posts: 1178

Joined 2012-02-17

PM

kelmer - 09 July 2012 08:25 AM

The score is superfluous. If the text is well written, if the points are well argumented, then the score is irrelevant, it’s just a summary.

And if the score does not match the text, it suddenly becomes a dangerous animal. As the others have explained already, I don’t tell reviewers what to change; I just demand that the text and score match.  Of course that’s an editorial issue. 

I agree with you the the scores (once matched with the text) are essentially irrelevant.  But I assure you, there are many people who place far more value in them than us.  See the comments following our Deponia review for proof just last week.  Or just look up Gamerankings or Metacritic. 

Incidentally, I usually haven’t yet played the games being reviewed.  It’d be kind of hard to impose my own will even if I wanted to.  (And if I did, it’d be the writers screaming bloody murder, not the readers.)

 

     

Editor-in-Chief, some obscure little site called Adventure Gamers

Total Posts: 3

Joined 2004-02-05

PM

It was a misunderstanding, then.

I still believe it would be healthier if it was the whole team commenting on these mistakes/incoherencies between score and review, but I guess that’s just my personal taste.

     
Avatar

Total Posts: 7620

Joined 2011-10-21

PM

* I wrote this before I read these last five posts, so I didn’t consider them.

Oscar -

I think so too. If you remember that reviews are meant to help you decide whether or not you will like the game, scores aren’t too much of a problem. I’m not going to skip over a 2-star review of a game that catches my eye but I’ll read it to see what problems the reviewer had with it, and I’ll read a highly scored game review even if I know it’s something that I’ll probably not enjoy.

^ This.

The reviewer’s rating is just a rating made by someone with different tastes than mine. It’s not gospel, and neither are the user averages (although these deserve a slightly more prominent spot on the site). They are merely an indication.
What AG does VERY well is giving a proper impression of a game if you read the entire review (and not just the rating and the pros/cons). Even the 5-star rated games don’t get an all-praise review, and that’s a big plus.
Often, the dullest reviews are the 5-star and the 1-star reviews because they’re totally biased. AG doesn’t really have that problem, imo, which is why I like these reviews a lot.

Yes, if you look at and compare every single title, you’re bound to find some discrepancies in the ratings, but that’s often because you’re comparing apples and oranges. The real rating is found within the review itself. And even then it’s still just the reviewer’s opinion.
It’s enough to get certain games on, and other games off my radar, but I’ll still prefer to form my own opinion after playing a game…

Kasper F. Nielsen -

Other than that - no matter if I agree with the reviewer or not - I always find the reviews to be interesting reads with justifiable scores.

^ This.

AdventureGamers does a really good job with both the reviews and the ratings. There’s no need to look at the ratings with a magnifying glass. Just take them for what they are, and read the full reviews instead.

     

Now playing: Blade Runner (post-CPT) | The Witcher: Enhance Edition (on hold) | Danganronpa: Trigger Happy Havoc (on hold) | Professor Layton and the Azran Legacy (3DS)
Recently finished: Whispers of a Machine (CPT) - 4/5 | Beneath a Steel Sky (CPT) - 3/5 | 3 in Three - 3.5/5 | Puzzle Gallery: At the Carnival - 2.5/5 | The Fool’s Errand (replay) - 3.5/5 | The Dig (replay) - 4.5/5 | Return of the Obra Dinn (CPT) - 4/5 | Beavis and Butt-Head in Virtual Stupidity - 3.5/5 | League of Light: The Game (CCPT) - 3/5 | realMyst: Masterpiece Edition - 2.5/5 | Contradiction - 3/5 | Tex Murphy: Mean Streets - 2/5 | The Last Express - 3.5/5 | South Park: The Fractured But Whole - 4/5 | Indiana Jones and the Fate of Atlantis (replay, CPT) - 5/5

You are here: HomeForum Home → Other → Site & Forum Feedback → Thread

Welcome to the Adventure Gamers forums!