Reviews are awfully subjective and badly analyzed. Rating system is faulty as well.
 

I was browsing through some, and couldnt help but
write something, because I like the news portion of this site and
you guys seem to devote a lot of time also into the reviews.
So here are a few advices:

The rating system is non-functional because
there seems to be a lot of staff, so ones 3,5 will be another ones 4.5,
this makes comparsons betweens games futile.

With poorly analyzed, id like to take as an example the
Broken Sword review. The reviewer is so happy due to the
animated "juicy, action packed" ending, that he completely forgets to
judge it properly. (rip-off from raiders, overall shallow and more cliches than 3 B-movies combined)

Not to mention the perfect score for something thats far, far away from a flawless adventure. The only really thoughtful thing in BS, was the dialogue, which is really up there. The rest was and is highly debatable.

Plus, I HIGHLY doubt that everyone who has played their fair share of adventures (read knows what he's talking about..) would rate Broken Sword anywhere near Sam & Max or Indy 4, let alone above them.

As for another example of a 5 of 5 game, Longest Journey:
This is so loaded with hype, some might overlook the most annoying protagonist ever, the "tough teenage feminist, who takes not crap from nobody, always with a tedious witty oneliner on the short line".
I deeply hated her by the end of the game. The Story while nicely executed was done do death prior. Magic vs. Technology, with a LOT of symbolic trite.

LJ has a lot of fanboys on the net, and maybe for some this is indeed the better adventure in recent time (I found it average at best) but Broken Sword is badly misjudged.

;) Now aren't you being a bit subjective?

Precisely! :P

of course reviews are subjective! lol, the reviewers tell u what they like and displike about the game. it depends on whether you trust their judgement or not. no review is truly objective. this is especially true for things like video games which require the gamer's tastes and their experience with past games.

every game site i know of have varying opinions on the same games. it's all because the reviewers have their own ideas if the game they're playing is good or not. this is why we should not judge one's opnion of something on one review. :)

Well, I have played my fair share of adventures, as well as your fair share and your mother's fair share also, and I'm telling you Broken Sword is one of the greatest adventures of all-time and I completely stand behind the five-star rating. The TLJ review probably could be toned down to a 4.5/5, but every publication readily admits sometimes they get carried away with original reviews of games. PC Gamer gave Alpha Centauri a 98% when it first came out. They gave Black & White and Neverwinter Nights each a 95%. Think they don't regret those scores now? Doesn't make their rating system faulty.

The VAST, and I mean VAST majority, of experienced, credible adventure game fans would sincerely disagree with your analysis of both of these adventures. So maybe you should gauge sentiment a bit before making such wild, generalizing statements. As far as the rating system being "non-functional" because different people review games...if you wouldn't mind sending me a link to a publication where one person reviews every single adventure game that comes out, I'd love it. But each reviewer understands our criteria and attempts to find a score that fits in our scale well. A 4.5 from me means the same as a 4.5 from Robert. Now, that doesn't mean Robert would have given 4.5 to Runaway as I did, but it sure doesn't make anything "non-functional" other than your ability to constructively criticize us.

I always welcome criticism. In fact, we're in very serious discussions right now on methods to improve our whole rating system. But seriously, don't come in here and say "Broken Sword has more cliches than 3 B-movies combined" and "TLJ has the most annoying protagonist ever" and accuse us of being subjective.

I'm glad you enjoy the news portion of our site, and I hope you continue reading. Thanks.

Aww, that's sweet.

Oops, Evan said more or less what I was in the middle of writing and editing. Lame.

Evan is right! (though perhaps a bit full of himself :shifty: )

YOU'VE BEEN A BAD BAD BOY JOHAN! You didn't take out the trash! Now mommy's gingerbread is going to SMELL LIKE POO POO! BAD BAD JOHAN!

*The TLJ review probably could be toned down to a 4.5/5, but every publication readily admits sometimes they get carried away with original reviews of games.*

What is it with the perfect score of some games anyway?
Thats absurd. What if a game in the future is better than the godly Broken Sword? (low and behold...) You gonna paint-spray half a star on the 5?

*if you wouldn't mind sending me a link to a publication where one person reviews every single adventure game that comes out, I'd love it. But each reviewer understands our criteria and attempts to find a score that fits in our scale well. A 4.5 from me means the same as a 4.5 from Robert. Now, that doesn't mean Robert would have given 4.5 to Runaway as I did, but it sure doesn't make anything "non-functional" other than your ability to constructively criticize us.*

Puplications, before giving a final score, discuss in a roundtable their reviews and come to a shared rating. Thats not the best way to do it, but its better than to just go ahead and say "Robert maybe would have given it something else - but, no its not non functional"

Furthermore the most accurate way to review is by one single person who has got taste. I also run a review site (though in another field)
for four years and ive done every single one alone. With a rating system that goes from 1-10 in half increments and theorethically stops at 9.

Being overly subjective means have your view clouded from facts, somehting that reviews life from. There will and should always be a certain degreee of personal influence included, but it should be counterbalanced with the above.

So what you're saying is that no game should ever get a perfect score? In that case, why bother even rating games at all? It would be pointless to have a score you never gave out just as some kind of stupid statement.

Oh thats a brilliant reply. The perfect game? Made by God, published by Allah or what?

Objectivity is nearly impossible.

As we can very well see from this thread.

I think we need to be realistic McKracken, adventuregamers has a 5 star system with half stars. That allows only 11 gradings of games. You can't just arbitrarily limit it to 10 grades and say the 11th is just there for a game that may never be made. The star system doesn't mean the game is perfect but it does means it is better than another game and 5/5 makes it one of the best games available. I think part of the problem with the Ags system is not that BS got a 5/5 but that the database doesn't include some of the other games that should also have that score yet, which makes it harder for people to compare. Also if a better game does come along, we have to be objective and say lets judge these games when they were made. otherwise we'd have to constantly deflate the grades as time goes along.

And you said earlier that Broken Sword is not up to the standards of Indy 4 or Sam&Max and that most gamers would agree. Again thats highly subjective and I for one would definately not agree. For example I personally find the humour in both of your examples borders on puerile at times and are aquired tastes - equally subjective. Of course I'm sure people could say similar negative things about Broken Sword too, and so we end in a big arguement about how do you reflect everyones opinions at once. In the end Adventuregamers, Justadventure etc have to make a judgement call and say to themselves "what did I think of the game" and then "how similar is my opinion/taste/experience to the majority of other adventure gamers?" before fixing a star.

There are definately ways it could be improved. Like I said before expanding the review database with a few mini reviews perhaps, for games unlikely to make it into FlashBack friday anytime soon, could help people understand how the scores relate to their own opinions of games. A facility for mini "second opinions" to be added by other staff members whenever they like after a review, to gradually increase the range of views offered per game. Also I think a readers score system might be a good idea, to give people an idea what the general community thought. The figures from such a system could also be used to show how close an individual reviewers scores are to those of the readers, either so the readers know or just so the reviewer knows.

There does seem to be two solid ways people score: The "school grading" scale or a pure 0-10 scale. The school grading scale is done by virutally all the major sites. Any game lower than a 7 is poor (a failing grade in school) and shouldn't be touched. Others grade on a pure 0-10 scale where most games should either be scored a 4,5,or 6 and only a handful of titles get a 7-10. Not a lot of people use this scale, but when it is used, it riles up a lot of people on why the flavor of the month was scored a 6, so I guess that's why it isn't used too much.

--BPF

**The star system doesn't mean the game is perfect but it does means it is better than another game and 5/5 makes it one of the best games available. **

Confusing and contradicting. Youve got 5 stars, you give 5 stars. There is nothing left for improvement. You can look at that from whatever angle you want, its absurd.

**And you said earlier that Broken Sword is not up to the standards of Indy 4 or Sam&Max and that most gamers would agree. Again thats highly subjective**

It is not subjective. We know from professional reviews (print) that these two games averaged to be rated higher than Broken Sword.

**make a judgement call and say to themselves "what did I think of the game" and then "how similar is my opinion/taste/experience to the majority of other adventure gamers?" before fixing a star.**

Youre implying that a reviewer should also include the likelyness what the anonymous masses thought of game XY, ino his final rating? Disgusting.

**I think a readers score system might be a good idea, to give people an idea what the general community thought.**

The "community" is the lowest common denominator, which translates to fanboys and imbeciles. (harsh but true) Give them a voice and watch the nerds flock together in a truely gruesome fashion.

Everybody's a critic - this was never truer than it is today.

It does work. If it would make you happier, you might think of AG:s scoring system as a 0-6 scale where no game ever gets over 5.

Are you trying to be funny? Of course that is your *subjective* view. Broken Sword is for me a clearly superior game to S&M and slightly above Indy 4. But since you think that *your* opinion is the "objective" view I don't think I can convince you that your view is highly subjective too.

A reviewer should include as little as possible of what anybody think including the reviewer. It should convey what type of game the reviewed game is to help the reader select what games they like. You also quite forget that the score is just a small part of a review, you should read the whole reviews also. Btw, I think the best rating system is no rating system at all since that forces the reader to actually read the review and not just complain about the rating at the end of it.

Yes, but the reader score was added as a suggestion as an *addition* to the reviewers score, not as a replacement.

Its because youve got bad taste - it happens.
Again, looking at professional reviews, both games averaged higher than Broken Sword.

You might have also enjoyed Titanic the motion picture.
But that doesnt take away from the fact that a serious critic would decorate its toilet seat with the script.

Oh and pardon me for not taking your 0-6 imaginary friend, very serious - at all. ;)

*ROFL*. You are trying to be funny.

Everyone that doesn't agree with your taste has bad taste? Good one.
No, Titanic isn't my favourite movie (even if it wasn't that bad). You seem to have a rather strange view of what a reviewers job is. A reviewers job is to convey what a movie/game/book/music is and try to tell if the reader will like it or not. The reviewer is there for the readers. That so called "serious" critic of yours wouldn't do his job proper if he did what you imply.

I think we're missing one important point here: the purpose of the stars. I think the stars are only convenient for a quick look at the overall quality of the game.

If game A gets 5 stars and game B gets 5 stars too, then of course it's possible that game B is slightly better, which you can't see back in the rating. But who judges a game by only looking at a few stars? If game B is better than game A then you can read that back in the review.

What I'm saying is that I consider these stars as just a quick indicator, a rough pointer. If a game scored 5 stars it doesn't mean other games in the future can't be better. A better game would score 5 stars too, and looking at it's score you'll be able to tell that's a great game.

--Erwin

Just to make you observe this: It was you who invented it in the first place, you just called it 1-10, where you can't get higher than 9. I hope you don't take your own scale seriously either, then.