[Fien]

Every time I play an adventure which has me running around like the proverbial chicken without a transportation map or a game with pointless and endless conversations, I wonder if developers listen to gamers and more specifically to beta testers.

Or maybe I should ask first if beta/alpha testers give developers their honest opinion of the game. I've never tested games and don't want to either, I know I would not enjoy it. But the few times I was invited to criticise a game before it was released, when changes could still be implemented, I found it much harder than I had expected. I didn't want to hurt the developers' feelings, so I was as tactful as I could (yes, really!) and stressed the strengths of the game.

I know there must be many beta testers here at the AG forums. Do you offer (constructive) criticism? Do developers follow up on suggestions? My question is not about specific games, I'm not asking people to betray confidence.

[Jelena]

I've been a beta tester twice (and I enjoyed it a lot I might add).
The first time I could tell that the developers had listened to the test group. We had an area in the game forums where we could discuss our impressions and opinions on what could be done in a different way.
Later on when playing the game I noticed some changes to the better that had been made after our opinions.
It was very interesting and gratifying to be a part of the final product.

[Jackal]

This may seem like semantics, but it's an important distinction. It's not actually beta testers you're asking about, but playtesters. The (apparent) lack of playtesting in adventures has been a pet peeve of mine for years.

Beta testers (though no doubt contributing thoughts on the quality of the play experience) are meant to be testing a finished game for functional issues. I think virtually all games get beta tested, and testers are probably listened to as much as possible. Problem is, by the time a game is in beta, it's too late to change much of the game itself, just fix bugs and such.

I'm totally convinced that most developers don't do any playtesting at all, or if they do, they tend to cherry pick people most inclined to like their work as is. Those people likely give a totally honest opinion, just not a particularly constructive one. Alternatively, the developers aren't asking their testers the right questions. Ask ten people their subjective gameplay experiences and get ten different answers, which lands you right back where you started. What testing should do is identify bottlenecks and trends and unforeseen gameplay problems. If everyone's getting stuck or bored or frustrated at the same place, even if their opinions of the game are totally different, that's something that will benefit the game for everyone.

The few companies that pay close attention to playtesting (best done in-house, since you can actually observe the player experience) seem to consistently get their games right. Those that don't... not so much.

[tabacco]

We make changes based on playtest feedback all the time!

[mosdl]

I'd say that the internal testers at game companies mainly look for bugs and ways to break the game, which is what QA people usually do.

Where I work (not a game company mind you), we do get people in to test our products and film them trying to perform tasks and all for about 2 hours each. With a game, 2 hours isn't going to do it.

It is really up to the designers of the game to make sure the game is fun to play (transportation map, good UI) in the early phases of development. Because as said above, once you hit beta, that is just bug fixing, no new features are usually allowed.

[deepdish]

I totally feel you on this.

There's been instances in some Adventure games that I've been playing where I think to myself: "Really? Am I really expected to do (insert imaginatively inane task)?"

And then, fed up, I'll look at the walk-through and think to myself: "Really? I was expected to figure that out?"

Spoiler:

And how about Evidence: The Last Ritual? The whole bit about having to go to Google Maps and find the gas station you saw in the video, then align the outline of the city on the map with the shape in the puzzle. Really.

[Jelena]

Originally Posted by mosdl

Because as said above, once you hit beta, that is just bug fixing, no new features are usually allowed.

Interesting.
When I was beta testing/did quality assurance the other testers and I had an opinion on when to pick up an inventory item. In the version we tested we couldn't pick up that specific item until it was intended to be used. That we found annoying since we had to run upstairs and downstairs several times in order to check whether it was time to pick it up. In the final version one could pick it up right away.

[tabacco]

Definitions of terms like 'alpha', 'beta', etc. vary a little bit from developer to developer. Usually, 'beta' implies a product is feature complete, and the only changes from then on are fixes. Stuff like an object needing to be picked up earlier is a pretty minor change, so that's why it was made at that point. Now, if someone had suggested a small story rewrite, that probably wouldn't have happened once a product reached beta.

Developers' goals for external testing are often pretty different too. Sometimes developers are just looking for help with QA, to find bugs. Other times they're genuinely looking for feedback on the gameplay or the story or what have you.

[Fien]

Thanks for the enlightening responses.

[PolloDiablo]

I've beta tested about a dozen adventure games, some freeware and some commercial (but always from small, less than five people teams). It varies from developer to developer what kind of feedback they like.

In my experience most of them are open to suggestions from beta testers. They can't always implement everything we suggest, even when they agree they would be a great improvement (for instance because there isn't enough time until the publisher absolutely has to have the final gold master, or because of technical limitations).

However, during beta testing one often plays the same chapter or solves the same puzzle over and over again, or deliberately tries to do something the player should not be doing (solve puzzles out of their proper order, visit locations you shouldn't be aware exist, flush quest-items through the toilet etc). This automatically means you are not experiencing the game as a 'normal' player would and hence you have a different opinion about difficulty, length, irritation factor etc. Also, you often get cheats to bypass certain chapters or a walkthrough to get you to that specific bit you should be testing.

I agree with Jackal that there should be more developers asking for feedback from playtesters. I often try to finish a game I should be beta testing once, before starting the proper test, but there's not always time for that. And sometimes we are only given one chapter at a time, then when that's been tested for a few weeks we get the next. After a while, it's very hard to have a complete picture of the entire game.

[Fien]

I was hoping you would respond, PolloDiablo. You also have experience playing games on site, in an earlier stage of development, right? So do developers often implement your suggestions?

[Martin Gantefoehr]

It's a difficult issue. Feedback and re-implemenation cycles require time, money, and a pretty high level of professionalism (and thick skin) on the part of both the developers AND the playtesters.

In the early stages of development, a game is often a terrible mess, with myriads of placeholder elements, and a lot of features missing or half-working. Many important aspects (game flow, look & feel, consistency, continuity, dramatic power, dialogue flow, difficulty) are quite difficult to test until all the necessary elements come together in a near-final version. Before that stage, testers AND developers will need to fill many voids with their imagination. (And interestingly, when imagining something, it often looks and works much better than the real thing ).

Once a game HAS reached beta stage, and all elements are there, the picture becomes clearer. But at the same time, the game has become by several orders of magnitude more complex and vulnerable to changes. The assets have been produced, the money has been spent, the time has been used up. The team is probably exhausted, and with the release deadline approaching, they may now also be, well, unreceptive to fundamental redesign requests.

This is why beta testing often focuses on eliminating bugs, tweaking user friendliness, and adjusting difficulty. Feedback and change requests regarding fundamental design decisions need to be brought in long before Beta. It is simply not a good idea to introduce brand new engine features, interface elements, characters and subplots while in Beta. This will VERY likely result in A LOT of unforeseen trouble (and work), and ultimately push back the release.

[PolloDiablo]

Originally Posted by Fienepien

I was hoping you would respond, PolloDiablo. You also have experience playing games on site, in an earlier stage of development, right? So do developers often implement your suggestions?

Yes, I am currently involved with a project for which I have visited the developers on site. The game was then in alpha stage and they watched me play, asking me why I did or didn't do certain things, observing my choices and comments all the time. After an hour or two they left me to play on my own, but since it was alpha stage I kept running into bugs that prevented me from progressing much. The game is now in beta and I am involved again. It must be said these people are not game developers in the traditional sense, nor is their game an adventure in the traditional sense. It is however obvious that they paid close attention to what the alpha testers said and have implemented many of these things. Some of our suggestions did not correspond with their creative ideas and that's okay ofcourse.

It's hard (impossible) to say how often my suggestions are implemented because it's not always clear whose idea something is. Some developers set up a forum on which beta testers can discuss their findings and comments. Something someone says can trigger an idea in someone else, or perhaps the developer had already been playing with the idea and sees the wish for it to be implemented confirmed by the testers. Other developers keep the testers strictly separated and receive all feedback via email or a private wiki, so no-one knows what others may already have reported. It's also a matter of how radical a change would be. What sounds like a simple tweak to me may involve weeks of scripting for the developers... but I'm generally happy with my influence on the final results.

[Fantasysci5]

tobacco-I love your pic. SAM!!! lol Anyway, ehm.

I've wondered this, too. Sometimes gameplay leaves me wondering if anyone played it through and gave suggestions. I can't really talk, though, since I've never beat-tested or written scripts (but I want to! ) lol