View Single Post
Old 08-21-2007, 10:15 AM   #2
Squinky
Unreliable Narrator
 
Squinky's Avatar
 
Join Date: Apr 2004
Location: Le Canada
Posts: 9,873
Send a message via AIM to Squinky Send a message via MSN to Squinky
Default

Squee! I'm actually planning on making an AA article in the future about the testing process. I worked in QA a couple of years ago, so I find this subject to be of great importance and personal interest.

So far, I've only allowed one person to play Chivalry in its unfinished state. I did this more to give myself personal motivation than anything else; however, the comments I got back were very helpful in terms of what was enjoyable and what was frustrating, and even inspired me to change/add a couple of things. Back when I was testing TGTTPOACS, I installed a message board to enable my testers to report suggestions as well as bugs, and I was able to decide democratically which suggestions to implement and which ones to leave as-is. Most notably, I added a handful of extra dialogue lines to acknowledge actions that certain people attempted but only received a default response for, and added functionality to allow characters to move more quickly via double-clicking.

I've never thought of using raw data before; I don't think it should be too difficult to create a plugin in one of the major adventure game engines that logs a transcript of sorts detailing a player's actions. (Maybe I'll create one if I'm not too busy/lazy, but somehow I find that doubtful; finishing the game itself is my top priority.) This is, I believe, how IF is tested. Another option would be to ask testers to record themselves playing through Fraps or something, though this would likely be a tad more time-consuming depending on the number of testers one uses.

Still, I have to nitpick about the fact that only measuring raw data isn't really enough in my view. When people play my games, what matters most to me is what they thought and felt about the narrative and the themes derived therefrom, and whether the gameplay helped or hindered the expression of said themes. I think that this can only be measured qualitatively rather than quantitatively, by actually asking playtesters about their impressions. Hence, if I only have time to employ one method, this is definitely the one I'd choose.
__________________
Squinky is always right, but only for certain values of "always" and "right".
Squinky is offline