Quality Care for Quality Assurance

When I first started work at Paragon, Joe "Hero 1" Morrisey taught me a valuable lesson about people who worked in Quality Assurance, QA. He said that he has seen many people in the industry abuse people in QA and treat them as if they weren't human. Joe advised me to always treat people in QA with respect and dignity, and to always remember there's a person behind that bug report that I received. This is a lesson I've taken to heart, and something that I try to live through my design work.

A lot of stories from Penny Arcade's The Trenches shock me, wondering how anyone could treat people in QA like that. There is a tendency for people to view QA as a "lesser" role than that of design, art, or programming. This is something that I never understood, because when you really get down to the heart of it, the role of QA to is to make sure that you don't look like an idiot. I can safely say that much of my work that went live was only as good as it was because of people in QA running it and finding things that I never would have thought could go wrong, even if I did think very hard about it (and I did!). People in QA are the watchful guardians (queue Batman music) who make sure that the collaborative efforts of design, art, and programming all actually work through a myriad of test cases. If you ask me, this sounds like a pretty damn important task to me, which is why I try my best to always be respectful to QA.

I say try my best because sometimes as a designer, you can forget that, and here's why. Your work (and I speak for myself in this) is like your child. You're proud of it, you think it's great, it's the best kid in the class. The people in QA are the teachers who tell you that your kid isn't the A student you thought he was, that he's getting C's in Math and D's in English and that you really ought to see what's up with him crashing when he talks to people. In short, they're the ones telling you that you didn't do something right, and that can be aggravating or downright insulting to see someone telling you that you're not as good as you thought you were.

That being said, it's also something that, once you realize it, you can get over it and appreciate the feedback. It's great to get those bugs in from QA, because they're the ones who found it, not the players! A showstopper bug where you can't progress through a level is much better to find testing rather than in a live environment. If done right, you start to appreciate all the bugs of yours that QA finds, because it's another bug that didn't make it to live.

However, there are things that designers should do to help QA out to make their lives easier; if QA's lives are easier, it means more bugs are found, more feedback can be given, and the product as a whole comes out more polished. The first thing is to test your content before it goes to QA. Nothing is more wasteful than sending QA something you haven't tested and making them waste their time typing out bug reports for something that you could fix in two minutes. It's fine if you miss something while you're testing, everyone does, but if you just haven't plain walked through it, then you have no business sending it to anyone else.

Second, and this is more from a technical angle, provide QA with the tools to properly debug missions in a quick and efficient manner. This not only helps QA test things but also helps designers test their own work faster. On City of Heroes, I wanted to check for souvenir clues to see if a player did or didn't complete a story arc. The only tool we had at the time to grant a souvenir clue was to complete the final mission of the arc that gave it away, and to remove it you had to reset all of your mission info. In a nutshell, it was a giant pain. I talked with Tim "Black Scorpion" Sweeney about it, and he came up with two quick commands: scadd souvenir_name and scremove souvenir_name. Now, when I wanted to test things based off of souvenirs, it was a quick 10 second command that I or QA could run. As long as I put it in the outline of the mission, QA would have an easier time.

This leads me to my third point, which is writing mission outlines. If you want your mission to shine, then QA needs to understand how it's supposed work, inside and out. They should know all of the surprises, twists, and turns you put in there, so that way they can hammer on it and figure out all the ways it could go wrong. (Fun fact about City of Heroes: masterminds and stalkers broke EVERYTHING.) If your QA team is flying blind in your mission, then you can bet they're going to miss something, or that it's going to take them longer to figure out how things work. The longer QA takes to look at individual things, the higher your risk of something slipping through the cracks.

These are just some of the things I've learned and try to live by when interacting with folks in QA. Andy Maurer, a great programmer from Paragon, often said that being a good tester is a talent, one that's valuable, and I couldn't agree more. The ability for a person to think of a myriad of ways that something could go wrong and efficiently test it is a skill that I feel is not valued nearly enough in the games industry, as shown by the various tales from the Trenches. People in QA are there solely to help everyone in the team look better, and that's a fact. Every great game made has a great team behind it, and that doubly includes the QA department that more often than not gets blamed rather than praise, so take the time to appreciate them and give them the credit they deserve.