How to get the most out of usability testing
Usability testing is a pivotal part of UX design. To make sure you get value from usability tests there are certain things you need to do, regardless of the type of test.
What insights do you want?
Even before you decide on a specific method you need to clarify exactly what you want to learn from the tests. Write down what think you know, what you’d like to know, and what you’re not sure about. Having a high-level outline helps keep testing productive when the inevitable surprises happen.
Who should be involved?
Team members & stakeholders
Every Res.im team member watches usability tests, whether live or recorded. It’s also important for the client or stakeholders to see real users interacting with a site or product. The consistency, actionability, and memorability of insight from first-hand exposure to real users is impossible to achieve by second hand.
While adding observers to live tests is often desireable, it’s best to only have one or two people in the room with the participant: one moderator and maybe one co-moderator or scribe. More than that puts participants in an unrealistic situation with too much pressure. If you don’t have a facility with a two-way mirror, other people can watch remotely using a screen-sharing tool like GoToMeeting, whether they’re in the next room or another city.
Moderated tests tend to run more smoothly when a skilled moderator leads. A co-moderator is often good to have, especially if that person is the designer or key decision maker, but the principle moderator should be solely responsible for steering the session.
The moderator doesn’t need to be a full-time researcher or usability tester, but they need to have the right soft skills. A good moderator puts participants at ease and makes the session feel natural without it actually being natural. More importantly, skilled moderators are able to mitigate biases in the how questions and tasks are framed. This can be especially difficult when sessions go in unexpected directions and you need to come up with questions and prompts on the fly.
Which method(s) should you use?
At Res.im we do several types of usability tests, depending on the what we’re testing, who the users are, and what sorts of insights we want. Each method is enlightening (and challenging and fun) in its own way.
Moderated, in-person usability testing is perhaps the most natural method. At its simplest, moderated usability testing means asking someone to use what you’ve made, then watching them. Moderated testing gives you opportunities to ask the user why they did something a certain way, what else they might have expected, etc. It helps you discover things you might not have thought to ask about or look for before the test started.
We often invite participants to test in our own office or at our client’s site. But if circumstances allow, we may also go out into the field to gain contextual insights about the user’s own environment and how a digital experience relates (or doesn’t) with offline experience.
When time and budget don’t permit us to meet the participants in person, we conduct moderated usability tests remotely using a screen-sharing tool like GoToMeeting. Remote testing can be slightly more challenging to moderate than in-person but easier to schedule if you’re targeting a unique group. Since remote tests let users to use their own setups, they’ve occasionally given us valuable insights for user research.
We’ve also conducted remote moderated testing with a group of participants simultaneously, which is challenging, but Res.im believes in experimentation. This approach is an efficient way to get rapid feedback from a geographically distributed group within a short time frame to supplement other research and testing.
Remote unmoderated (or self-moderated) testing tends to be the quickest but shallowest way to get feedback on a product or site. Remote tests can be easy to set up, but unmoderated tests pose the highest risk of going off course, especially if questions or tasks are underdeveloped or poorly written.
Because unmoderated testing is often done by anonymous participants (many of whom frequently test sites on a regular basis and are familiar with common usability issues), they usually don’t provide insight into your “typical” users’ needs.
On the positive side, unmoderated tests have the benefit of taking the evaluator’s personality and other interpersonal variables out of the equation. That makes them useful for more quantitative assessments, like comparing the average task time.
Because remote unmoderated tests can be done fairly fast and cost-effectively, they can be used to test very specific aspects of user experience like where people click first on an interface (aka first click testing) or how they navigate a menu tree (aka tree testing).
What tools should you use?
In theory you don’t need a lot of tools to do moderated usability testing. You could walk up to someone, hand them your phone and ask them to try your app. But in practice it’s too easy to lie to yourself when you do that. There’s no record of the test, no way for you or anyone else to refer back to it later to verify how you thought it went.
Recording and documenting in detail is important because we have some totally natural and difficult-to-avoid tendencies to notice and remember what we want to, especially when it comes to feedback about ourselves and our work. It’s virtually impossible for even the most experienced researcher to remember things objectively.
There are a lot of tools to choose from. Some do one thing well and others are suites that go help you recruit participants, automate testing, and even analyze and present the results. Occasionally we use a couple of tools in combination. We don’t always use the same usability testing and recording tools but here are some we’ve used:
The bottom line there are a lot of different ways to test the usability of your product, depending on your goals, budget, timeline, target users, and stage of work. The most important things to worry about are your goals and types of insights you want. From there, experience will help fit the right type of test to your situation.