Usability tests and contextual interviews are at the heart of what we do. Only by objectively watching people use things can we really get a full sense of what will work and why. Analytics is essential too, but only tells half of the story (what people do, but not why). Subjective user feedback through interviews, surveys, and group sessions is also part of our process, but it can be misleading on its own, because people tend to say one thing but do another.
Everyone on the ResIM team gets some kind of user exposure each month, regardless of project and role. We did this comparison as a team exercise to practice observing, empathizing, and questioning our assumptions about what works and what doesn’t.
The task was to find a tent that’s big enough to sleep 4 people and costs less than a specified price.
We set up 3 unmoderated tests for each site (for a total of 6) — not enough for rigorous assessment but enough for some interesting observation and discussion.
Large retailer visited by Canadians 250 million times per year (in store) for “everything for life in Canada.”
Canadian Tire is where you’d go to buy a hockey stick, patio lights, spark plugs and a frying pan on a Saturday afternoon.
Canadian retail co-op that sells outdoor gear, clothes, and services to 4 million members.
MEC is where you’d go to buy reliable climbing ropes, a lightweight camp stove and a 85L backpack for a backcountry expedition.
Canadian Tire listed Camping under “Playing” but not “Outdoor Living"
MEC’s navigation structure listed “Hiking & Camping” under “Gear” and “Activities”
Camping products make up a large proportion of MEC’s catalogue, so tents are naturally prominent. (One tester characterized it as “a camping website.”)
The Canadian Tire site posed more of a challenge. Camping is just one of over 100 categories listed in Canadian Tire’s main navigation. We suspected it could be problematic that tents can be found under Playing but not Outdoor Living. Fortunately the testers were easily able to discover tents on the homepage, even when they weren’t able to find them in navigation.
The first Canadian Tire tester started looking in “Outdoor Living” first, as we suspected someone might. But within 40 seconds she had scrolled down the homepage and found a tent among other camping products. She also looked at related products to try to find the best tent.
The second Canadian Tire tester didn’t even look in the product menus. She started scrolling down the homepage and saw camping products near the bottom. She was able to find a tent in less than 20 seconds. Within another 30 seconds she had looked at multiple tents using the related products feature and confirmed she had found one that met all the criteria.
The third Canadian Tire tester was the only one out of the three to successfully use the main navigation — possibly because she methodically looked at every menu from left to right, so she found Camping under Playing before even seeming to notice Outdoor Living. Like the others, she had no problems finding the right tent once she found the category.
Canadian Tire’s product names were clear about how many people each tent sleeps
MEC’s product listings are less explicit
The first MEC tester clicked on something called a “Big Agnes Tumble 2 mtnGLO Tent.” He inferred the tent’s 2-person capacity from a line in the description about “double doors and vestibules.” He wondered about the meaning of the numbers in product names and eventually guessed that “MEC Camper 4 Tent” referred to 4-person capacity.
After clicking a random product (which happened to be a gear shed) to see how many people it slept, the second MEC tester determined that “it doesn’t say.” He didn’t notice the numbers in the product names so he didn’t figure out what they meant until going back to the catalogue.
The third MEC tester correctly guessed what the numbers meant within a few seconds and confirmed by clicking through to product details.
As it turned out, only one of the MEC testers initially guessed that the numbers in the product names referred to sleeping capacity (though all three guessed it eventually). This little bit of ambiguity may or may not be a real problem for MEC — which presumably has more gear-savvy customers, on average. It’s difficult to judge an organization’s design decisions without knowing all the factors that went into them.
Canadian Tire’s site went out of the way to put sleeping capacity front and center
There was no filter for tent capacity on MEC’s list of tents and shelters
There was at least one other hitch on the MEC site — the type that often goes overlooked without usability testing or evaluating the site from a user perspective. (Note that MEC has made changes to improve this since we did our tests.)
The first MEC tester quickly navigated to the list of “tents and shelters.” He noticed it contained 477 products so he clicked the “tents” subcategory in the sidebar to narrow it down. A filter for sleeping capacity appeared but he sorted the list from highest to lowest and started scrolling to find tents under $400. He said he liked the filter options but didn’t seem to notice sleeping capacity, which only appeared after he had already started narrowing down the list.
The second MEC tester clicked on “tents and shelters” and immediately looked for a filter for sleeping capacity, which wasn’t available. He decided to filter by price instead, and settled on a list of products between $100-$500, which included accessories and other non-tents. He arbitrarily clicked on a gear shed, which has no sleeping capacity, which caused frustration before eventually finding a suitable tent.
The third MEC tester skipped “tents and shelters” and went directly to a list of tents via site search (which we asked participants not to use). Unlike the first two MEC tests, a filter for sleeping capacity appeared near the top of the sidebar. He quickly clicked on “4-person” followed by the price filter to find tents matching the criteria.