Usability Showdown

Canadian TirevsMEC

Usability tests and contextual interviews are at the heart of what we do. Only by objectively watching people use things can we really get a full sense of what will work and why. Analytics is essential too, but only tells half of the story (what people do, but not why). Subjective user feedback through interviews, surveys, and group sessions is also part of our process, but it can be misleading on its own, because people tend to say one thing but do another.

Background

Everyone on the ResIM team gets some kind of user exposure each month, regardless of project and role. We did this comparison as a team exercise to practice observing, empathizing, and questioning our assumptions about what works and what doesn’t.

About the Test

The task was to find a tent that’s big enough to sleep 4 people and costs less than a specified price.

We set up 3 unmoderated tests for each site (for a total of 6) — not enough for rigorous assessment but enough for some interesting observation and discussion.

The sites were canadiantire.ca and mec.ca. Both have large selections of tents and some interesting differences in terms of user experience.

Side by Side

Canadian Tire

Large retailer visited by Canadians 250 million times per year (in store) for “everything for life in Canada.”

Canadian Tire is where you’d go to buy a hockey stick, patio lights, spark plugs and a frying pan on a Saturday afternoon.

MEC

Canadian retail co-op that sells outdoor gear, clothes, and services to 4 million members.

MEC is where you’d go to buy reliable climbing ropes, a lightweight camp stove and a 85L backpack for a backcountry expedition.

Making Products Both Findable & Discoverable

Canadian Tire listed Camping under “Playing” but not “Outdoor Living"

MEC’s navigation structure listed “Hiking & Camping” under “Gear” and “Activities”

Camping products make up a large proportion of MEC’s catalogue, so tents are naturally prominent. (One tester characterized it as “a camping website.”)

The Canadian Tire site posed more of a challenge. Camping is just one of over 100 categories listed in Canadian Tire’s main navigation. We suspected it could be problematic that tents can be found under Playing but not Outdoor Living. Fortunately the testers were easily able to discover tents on the homepage, even when they weren’t able to find them in navigation.

1st tester

The first Canadian Tire tester started looking in “Outdoor Living” first, as we suspected someone might. But within 40 seconds she had scrolled down the homepage and found a tent among other camping products. She also looked at related products to try to find the best tent.

2nd tester

The second Canadian Tire tester didn’t even look in the product menus. She started scrolling down the homepage and saw camping products near the bottom. She was able to find a tent in less than 20 seconds. Within another 30 seconds she had looked at multiple tents using the related products feature and confirmed she had found one that met all the criteria.

3rd tester

The third Canadian Tire tester was the only one out of the three to successfully use the main navigation — possibly because she methodically looked at every menu from left to right, so she found Camping under Playing before even seeming to notice Outdoor Living. Like the others, she had no problems finding the right tent once she found the category.

Prioritizing Key Product Attributes

Canadian Tire’s product names were clear about how many people each tent sleeps

MEC’s product listings are less explicit

1st tester

The first MEC tester clicked on something called a “Big Agnes Tumble 2 mtnGLO Tent.” He inferred the tent’s 2-person capacity from a line in the description about “double doors and vestibules.” He wondered about the meaning of the numbers in product names and eventually guessed that “MEC Camper 4 Tent” referred to 4-person capacity.

2nd tester

After clicking a random product (which happened to be a gear shed) to see how many people it slept, the second MEC tester determined that “it doesn’t say.” He didn’t notice the numbers in the product names so he didn’t figure out what they meant until going back to the catalogue.

3rd tester

The third MEC tester correctly guessed what the numbers meant within a few seconds and confirmed by clicking through to product details.

As it turned out, only one of the MEC testers initially guessed that the numbers in the product names referred to sleeping capacity (though all three guessed it eventually). This little bit of ambiguity may or may not be a real problem for MEC — which presumably has more gear-savvy customers, on average. It’s difficult to judge an organization’s design decisions without knowing all the factors that went into them.

Contextualizing Search Filters

Canadian Tire’s site went out of the way to put sleeping capacity front and center

There was no filter for tent capacity on MEC’s list of tents and shelters

There was at least one other hitch on the MEC site — the type that often goes overlooked without usability testing or evaluating the site from a user perspective. (Note that MEC has made changes to improve this since we did our tests.)

1st tester

The first MEC tester quickly navigated to the list of “tents and shelters.” He noticed it contained 477 products so he clicked the “tents” subcategory in the sidebar to narrow it down. A filter for sleeping capacity appeared but he sorted the list from highest to lowest and started scrolling to find tents under $400. He said he liked the filter options but didn’t seem to notice sleeping capacity, which only appeared after he had already started narrowing down the list.

2nd tester

The second MEC tester clicked on “tents and shelters” and immediately looked for a filter for sleeping capacity, which wasn’t available. He decided to filter by price instead, and settled on a list of products between $100-$500, which included accessories and other non-tents. He arbitrarily clicked on a gear shed, which has no sleeping capacity, which caused frustration before eventually finding a suitable tent.

3rd tester

The third MEC tester skipped “tents and shelters” and went directly to a list of tents via site search (which we asked participants not to use). Unlike the first two MEC tests, a filter for sleeping capacity appeared near the top of the sidebar. He quickly clicked on “4-person” followed by the price filter to find tents matching the criteria.

It isn’t entirely fair to judge design decisions when we don’t know what went into them. There isn’t really a universal answer. It ultimately comes down to whether people can find what they need, where and when they expect it. Figuring that out is why ongoing user exposure and testing is so valuable.

Interested in gaining practical, customer-centric knowledge on how to improve your website or application? We can help with usability testing.