Today we are talking with Shiloh Barnat, vice president of interaction design, about usability testing.
Usability testing is often incorrectly considered a “nice to have” component rather than a critical one in web development. What advice would you give to someone trying to convince stakeholders that usability testing is essential?
Stakeholders need to understand that usability testing is the best way to know if your site really works well for the people who will actually be using it. Any site can have fast servers, clean code and pretty pictures, but the really good ones stand out because they offer a good experience. And if your site sucks, people just go elsewhere.
Besides, testing with real users replaces guessing and politics with solid metrics and evidence so you can make smarter decisions.
And if your stakeholder still isn’t convinced, remind them improvements made to customer experience as a result of usability testing feed the bottom line. Improved success rates and reduced time-on-task lead to higher conversion rates, more sales and increased profit from the site … not to mention more satisfaction, which makes loyalty more likely, which pays for the budget to do usability testing.
If an organization doesn’t already have a user experience (UX) practice, what’s the best way to get started?
Get a hero. Put somebody who is passionate about user experience in charge of it and make sure they have the time, budget, training and permission to lead the way. You’ve got somebody in charge of most of the other pieces — IT, marketing, accounting, production, fulfillment, service. Good customer experience means all those pieces have to work together, but that probably won’t happen unless it’s somebody’s job to make it happen. And if you don’t happen to have a UX hero on hand, Lokion is here to help. We’ve mentored several large companies through solidifying their UX practices. We can help map where to apply which UX tactics, provide training and stick around to assist with projects until your team gets the hang of it.
There’s also a corporate culture component — you can’t just expect your hero to wave a wand and make it all better. The whole organization has to pull together and focus on UX. Without a supportive culture, your UX manager or team will only be able to make great recommendations that never happen. And they’ll quickly get frustrated and go find jobs in more serious customer-centric operations. Plus if your UX hero is the only one around who “gets it,” then where will you be when your hero leaves? It takes a village to sustain a really good customer experience.
If you’ve got no budget, a little user experience planning and usability testing is better than none. Begin by gathering some guerilla user feedback through friends and family if you have to. Then when improvements happen because of it, measure the impact and shout it from a mountaintop. Hopefully each project will nudge your organization further along the path to having a mature UX practice that continually makes life better for your customers.
Where is the best place to inject a usability testing cycle into a project plan?
Ideally, early and often. We’ve tested concept sketches on paper before even kicking off design, because if your users don’t even want a feature or don’t understand your concept, why waste all that time and effort designing or building it? And we all know it gets harder and harder, not to mention much more expensive, to make changes as you get further along.
After testing options early, refine through more testing as the design comes together. Then, validate the details in a full prototype before it goes live. That way you have the trail of continuously improving metrics like success rates and time-on-task to prove that your user experience planning and usability testing is making a real difference.
But if you only get one round of testing, I’d suggest doing your usability testing on low-fidelity interactive wireframes before applying creative design so your users focus on the flow of the experience rather than getting hung up on aesthetic issues.
What’s the best way to find participants for a usability study?
This can be the trickiest part. It depends on who will ultimately be using your site. Ideally, you want your participants to actually BE those users, or at least be sort of like them. If you’re testing an intranet or a site aimed at a particular industry niche, it’s easier because you have a captive audience. If you’re testing a broader public site or application, try to match your recruitment criteria with the demographics of the various user types you expect will use it. If you’re testing something meant for existing customers, then start by exporting a customer list matching the criteria for the types of customers most likely to use it (be sure you have their permission to contact them for feedback).
Finding participants can be very time consuming, so you could contract a recruitment company so you don’t spend all your time finding the right participants. Some caveats: get references first, and be careful using their data sources because you don’t want your study to be tainted by professional testers who are just in it for the money and know how to give you the answers you’re looking for instead of the truth.
You may have the perfect vision of the perfect user, but it may not be reasonable to expect to locate this mythical unicorn — much less several of them — with availability to participate in your study. You’ll probably need to prioritize recruitment criteria so you cover the must-haves, but remain flexible on the nice-to-have attributes. And you’re way more likely to get good participants if you reward them appropriately with incentives that match the value of the time you are asking them to take out of their lives — $75–150 is standard for an hour session nowadays. Cash works well, but so do gift cards or even company swag if it’s an internal project.
What are the differences between doing usability testing in-house versus using a vendor?
Fresh eyes are always good. Folks in-house so thoroughly understand how your business works that it’s often difficult to remove that familiarity and approach something from the view of your customers. It can really help pair your knowledge with an outside vendor to evaluate usability and provide a different perspective.
Experience is a big factor, and for usability that boils down to time in the lab and sessions. For the vendor, usability testing is probably a core competency. While for an in-house team, user experience may not be their primary job and, therefore, they may not have had the opportunity to do a lot of it. Lokion’s usability experts have moderated hundreds of sessions and are pros at the nuances of scripting without leading, drawing golden nugget verbatims out of shy users and deftly navigating any number of technical issues that may arise.
What are some of the most common pitfalls for a usability project, and how does Lokion work to avoid them?
Lots of companies claim they care about user experience and “do usability,” but they really don’t. They try to prove it with a lot of survey data, focus group sessions, automated testing and fancy heatmaps. But none of these inputs really get to the heart of the matter to help you figure out WHY people do what they do and HOW to make it easier for them to do what you need them to do. Lokion sticks to doing the basics very well. Yes, we can do surveys and focus groups and automated testing and even heatmaps, but what you probably really need is hard data like time-on-task along with real insight into users’ behavioral motivations that leads to resolutions for sticky user experience challenges.
Another pitfall we see is bowing to the HIPPO (Highest Paid Person’s Opinion). Sometimes usability testing is replaced by reviewing with the boss, who is NOT usually an intended user. Yes, their opinion, buy-in and approval are important, but that’s not the same as validating the experience with real users. And their opinion shouldn’t outweigh data and insights gathered directly from real usability testing. At Lokion, we often cater our presentation of findings to satisfy the HIPPOs, but we seldom include them in usability testing… unless, of course, we are testing an executive dashboard that actually is intended for them.
Finally, another common problem area is deliverables that fail to communicate. I’ve seen some rote usability study reports that don’t give anyone any idea what’s really wrong with the user experience much less how to make it better, plus no evidence to make the case to get the budget to fix it. One of the first questions we ask when planning a Lokion usability study is, “Who do we need to convince of what?” and “What turns them on?” – Are they numbers people? Do they like pretty pictures? Do they need to hear it straight from the customers? Knowing the answers to these questions helps us tailor our reporting to communicate the outcomes in a way that will be more likely to motivate action to fix problems … which is ultimately the point of doing all this.
Thanks so much to Shiloh Barnat for answering a few of our questions. Have more questions for Shiloh? Please comment or send her an email.This entry was posted in Lokion and tagged customer experience, Strategy, Usability. Bookmark the permalink.