Lokion Interactive

Today we are talk­ing with Shiloh Bar­nat, vice pres­i­dent of inter­ac­tion design, about usabil­ity testing.

Usabil­ity test­ing is often incor­rectly con­sid­ered a “nice to have” com­po­nent rather than a crit­i­cal one in web devel­op­ment. What advice would you give to some­one try­ing to con­vince stake­hold­ers that usabil­ity test­ing is essential?

Stake­hold­ers need to under­stand that usabil­ity test­ing is the best way to know if your site really works well for the peo­ple who will actu­ally be using it. Any site can have fast servers, clean code and pretty pic­tures, but the really good ones stand out because they offer a good expe­ri­ence. And if your site sucks, peo­ple just go elsewhere.

Besides, test­ing with real users replaces guess­ing and pol­i­tics with solid met­rics and evi­dence so you can make smarter decisions.

And if your stake­holder still isn’t con­vinced, remind them improve­ments made to cus­tomer expe­ri­ence as a result of usabil­ity test­ing feed the bot­tom line. Improved suc­cess rates and reduced time-on-task lead to higher con­ver­sion rates, more sales and increased profit from the site … not to men­tion more sat­is­fac­tion, which makes loy­alty more likely, which pays for the bud­get to do usabil­ity testing.

If an orga­ni­za­tion doesn’t already have a user expe­ri­ence (UX) prac­tice, what’s the best way to get started?

Get a hero. Put some­body who is pas­sion­ate about user expe­ri­ence in charge of it and make sure they have the time, bud­get, train­ing and per­mis­sion to lead the way. You’ve got some­body in charge of most of the other pieces — IT, mar­ket­ing, account­ing, pro­duc­tion, ful­fill­ment, ser­vice. Good cus­tomer expe­ri­ence means all those pieces have to work together, but that prob­a­bly won’t hap­pen unless it’s somebody’s job to make it hap­pen. And if you don’t hap­pen to have a UX hero on hand, Lokion is here to help. We’ve men­tored sev­eral large com­pa­nies through solid­i­fy­ing their UX prac­tices. We can help map where to apply which UX tac­tics, pro­vide train­ing and stick around to assist with projects until your team gets the hang of it.

There’s also a cor­po­rate cul­ture com­po­nent — you can’t just expect your hero to wave a wand and make it all bet­ter. The whole orga­ni­za­tion has to pull together and focus on UX. With­out a sup­port­ive cul­ture, your UX man­ager or team will only be able to make great rec­om­men­da­tions that never hap­pen. And they’ll quickly get frus­trated and go find jobs in more seri­ous customer-centric oper­a­tions. Plus if your UX hero is the only one around who “gets it,” then where will you be when your hero leaves? It takes a vil­lage to sus­tain a really good cus­tomer experience.

If you’ve got no bud­get, a lit­tle user expe­ri­ence plan­ning and usabil­ity test­ing is bet­ter than none. Begin by gath­er­ing some guerilla user feed­back through friends and fam­ily if you have to. Then when improve­ments hap­pen because of it, mea­sure the impact and shout it from a moun­tain­top. Hope­fully each project will nudge your orga­ni­za­tion fur­ther along the path to hav­ing a mature UX prac­tice that con­tin­u­ally makes life bet­ter for your customers.

Where is the best place to inject a usabil­ity test­ing cycle into a project plan? 

Ide­ally, early and often. We’ve tested con­cept sketches on paper before even kick­ing off design, because if your users don’t even want a fea­ture or don’t under­stand your con­cept, why waste all that time and effort design­ing or build­ing it? And we all know it gets harder and harder, not to men­tion much more expen­sive, to make changes as you get fur­ther along.

After test­ing options early, refine through more test­ing as the design comes together. Then, val­i­date the details in a full pro­to­type before it goes live. That way you have the trail of con­tin­u­ously improv­ing met­rics like suc­cess rates and time-on-task to prove that your user expe­ri­ence plan­ning and usabil­ity test­ing is mak­ing a real difference.

But if you only get one round of test­ing, I’d sug­gest doing your usabil­ity test­ing on low-fidelity inter­ac­tive wire­frames before apply­ing cre­ative design so your users focus on the flow of the expe­ri­ence rather than get­ting hung up on aes­thetic issues.

What’s the best way to find par­tic­i­pants for a usabil­ity study?

This can be the trick­i­est part. It depends on who will ulti­mately be using your site. Ide­ally, you want your par­tic­i­pants to actu­ally BE those users, or at least be sort of like them. If you’re test­ing an intranet or a site aimed at a par­tic­u­lar indus­try niche, it’s eas­ier because you have a cap­tive audi­ence. If you’re test­ing a broader pub­lic site or appli­ca­tion, try to match your recruit­ment cri­te­ria with the demo­graph­ics of the var­i­ous user types you expect will use it. If you’re test­ing some­thing meant for exist­ing cus­tomers, then start by export­ing a cus­tomer list match­ing the cri­te­ria for the types of cus­tomers most likely to use it (be sure you have their per­mis­sion to con­tact them for feedback).

Find­ing par­tic­i­pants can be very time con­sum­ing, so you could con­tract a recruit­ment com­pany so you don’t spend all your time find­ing the right par­tic­i­pants. Some caveats: get ref­er­ences first, and be care­ful using their data sources because you don’t want your study to be tainted by pro­fes­sional testers who are just in it for the money and know how to give you the answers you’re look­ing for instead of the truth.

You may have the per­fect vision of the per­fect user, but it may not be rea­son­able to expect to locate this myth­i­cal uni­corn — much less sev­eral of them — with avail­abil­ity to par­tic­i­pate in your study. You’ll prob­a­bly need to pri­or­i­tize recruit­ment cri­te­ria so you cover the must-haves, but remain flex­i­ble on the nice-to-have attrib­utes. And you’re way more likely to get good par­tic­i­pants if you reward them appro­pri­ately with incen­tives that match the value of the time you are ask­ing them to take out of their lives — $75–150 is stan­dard for an hour ses­sion nowa­days. Cash works well, but so do gift cards or even com­pany swag if it’s an inter­nal project.

What are the dif­fer­ences between doing usabil­ity test­ing in-house ver­sus using a ven­dor?

Fresh eyes are always good. Folks in-house so thor­oughly under­stand how your busi­ness works that it’s often dif­fi­cult to remove that famil­iar­ity and approach some­thing from the view of your cus­tomers.  It can really help pair your knowl­edge with an out­side ven­dor to eval­u­ate usabil­ity and pro­vide a dif­fer­ent perspective.

Expe­ri­ence is a big fac­tor, and for usabil­ity that boils down to time in the lab and ses­sions. For the ven­dor, usabil­ity test­ing is prob­a­bly a core com­pe­tency. While for an in-house team, user expe­ri­ence may not be their pri­mary job and, there­fore, they may not have had the oppor­tu­nity to do a lot of it. Lokion’s usabil­ity experts have mod­er­ated hun­dreds of ses­sions and are pros at the nuances of script­ing with­out lead­ing, draw­ing golden nugget ver­ba­tims out of shy users and deftly nav­i­gat­ing any num­ber of tech­ni­cal issues that may arise.

What are some of the most com­mon pit­falls for a usabil­ity project, and how does Lokion work to avoid them?

Lots of com­pa­nies claim they care about user expe­ri­ence and “do usabil­ity,” but they really don’t. They try to prove it with a lot of sur­vey data, focus group ses­sions, auto­mated test­ing and fancy heatmaps. But none of these inputs really get to the heart of the mat­ter to help you fig­ure out WHY peo­ple do what they do and HOW to make it eas­ier for them to do what you need them to do. Lokion sticks to doing the basics very well. Yes, we can do sur­veys and focus groups and auto­mated test­ing and even heatmaps, but what you prob­a­bly really need is hard data like time-on-task along with real insight into users’ behav­ioral moti­va­tions that leads to res­o­lu­tions for sticky user expe­ri­ence challenges.

Another pit­fall we see is bow­ing to the HIPPO (High­est Paid Person’s Opin­ion). Some­times usabil­ity test­ing is replaced by review­ing with the boss, who is NOT usu­ally an intended user. Yes, their opin­ion, buy-in and approval are impor­tant, but that’s not the same as val­i­dat­ing the expe­ri­ence with real users. And their opin­ion shouldn’t out­weigh data and insights gath­ered directly from real usabil­ity test­ing. At Lokion, we often cater our pre­sen­ta­tion of find­ings to sat­isfy the HIP­POs, but we sel­dom include them in usabil­ity test­ing… unless, of course, we are test­ing an exec­u­tive dash­board that actu­ally is intended for them.

Finally, another com­mon prob­lem area is deliv­er­ables that fail to com­mu­ni­cate. I’ve seen some rote usabil­ity study reports that don’t give any­one any idea what’s really wrong with the user expe­ri­ence much less how to make it bet­ter, plus no evi­dence to make the case to get the bud­get to fix it. One of the first ques­tions we ask when plan­ning a Lokion usabil­ity study is, “Who do we need to con­vince of what?” and “What turns them on?”  – Are they num­bers peo­ple? Do they like pretty pic­tures? Do they need to hear it straight from the cus­tomers? Know­ing the answers to these ques­tions helps us tai­lor our report­ing to com­mu­ni­cate the out­comes in a way that will be more likely to moti­vate action to fix prob­lems … which is ulti­mately the point of doing all this.

Thanks so much to Shiloh Bar­nat  for answer­ing a few of our ques­tions. Have more ques­tions for Shiloh? Please com­ment or send her an email.