Five people is enough

Suggest a usability test with five people and you tend to hear one of two things. "Only five?" Or: "We can't afford a study." Both come from the same picture of testing: a big research project with a big sample and a long report.

It doesn't have to be that. For most small teams, it shouldn't be.

What five people can tell you

Watch five people try to do one important thing on your site. Book an appointment. Find a price. Submit a referral. The same problems come up again and again. Partway through, you start to know where the next person is going to hesitate.

That's what a small test is for: finding the problems that stop people. It's very good at that.

What it isn't for is putting a percentage on a slide. If four out of five people finish a task, that doesn't mean 80% of your customers will. It means four people finished, one didn't, and now you know why. I report it that way, and you should be wary of anyone who doesn't.

Set the bar before you watch

This is the rule I care about most. Decide what "pass" means before anyone sits down, and write it down.

"A first-time visitor can find our hours and call us, without help, in under a minute." "Three of five people complete the referral form without leaving the expected path." Whatever fits the task.

Why before? Because after you've watched, you'll want to be generous. You saw the person almost get it. They were close. The design is basically fine. Criteria set afterward bend to fit the result, and a test that can't fail isn't a test. On my projects, the criteria don't move once the sessions start.

A small test that earned its keep

On an enterprise case-management system for KPMG, we did more than sixty interviews and weeks of flows before anyone saw a screen. Then we tested the prototype with four people: two Managing Directors, a Manager and a Senior Associate, all daily users of the platform it would live in. Three tasks.

Three of the four finished every task. That confirmed the direction: senior people could get through a system they'd never seen in under two minutes. But the misclick rates came back high, from 60% to 70.6% depending on the task. People were tapping the wrong things on the way to the right answer.

Four people is a check on the right people, not a statistic. It was still enough to show exactly which screens were pulling taps off the path, and those got fixed before the build instead of after. The misclick numbers went into the report at full size. A stakeholder who sees a bad number, and then sees it fixed, trusts the next number you show them.

Watching beats asking

Some problems only show up when you watch someone work.

At NASA, I studied why scientists weren't moving from an old data-search tool to its replacement. Both tools could technically select files, so the problem wasn't on any feature list. But when people did their own real work in each one, it was obvious. The old tool let you select a whole set and act on it. The new one made you add files one at a time, then remove the ones you didn't want, one at a time. One participant asked, plainly, to be allowed to use ctrl and shift to select files. Nobody would have found that by asking "what features are missing?"

That study was thirteen sessions and a survey, bigger than five. The lesson holds at any size. People will tell you what annoys them. Watching them shows you the workaround they've stopped noticing.

When five isn't enough

Some questions need more people. If you want to compare two designs with confidence, or put a number in front of a board, you need a bigger sample, and I'll tell you so.

At Mattress Firm, we ran an unmoderated test with more than twenty sales associates per task. One task, creating a mattress-matching profile for a new customer and attaching it to an email, came back at 24% on the expected path. That's a number you publish, because it's the reason the next version existed.

But most small teams aren't there yet. They need to know where people get stuck on the one task that pays the bills. Five people, a bar agreed in advance, and an honest write-up will tell them.

What you get

A test plan with the pass criteria written down before anyone watches. Five sessions with people who match your real customers, moderated or unmoderated. Task success, time on task and misclicks. A change list, in priority order, that you can hand to whoever builds your site. It starts at $1,500 and usually takes one to two weeks.

If that sounds like your situation, let's talk.