Day 293 of the Daily · Product Management

The Five-User Test

5 min read · 4 principles · apply it by lunch

Morning. The instruction that makes this work is the one nobody can follow: don’t help them.

The question

Have you ever sat behind someone who’d never seen your product, handed them a real task and then stayed silent while they failed at it — or is everything you believe about how the thing gets used assembled entirely from your own hands on your own keyboard?

The idea

In 1993 Jakob Nielsen and Thomas Landauer presented a paper at the INTERCHI conference in Amsterdam under an unglamorous title — a mathematical model of the finding of usability problems — and it changed how software gets tested. Averaged across a large set of real projects, one person watched using an interface turns up roughly 31 per cent of its usability problems. Each extra person re-finds most of what the last one found, so the curve flattens fast: five people surface around 85 per cent of what’s there, and the sixth and seventh mainly hand you things you’ve already written down. Nielsen made the practical case in a 2000 essay, and the argument was economic as much as statistical — several small tests through a build beat one big one at the end, because you’re fixing while fixing is still cheap. Laura Faulkner stress-tested the claim in 2003, running sixty users on one interface and then drawing random sets of five from the pool: some sets found 99 per cent of the problems, others found 55. Five is a good average and a shaky single roll, which is an argument for running the test again rather than running it bigger. Sitting there while someone flounders feels like bad manners, so you explain what the button does — and destroy the only data the morning was going to produce. That is the trap, and it arrives dressed as politeness.

In the Daily · 4 principles and a template

The Five-User Test is Day 293 of 365.

In the Daily, each of its four principles is explained in full, with a one-page template to apply it the same day. A new framework arrives every morning at 7am for a year, across all fifteen courses of the FutureMBA curriculum.

$29 a month · cancel any time · 7-day money-back guarantee

More from Product Management

The next four in the course.

All 22 Product Management frameworks
Day 302 · Product Management

Affordances and Signifiers

Somewhere in your product sits a control that needed a tooltip, an onboarding step and a help article wrapped around it before anybody could work out what it did — so if all three were deleted tomorrow and nothing else changed, how many of your customers would still know where to click? The explaining is the part you can count. The control underneath it is the part somebody would have to fix.

See the framework →
Day 311 · Product Management

Which Side to Subsidise

If you run anything that needs two groups to turn up before either of them gets any value — buyers and sellers, readers and writers, players and developers — can you name which of the two you’re deliberately underpricing, say what the other side is paying extra for the privilege of reaching them, and tell me whether either of those numbers was ever a decision rather than something you inherited from whichever side signed up first?

See the framework →
Day 321 · Product Management

The Persona Spectrum

Take the last hundred people who started with your product and gave up inside a week: you’ve filed most of them under not interested, but how many were holding a baby, standing in a loud room, squinting at a screen in bright sun or working one-handed on a train — and could anyone on your team tell those two groups apart from the data you keep?

See the framework →
Day 331 · Product Management

Privacy by Design

Open your signup form and your database side by side, and for every field you ask a new customer to hand over, name the decision your business would make differently if that box came back empty — then, once you’ve been through the lot, how many are left that you’d defend to the person who typed them in? Somebody chose every one of those fields. Very few of them were chosen on purpose.

See the framework →