How to run a heuristic evaluation

A heuristic is a rule of thumb. Nielsen’s ten describe what usable interfaces tend to have in common: they show what is going on, speak the user’s language, let people undo mistakes, behave consistently, and so on. They are broad on purpose, so they apply to a checkout, a settings page or a booking app alike.

During an evaluation, each evaluator works through a set of realistic tasks and asks, at every step, whether the screen breaks one of the ten. Each problem becomes a finding, tied to the place where it happens and to the heuristic it breaks.

Each finding gets a pin on the screen and a line in the list.

Keep the list open while you evaluate. The question after each name is a short prompt to use on every screen. Nielsen Norman Group’s article explains each heuristic in full, with examples.

The output of an evaluation is a list of findings, and the way each one is written decides whether anything changes. Here is a typical first set of notes from a checkout review, then the same review written up as findings.

Nielsen’s severity scale runs from 0 to 4. A 0 means you don’t think it is a usability problem at all. A 1 is cosmetic, worth fixing only if there is time. A 2 is a minor problem with a low priority. A 3 is a major problem that should be fixed soon. A 4 is a usability catastrophe that has to be fixed before release.

To choose a number, weigh three things: how often the problem happens, how hard it is to get past when it does, and whether it keeps getting in the way after people have learned about it. A rare problem that blocks a payment can deserve a higher rating than a common one that costs a second.

Rate after the evaluation, once the lists are merged, so every evaluator sees every finding. One person’s ratings are unreliable on their own. Averaging several evens out the outliers.

One evaluator misses a lot. Different people notice different problems, so each extra evaluator adds findings the others missed, with fewer new ones each time. Three to five evaluators is the usual advice. Beyond that, most of what another person finds is already on the list.

Independence matters as much as the number. If evaluators review together in a room, the loudest opinion steers what everyone looks for. Working alone first and merging afterwards gives you the widest list.

A heuristic evaluation is an expert review. It predicts where people are likely to struggle, based on principles, and evaluators aren’t your users. They know the jargon, and they can’t forget what they already know about the design. Some findings will turn out to be harmless, and some real problems will be invisible to anyone who already knows how the product works.

A broken heuristic isn’t automatically a bug, either. A navigation menu hidden behind a button makes people recall where things are, but on a small screen it may still be the right trade-off. Record it, rate it, and let the team decide.

So treat the evaluation as a first pass that complements testing with users and doesn’t replace it. Use it to clear the obvious problems before a usability test, so the sessions can spend their time on what only real users can show you. Then use what you learn in testing to check which findings mattered.