The benefits and limitations of expert reviews

Posted October 2nd, 2008 by David Hamill

Usability testing is the Champagne of user research. It doesn’t get much better than watching real people using your website. However like Champagne, usability testing is expensive. It also takes a while to organise, carry out and present the results.

If usability testing is Champagne then an expert usability review is Cava. It is cheaper and the results can often be similar. This is when a usability expert reviews the site or areas of it for usability problems.

They are often described as heuristic evaluations. However I think this is just to make them sound posh. An heuristic evaluation relies on heuristic principles rather than the experience of the reviewer.

It’s quicker

A usability specialist can often deliver your expert review in the same week that you ask for it. If time is precious then an expert review can deliver results within your tight timescales.

It’s cheaper

You don’t need to recruit participants and you use less consultancy time, so it can work out to be a lot cheaper.

You can use it to get more out of your usability tests

If you’ve never done any usability testing on your site, there can be a lot of issues that will be obvious to a usability expert. So an expert review can help you fix these issues before you do some testing. You’ll then avoid having the study being dominated by issues that you didn’t need a usability test to find.

The limitations of expert reviews

The results are based on opinion. It will hopefully (but not always) be an expert opinion, but it is an opinion nonetheless. Usability testing results are (hopefully) based on observed facts. You know the issues exist because you saw people having problems.

You need an expert to carry them out. With a bit of coaching you can do your own usability tests. But expert reviews should be carried out by someone spends a lot of time watching people using websites. You have to trust the fact that the person doing the review actually has this experience.

It’s difficult to prioritise the findings. Even a very experienced usability specialist will struggle to prioritise the findings of their review. The very important findings will be obvious. Everything other than that is difficult to put a severity level on.

Some sites are not appropriate for expert reviews. If the subject of your site is specialised then an expert review may not be a good idea. For example if you’re a plant hire firm, a usability consultant is not best placed to provide you with insights into the motivation behind people looking to hire cherry pickers. Only the really outstanding issues will be found.

Some advice

Find out about your expert. If you are paying a company to review your site, find out about the person who is going to review it. How long have they been with the company? How much usability testing have they done there?

Like many consultancies, some usability companies hire people straight out of university. Many of these people will make excellent consultants but until they’ve spent a lot of time observing users, they are not  experts.

Test with real users as well. After you’ve fixed the issues from your expert review you should make usability testing your next step.

Tags: expert reviews, heuristic evaluations, tips