One technique that is becoming popular with some is focus group usability testing. I recommend that you avoid this technique completely. I’ll try to explain why.

What is focus group usability testing?

Instead of one-on-one sessions, the facilitator asks 6-8 people to carry out tasks simultaneously using individual computers. After they have attempted the task, the facilitator then asks the participants collectively if they experienced any problems. The problems they report become the findings of the study.

The idea is that you can pick up on themes and find common problems. You can involve lots of participants in a short period of time. It also means that you are using less of the facilitator’s time. So it’s cheaper.

Quality of findings

Defenders of focus group usability testing will argue that the number of findings returned from this method is comparable with one-on-one testing. However the number of findings is not an adequate measure of the success of usability techniques. The issue with testing simultaneously in focus groups is that you can not rely on the accuracy of your findings.

The problem with focus groups

Focus groups are arguably useful for many types of research. Gathering data on user behaviour is not one of them.

Humans are poor at describing their own behaviour. This is because they don’t often remember their precise actions. Social pressure also inhibits their ability to tell the truth.

This social pressure will be multiplied when they are trying to describe their actions to 8 people they have never met. The people most likely to inaccurately describe the problems they experience, are the ones with the biggest ego. These are the same people who will be doing all of the talking at your ‘test’.

Observing the user experience

The key to usability testing is observation. Anyone who has observed a number of usability tests will know that participants’ recollections of their actions often have little resemblance to reality. You need to watch them if you want to know what they did.

The causes of important usability issues often go completely unnoticed by the test participant. The issues are picked up because the participant is being observed.

I can think of only 2 ways to properly observe a room full of people who are simultaneously using your website. The first is to match every participant with an observer. This doesn’t work as a budget technique because it’s very expensive. It’s also pretty complicated to collect the results and will probably be very intrusive. The other option is to use the Lernaean Hydra as your test facilitator.

Cut costs not corners

There are many ways that you can make the most of a limited usability budget. The key is to cut out the expensive bits without affecting the reliability of your findings. A skilled usability professional will be able to suggest approaches to meet your budget without returning unreliable results.

Some examples include:

• Using fewer participants
• Recruiting your own participants
• Ditching a written report in favour of a one day workshop
• Using a meeting room instead of a lab

Of course, the biggest cost-cutter is to learn how to run your own tests. Truly helpful usability consultants will encourage you to do this and help you learn. I’ll write a more detailed post on this topic soon.

What do you think?

So what are your thoughts about group tests? Please leave a comment and let me know your opinion.