The Hudson River usability test

Posted February 23rd, 2009 by David Hamill

You never really know how well something works until people try to use it. Even the most logical seeming processes can have problems when they are tested with users.

The problems you find in usability testing, often appear totally obvious with hindsight. But until you see them, they are not obvious at all.

A real life usability test

I was watching a TV show about the successful ditching of US Airways Flight 1549 in the Hudson River. Like any usability geek, I managed to relate this incident back to the principles of user centred design.

On that day, a real-life usability test was taking place. A man called Jeff Skiles was the test participant. The item being tested was a manual that tells you what to do when your Airbus A320 suffers double engine failure.

A heroic landing

Like me, you were probably awestruck at the skill and professionalism of US Airways pilot Chesley Sullenberger and his co-pilot Jeff Skiles. They successfully ditched a passenger jet into the Hudson River, saving the lives of every passenger and crew member on board. Not bad for a day’s work eh?

The work of the co-pilot

Thankfully, passenger jets don’t suffer double engine failure very often. So when they do, there’s a manual with step-by-step instructions of what to do. It’s the co-pilot’s job (or whichever pilot isn’t flying the plane) to read through the manual.

In the case of Flight 1549, all attempts to restart the engines failed. So the Captain took the brave decision to ditch the plane into the Hudson River. The manual that co-pilot Jeff Skiles was using included instructions for such an eventuality.

Step-by-step instructions

The person (or people) writing this important manual, took a seemingly wise decision to provide it in step-by-step instructions. This made the instructions simple and easy to understand in the time of a crisis. Skiles went through the instructions, preparing the plane for a water landing. However, there was a small but important problem with this approach that would soon become apparent.

The ditch button

The flaw in the manual arose as the plane got close to the water. When ditching on water, there is a button on the Airbus A320 that seals the underside of the plane. This prevents it taking on water and sinking.

This button should be pressed just before impact. In a step-by-step instruction manual, the instruction to do so is on the last page.

By the time co-pilot Jeff Skiles was in a position to read such an instruction, the plane was a few metres above the water. He was probably busy providing all sorts of readings to Captain Sullenberger, so was unlikely to still be reading the manual.

Skiles didn’t press the button, so the Airbus A320 took on water after impact and began to sink.  Thankfully everyone was able to escape the plane and survived.

The benefit of hindsight

With the benefit of hindsight, we can see that a co-pilot is unlikely to be reading a manual, seconds before the plane he’s in hits the water. But until someone uses it to ditch a plane, such an insight doesn’t come to light. The manual’s reader needs to be pre-warned about the button, so they don’t miss the instruction altogether.

So what does this have to do with websites?

If you aren’t testing your site with real people, you are missing the opportunity to the find problems that are impossible to predict. Thankfully the risk of missing these insights will rarely be as serious as this example.

Websites don’t pass or fail usability tests

Everyone on Flight 1549 survived. So you could argue that the manual passed the test. But this isn’t a helpful way of looking at usability testing. It’s not something a website passes or fails.

If you approach testing as a phase that your site needs to pass, you are unlikely to be open-minded enough to get the full benefit of it.

Instead the aim is to find improvements that can be made. For example, re-ordering the flow of information, so that your users get the appropriate information at the point when they need it.

Enjoy this post?

If you enjoyed this post then you might also like some of these:

• Focus group usability testing
• Avoiding the R word
• Usability tests and the effect of learning
• Usability test tasks to avoid

You can also share this article using these buttons.

Tags: Usability testing