User testing like they used to

What to do when your users are niche and budgets are tight.

Testing with users is something we always try to recommend, but as soon as you start explaining the potential costs involved in traditional user testing, it’s often something that gets sidelined by clients in favour of more ‘tangible’ production. The equipment, the preparation time, incentives for participants, sourcing participants; these can all be costly additions and quickly eat into the budget.

There is a compromise: paper prototype user testing.

The niggling doubts convinced us that we needed to do some testing.

Recently we undertook a project with some very niche user groups which required complex functionality. We could only hypothesise about the effectiveness of our solution; the niggling doubts convinced us that we needed to do some testing.

Paper prototype testing involves creating a paper based version of the application using the wireframes which can be arranged to form a low-fidelity yet convincing interactive prototype.

We’d followed our tried and tested process - sketching with the team and producing wireframes - so we were already part way there. Laurie, who'd produced a lot of the wireframes, was careful not to bias the tests; he busied himself with something else, eagerly awaiting the results.

As with traditional user testing, I created a set of tasks for users to complete which would help us to pin-point any potential weaknesses.

Paper Prototype Testing

Using these tasks I could work out most of the possible routes through the application and break up the wireframes appropriately. Some larger changes between interactions required whole new wireframes, some smaller interactions, such as toggled button states required new layers.

These wireframes and component layers were printed and and cut out; I ran through the tasks, testing myself to make sure I’d covered most eventualities (I didn’t dwell on this too much; I know I’ll never think of everything).

Paper iPad

To make the tests a little more convincing, I made a custom testing frame which looked a bit like an iPad. With some clever tape wrangling I crafted a hinge to allow me to place wireframes underneath whilst keeping the frame in place. I also added some tabs to some of the layers so that I could emulate animations and cues adding to the realism and aiding comprehension.

Video recording

Like traditional user testing, I wanted to record the sessions so we could reference them later. We set up a tripod with a decent DSLR camera which records HD video pointing at the table and hit record!

Like with traditional user testing we had someone to take notes (thanks Iain!) as I was facilitating the tasks. We encouraged the participants to think out loud and explain what they expected to happen.

Having started the testing not knowing how effective this method would be at proving or disproving our hypotheses, we were pleasantly surprised to learn that we had a little bit more work to do! As silly as this sounds; it may have meant that some of our designs needed further refinement, but it also meant that the testing had served its purpose.

It’s amazing what a bit of paper and perseverance can do!