Alessandro Molinaro
09/05/19
Tech
Usability testing: the everlasting debate
Usability tests are a strange kind of beast. Everybody wants to do them (in principle) but, when the planning starts, there is always a good reason not to.
It’s a little bit like a medical check-up. Everybody claims they’re important but you never get around to having one (and what if they discover that I have something? I know I’m healthy!)
Why test?
There are a lot of reasons why testing is incredibly useful. The main one is that it allows the company to save a great amount of money in development costs (especially if testing starts with prototypes) because it’s less likely that we’ll build the wrong solution or that we spend a lot of time developing something right but difficult to use.
Additionally, we can assess what will be the ROI (Return of investment) for the tested product, which helps build a strong case with the client and the stakeholders.
Conceptual model vs Mental model
Usability testing usually involves systematic observation under controlled conditions to determine how people use the product from three different points of view: how well they use it (usability), how much the experience is satisfactory and if they use it in the same way we think they should.
Technically speaking: we try to determine if the conceptual model (the way in which the product should work/be used for the designer) coincides with the user’s mental model (how the users think it works).
Differences between what the companies think and what the user thinks are so common that unorthodox uses of products, if spotted in time, can become a resource for the company.
Just to give you an idea: Kleenex was introduced in 1924 as disposable face towel to remove make-up. Only later did they discover that people used them to blow their nose, avoiding fabric handkerchiefs that were more expensive and needed to be washed.
This pushed the company to find a more practical packaging to accommodate the unexpected use of the product because it involved a different context of use (I can blow my nose in the street or on a train, while make-up removal is more probably done at home).
A more recent example is the SMS messaging system. It was created to send service notifications, and the conceptual model was that no one would have written if they could speak. In fact they didn’t charge for the service at the beginning. They had the chance to observe that they were wrong. Wrong, and losing a bunch of money.
What is usability testing?
Usability testing is a type of behavioural research. Data is captured by observing what users do... Literally.
We set up the test laptop in a room and ask a participant to perform some tasks with our product. The participant, naturally, is a representative of the audience we’re interested in.
During the tasks we invite the participant to describe what they’re doing, thinking and feeling, while we observe and try to be as neutral as possible to avoid affecting their behaviour. So no “great” when they do something that we hoped they would, no disappointed faces when things go south, etc.
In a separate room, observers (stakeholders, devs etc..) can observe what’s going on, possibly moderated by another researcher.
At a glance seems a pretty simple business. Quite easy to explain, but complex to do it right. You don’t need a lab, so you can test in the office, at home (best in the user’s home), and even in coffee shops and other public places. The most important thing is to test, early and often.
Best practice: testing prototypes
The main reason to test prototypes, from a production point of view, is to have an informed design process that really focuses on real users, uncovering issues that no manual/blog/”Amazon-does this” can tell you. Plus it gives us the ability to build only parts that will really work, avoiding the need to scrap pieces of software once we discover they don’t.
It’s important to remember that even if we have software that is functionally stunning, if users cannot understand how to use it or they get frustrated trying to make it work, it’s unlikely that they’ll come back and try again.
Everlasting debate
Usually, there are some common arguments for not testing, especially within prototypes.
The main are:
- It interferes with the Agile process
- It slows down the process because devs needs to wait for the design to be completed and tested
- Prototypes will be discarded after the building phase, so it’s a waste of resources
- We can build it and we’ll apply UX/design after
For the time being, these are some very short answers:
- No, it can and should be integrated within the agile process, and the devs should be part of the discovery phase, e.g. the Google Sprint or the Lean UX process
- It doesn’t slow down the process because it saves development time later on
- Prototypes are iterative, and previous prototypes are the foundations of newer products; by creating a design system we’ll have a “Pig system” so nothing will be wasted and everything can be reused
- Unfortunately it’s not possible to “apply UX”. We can improve the usability by adjusting a bad interface, but the experience is way deeper than the interface alone. And usually it involves the back end. Yup. For real. No joke.
So what?
To summarise, there are a lot of good reasons to test both the prototype and the final products:
- Have a truly Used/Customer centred product
- Have a baseline to compare different versions
- Give a monetary value to the improvements
- Base the design on reliable data.
- Avoided building the wrong thing (or the right thing but badly)
I’m not saying that we should test at every step but...ok, no that’s exactly what I’m saying. We should test as early as possible and as often as possible. We should shape the product and the interaction on user expectations and abilities and always keep in mind that if a user is not able to do something on our product, it’s our fault. The best customer experience wins. End of the story.
Alessandro Molinaro
Read more by Alessandro Molinaro
