Recently, a colleague and I were discussing ways to encourage negative feedback during usability tests. Sometimes, participants can just be too nice.
In this case, she is both the lead developer and the usability tester for a software application. Not the ideal arrangement, for sure, but one that I think is probably pretty common for those of us working in the science + design world. We have to wear more than one hat.
She was worried about biasing her participants; that they wouldn’t want to hurt her feelings by criticising her work… and, by extension, her.
So how do we get them to give it to us straight?
No, I don’t suggest offering money :)
I suggested that it is largely about people skills. As the tester, we need to try and encourage the participant to think out loud, and to forget that we are there (and that we might have some vested interest in this project!).
My colleague Jenny is good at coaxing feedback out of participants, positive or negative. She is always ready to answer a question with a question. But often she is a buffer between user and developer, so the participant relaxes a bit, and might be more inclined to say what they think.
But what if the tester is the developer?
I decided to pose a question about it on the IXDA forum, just to see what other UX people out there thought.
One person suggested that we put plenty of references to “making it better” and “finding things before they cause problems for other people” into the [test] script.
In general, we want to keep the focus of the test on the application (and, of course, not on the user), maintaining that we are trying to find problems to solve, and that the participant can help us do that.
Trying to test each others’ applications might be good as well, where time and other constraints allow. This would also encourage some “cross-pollination” of ideas.
Someone else suggested giving the user a selection of adjectives (positive and negative) that they could freely choose from to help describe their experience. Of course, there we’re talking about opinion rather than behaviour, but it might be helpful in some cases.
As well as the participants needing to let go and say what they think, a developer / tester would also have to be sufficiently detached. I’ve mentioned before that in those circumstances, you’d need to be calm, and not begin teaching or guiding the participant.
If you show that you’re happy to receive any feedback, then that should set the participants more at ease.
Benefits of observation
As Jakob Nielsen (there he is again!) says, don’t listen to users.
One of the big advantages of sitting with a user, and observing where and when they struggle with something, is that you’re observing their behaviour.
Sure, they might say at the end, “Well, this version is much nicer than the last one. And I love the icons!”, but as long as you’ve made a note of the “pain points” they encountered, then you have gained an insight, and learned something about your application’s usability.
Bring it on!