Poor understanding of results will ruin your design. Testing seems easy to make but more problematic is the process of analyzing. In this article, I want to tell about the most common clutch problems.

1. Carryover effect

While you make competitive usability testing (a method where you test your product compared to 2–3 competitors' products) then it’s easy to get fake results. If you start tests at first with another polished and thoughtful product and after that, you show your simple MVP — you should be aware he will rate the experience a way worse. Imagine that you’re driving a brand new Ferrari and after you jump to an old, cheap Toyota. You would feel this car is horrible but if we reverse the situation — you start driving at first with the Toyota then after all you would probably give a higher rate of it.

2. Prior experience doesn’t tell the truth

Of course, it all depends on the context, and what you want to test but remember — if you have to make an important decision then testing a product with someone who had prior experience will not give you real results. It’s comfortable to test features with your friends but if you take them as participants all the time — they could already learn patterns and their’s mental model remembered the product. In the real world, your users will not have prior experience and they can interact in a different way.

3. Assuming the user knows what’s wrong

If the police would assume that all suspects tell the truth then everyone would be not guilty. However, if a user is telling you something is wrong then keep your distance from it. If users are told the flow is too long, it can mean it’s too hard or they don’t understand the steps. Your role is to dig deeper and take the right assumptions from it. I’ve written an article about it.

4. Suggestions while testing

Imagine a world where teachers tell students how to solve the exam’s exercises. Imagine that you’re operating by a doctor who has a degree because someone helped him during exams. However, sometimes I’ve seen the facilitator who gave suggestions how to finish the task or point out the participant as a source of the problem because he’s stupid. Just don’t do that, never.