A colleague and I recently had an argument. He claimed that research is a meaningless waste of company time and resources, because absolutely all data can be obtained from the analytics system.

As a product manager and researcher it was very strange to me, even amusing. So I decided that I would make a separate post with detailed answers and my comments on it.

P.S. A colleague changed his mind after my proof:)

It seems to me that this is a rather important topic, since many of us, in the course of our work, encounter people who are quite skeptical about research. And it’s important to understand that you can’t judge such people. We as researchers and product managers have to understand why people think this way, where they got this opinion.

And instead of expressing subjective points of view from each side, we need to switch to the language of facts and prove that we’re right. Let’s do that.

Quantitative Analytics.

What is quantitative analytics? These are established analytics systems in your product, like Google Analytics, Amplitude, and systems that show what your users are doing in the product.

The key point to remember is that quantitative analytics answer the question «What?» and «How much?» It’s important to understand that we can only make subjective hypotheses based on quantitative analytics.

Here’s a simple example. You have released a new feature that you think can increase the value of the product and engagement. You release the feature and see that only 20% of your entire audience visits it, and only 5% of that 20% are actively using it. Based on quantitative analytics, we conclude that customers don’t need the feature and we’ve wasted money and time. The reasoning makes sense, right?

But, alas, no. This approach is very negligent and wrong on the part of product teams. Why? First, let’s talk about what user research is.

User research.

User research is when you communicate with users in the format of an interview, survey or some other activity. And the purpose of that communication is to understand the answer to the question, «Why is this the way it is?» Why did users do it this way and not that other way? What motivated him? What goal did he want to achieve and what did he expect to see?

And this is really important. If we go back to the example with the feature we talked about above, it’s important for us to understand — why did this happen? Why did only 20% go to this function and only 5% of the 20% use it? If we had only quantitative data we would have already declared it a failure and closed it, but if we make a research we will find out that 80% of users don’t see the new feature on their home screen and 15% of those who do see it don’t know how to use it properly.

And then we redesign the home page, we highlight the new feature correctly, and we send emails about the new feature, and we send emails about the new feature that’s out. And for those who use the feature, we add a brief onboarding on how to use it.

What’s the result? The feature was successful for 85% of users. It increased the value of the product and engagement. Users are happy, business is happy.

An important note.

It’s important to realize that quantitative analytics and user research are interchangeable. They complement each other. We can’t answer the question «Why?» if we don’t have the context, and we also can’t answer the question «What?» and «How much?» if we don’t have the data.

Keep this in mind and don’t neglect these parts of product development. These are very important things to always consider if you want to make successful and useful products.