How to deal with cognitive bias in Product - Part 1 - Mind the Product

How to deal with cognitive bias in Product

In the first of a two-part series, Mind the Product’s Principal Strategist Christine Itwaru examines different forms of cognitive bias and looks at what product managers can do to combat them

---

We’ve all encountered the b-word at some point in our professional and personal lives. It’s the word that forces you to say: “Wait, what was that? Did that just happen to me?”

That b-word I’m referring to is bias. It’s a word that’s difficult for many of us to hear, process, and come to terms with. In product management, we encounter bias all the time – in conversations with stakeholders, engineers and customers, in our planning sessions, and even during data reviews.

Psychology Today defines bias as the following: “Cognitive biases are repeated patterns of thinking that can lead to inaccurate or unreasonable conclusions. Cognitive biases may help people make quicker decisions, but those decisions aren’t always accurate.”

Bias essentially reduces our cognitive load, so we are able to make decisions more quickly. However, as we all know, biases tend to have a negative impact on decision-making. Many of us can attest to making biased decisions that have had an impact on the long-term success of a feature, the health of our team, or even our own mental health.

I recently gave a talk covering 10 common cognitive biases that impacting product teams today, and how to overcome them. The response was bigger than I imagined. Many people contacted me to let me know they have asked their peers to review the biases I laid out (and others). They’ve also checked themselves many times, and have opened up to their teams about how bias has impacted their decision-making in an effort to change their behavior going forward. I’m sharing these biases through a two-part series in the hope that more individuals and teams take note, keep each other accountable, and make better long-term and healthy decisions for their product and teams.

1. Confirmation bias

Confirmation bias is our brain’s underlying tendency to notice, focus on – and give greater credibility to – information that aligns with our existing beliefs. It’s a short-cut that our brains use to make sense of the huge amount of data that’s being thrown at them all the time.

In many ways it’s a survival strategy – we can’t create a new construct each time we’re presented with new data. So, we fit the new data we receive into existing constructs. Whether we want to admit it or not, confirmation bias also serves as a way to protect our self esteem. We don’t like to be told we’re wrong, especially when we’re wrong about something that we feel strongly about AND especially if it’s our own idea.

One place where confirmation bias can show up a lot for us is during user interviews. Think about what a user interview is. You want to learn about pain while at the same time present possible solutions to your customer’s problem. What if a product manager comes back from user interviews and reports that their idea was flawless? What if you found out that the only idea that was presented was that product manager’s idea? The idea they thought was the best thing since sliced bread? Well, that wouldn’t go over well for the product manager in the long run. They’ve just taken the user interview and turned it into a sales pitch rather than a research and validation session.

In this case, the bias shows up in qualitative data, but it can easily nudge its way into other types of data. If you are writing a survey or analyzing behavioral data, the questions you ask, how you ask them, and how you interpret and report the results are all prone to confirmation bias.

Combating confirmation bias

What can we do to reduce the chances of this bias showing up?

• Acknowledge and state this bias. Doing so reminds you to check yourself in the future. Sharing this allows others to help keep you in check.
• Form testable hypotheses to remove ego and place the focus on results.
• A great skill to work on early in your product management career is that of forming research questions. I recommend this to all who are starting out, and challenge all of us to routinely have someone check us as we progress. When defining research questions be aware of your bias and check to be sure that your questions are as objective and neutral as possible.
• For analysis, actively focus on points of data that do not confirm your hypothesis as your brain will naturally focus on data that confirms your hypothesis. Play devil’s advocate by looking at what your brain doesn’t want you to confirm quickly.
• For reporting, check to be sure that the way that you interpret and present the results are as objective and neutral as possible before drawing final conclusions and making recommendations.
• Triangulate your data sources! Do this especially when the decision is risky. Make sure you are using multiple data sources to inform your decision-making process

2. Ostrich effect

In my opinion, the ostrich effect is the cousin to confirmation bias. We define this bias as choosing to ignore information that threatens our preferred way of doing something or our points of view on a question.

It’s like burying your head in the sand when you want to ignore information, and avoid any discomfort – hence the ostrich effect. What happens here is you’ll likely outright ignore information that contradicts your beliefs or opinions and you’ll refuse to acknowledge risks or negative outcomes.

As a product manager, one of the most important things you need to do is evaluate feedback and make informed decisions to improve or kill your feature. This is especially important when you first launch. If a product manager has set up all the right feedback mechanisms but ignores negative feedback, they’re putting their head in the sand and allowing the ostrich effect to take hold.

What the product manager should do is partner with customer teams to understand more and act on the feedback by improving the experience. Staying positive in this situation leads to increased churn and decreased revenue retention.

This bias forces us to remember that we are a part of a team, and we need to look at how we serve both our team and our customers. By ignoring the negative, you’re saying your route based on the positive data is the best idea. Most of the time our individual ideas are not the best ones, and it takes working as a team and evaluating data fairly to get to the right ideas.

I’ve seen this bias show up many times in product managers who are new to the role, and who believe staying positive and focusing direction on the good will make them look successful. It doesn’t always work that way! We need to work towards scalable solutions that meet the needs of many customers, not just a few. This means understanding the bad and pairing it with the good.

Combating the ostrich effect

• Acknowledge the bias. It will help you to recognize that avoiding or denying reality will only make it worse in the long run.
• Actively seek out a healthy number of points of view and work on being open to feedback on what you’ve released and are presenting.
• More tactically, stand up transparent systems for checking progress and feedback on a regular basis. These can be dashboards, voice of customer readouts, or feedback products. These help you to see the bigger picture in a routine manner and encourage dialogue that helps you move beyond your bias and stick to the results.

Once ostrich has happened it’s pretty harsh. You can and should make sure you take note of the feelings you felt, and signs to look for to avoid it from creeping up on you next time a decision is needed.

3. Clustering illusion

Clustering illusion is when we spot a pattern in the data where there is none.

This one is dangerous because you actively pull a small subset of information to the forefront and likely want to believe (or actually believe) that the information will be consistent going forward.

Let’s take a real-life scenario: An investor looks at a stock going up for five days and thinks it’s always going to be that way. Or, someone bets on a sports team because it’s won four times in a row, but hasn’t looked at the team’s overall history and stats. Bet is placed, stock drops or team loses. This is a dangerous bias because we may start discerning “signals in the noise” where there aren’t any.

If we think about product, this is like a product manager releasing a new feature and choosing only to look at the first three days worth of data to determine what move to make next for the feature. Even worse, we look at that first week, and build a six-month roadmap from it. This product manager is only looking at a small clustering of data points to drive some pretty big decisions. Their decision will likely cause unnecessary work. Not just for their team, but for marketing, customer teams and others.

More importantly, no product manager wants to see something negatively impact customers where they’ll need to pull back. That impact has downstream effects on other teams that cause stress all around.

Most likely in that scenario there were specific events that had an impact on usage in those first three days – such as marketing, customer success outreach and in-app messaging.

Combating clustering illusion

• As product managers it’s important for us to understand basic probability: Know that random events can appear to form patterns or clusters, even when they occur randomly. This helps us to recognize when patterns are a result of chance, or if they could be statistically significant. In the absence of a data scientist or analyst, it’s especially important to work on this skill.
• Look for alternative explanations as to why the data appears that way. I mentioned above the short time frame and activities around it – consider all of these. Drill into what factors were actually at play.
• Lastly, increase sample size, or time frame – go bigger to avoid false signals in the noise.

4. Halo effect

When we experience the halo effect, we form an opinion of someone or something based on a few traits – positive or negative. We may look at these traits and assign some credit or weight to them.

I’ve seen this show up with product managers who have received high praise for their work that has positively affected the business and customers. Because of this praise, they are often looked at as the gold standard, may receive additional resourcing, not be subject to the same reviews as others, and it sets the tone that they’ve got it all right, all the time. It may even allow for other features to be overlooked because the feature that was praised is generating such positive feedback that it overshadows the negative reviews of others.

Remember that one success story in product doesn’t predict the rest of your future as success. The product manager is likely to repeat behavior, have certain expectations of stakeholders, and seek out feedback from customers who are power users of their praised feature rather than look at the complete picture and seek out challenging points of view. There’s a bit of hubris in this bias that needs to be recognized (and checked).

Combating halo effect

• Product managers need to remain objective and data-driven in their decision making. We can’t rely solely on the first positive impression and the behavior or data that was tied to that initial success. This means we need to seek out more data points as we mature our products. Lean on avenues like user interviews, usage data, and competitive intel to help paint a better picture of the overall landscape. Very important – review, share, and discuss this data with your team.
• When we establish plans, setting clear success metrics and criteria will keep us on track to help us check ourselves and be transparent with our partners and stakeholders on our progress and learnings.
• Always encourage constructive feedback on the work you do with your team. This keeps you humble and focused on what it is you need to get better at going forward rather than look in the rearview mirror at what you did best before.

5. Sunk cost fallacy

Sunk cost fallacy is when we follow through on something that may not be right, or even doomed to fail, simply because we’ve already invested in it.

Confession: I myself have absolutely had this bias show up in real life for me plenty of times. For some people it could be that they’ve booked a vacation and an unexpected event happens. For others, they’ve sat through 30 minutes of a show or a concert and realized it’s pretty bad. No matter the situation, we want to feel like the investment – financial, emotional, mental, energy and time – was somehow worth it.

Pause for a moment after reading this paragraph and reflect: What feature have you invested any of those things above into that you felt you needed to “see through”, even though you already knew it was going south? How did that make you feel? What happened as a result of you not pausing or pulling the plug? What was the cost after that?

The problem with sunk cost fallacy is that it leads us to make decisions that will most likely cost us more in the long run. This mindset does not yield outcomes we were hoping for in the beginning.

For product managers, this happens a lot when we continue to invest in a feature we’ve built that gets low adoption rates. Or, the team has worked on a new feature they believe will sunset another, but it receives pretty bad feedback. It also shows up when we were handed something to be built rather than bringing our ideas to the table on what should have been built in the first place. In this case, the belief is that stakeholders will want to see it through.

This bias is a deep one. We talk a lot about imposter syndrome in our business. Our natural imposter syndrome as product managers makes us want to prove to others that we have it all together. Aren’t we also expected to have the answers to everything?

Something in us is telling us to pause or stop, but the image of it all forces us to keep going.

And, breathe.

Combating sunk cost

• We need to name it, and we need to state we’re experiencing some sort of feeling here that’s making us uneasy, as it’s challenging a previous decision.
• Next, come to peace with the fact that the costs are sunk. Unfortunately we can’t recoup the time we’ve already invested. Money however is something we can likely catch up to later on with better decisions that lead to more profitable features. We need to focus on what’s best going forward with respect to time, energy, and overall investment.
• Focus on data that will help you move forward in a different direction, and allow you to set new goals towards an objective for the company and/or drive a better customer experience. Often, a whole group of cross-functional partners are so lost in the thing that was agreed upon that they fail to see the possibilities in other features that could drive more growth and retention because they’ve ‘always been there’.  An example here is allowing yourself to look at traditionally underserved areas of the product that are seeing spikes in usage and increased retention and digging deeper into untapped potential.

If there’s one thing I’d like readers to walk away with from this article, it’s that acknowledging our biases is the first step towards combating them and making better decisions that drive more positive long-term outcomes. This is true in product, and in real life.

Working on combating the biases above will also increase the time you spend with customers, create stronger feedback loops, build relationships with your team, partners, and stakeholders, and strengthen your overall product gut. We’ll tackle the importance of your product gut in the next article, along with the rest of the biases.

Discover more great content on Cognitive bias