The journey to product experimentation by Split

In this spotlight session sponsored by Split at #mtpcon London 2022, Tu Nguyen and Shannon Cassidy discussed how product managers can create a culture of learning and experimentation for themselves and their organisations.

Watch this video or read on for key highlights from the talk.

Using feature flags to make informed decisions

Feature flags can be an effective tool to control what features you choose to roll out to users to solidify your hypothesis. It gives teams the opportunity to turn off and on specific features to different user groups and see what effect this has. Shannon explains how feature flags can provide psychological safety when rolling out new features. Additionally, it provides enhanced control over feature decisions.

Upon creating this foundation where you feel safe and prepared for any procedure, you can then experiment with your hypothesis-based testing, and iterate, Shannon says. It takes practice to embrace the concept of a feature flag and evolve the team around that. “It’s all about testing small incremental features more frequently,” she adds, “it takes time for a team to get used to this,”

From looking back on your feature flag, you can assess what small features had an impact and step into your customers’ shoes, Tu says. “You can understand what they’re looking for and what they don’t want,”

When using feature flags, Tu explains how you must ask yourself: what problem am I solving, is it reducing friction, and what opportunities are there? Additionally, she explains to always “Look at the data, ask questions, and understand what the story is behind all of the data points presented to you.”

Embrace the experimentation challenge

Because there is much less risk with using feature flags, it becomes much more straightforward to experiment with new hypotheses that may not be fully informed. To further develop and embrace that experimentation and growth mindset, Shannon lists six key practices to inherit:

• Be curious and ask questions
• Understand the impact of releases
• Start small and keep iterating
• Stay humble and embrace mistakes
• Once you have all of the data, make decisions based on it
• Collaborate and gain feedback from users and the wider team

Take the journey together

Tu explains how it’s important to bring the whole team on the journey as you become more experiment-led. She says to follow these components when embarking on this journey:

Leadership: Be present, supportive, and informed. Join in on celebrations of milestones, commiserate failures, and join the team in highlighting success.

Planning: Use your product roadmap to create and explain changes. This can help to track your progress with a feature.

Maintenance: Always be looking to reinforce measurement and reassessment based on your feedback and insights.

Management: And finally, relentlessly ensure that change remains on track, technology, and people. Bring your key findings to the wider team to obtain their support. Telling your story can go a long way, Shannon says.

Following these components will ensure that you are following the process of building a culture of experimentation and learning within your team and the wider organisation. The data and insights that you gain from experimentation will speak for themselves.

There’s more where that came from!

Explore more #mtpcon London 2022 content or use our Content A–Z to find even more product management insights.