BY The Product Experience ON FEBRUARY 9, 2022

Making your roadmap work hard for you is one of the critical skills of a Product Manager. Find out how Jeremy Levy, CEO at Indicative, leverages customer data to inform his product roadmap. In this episode, we also discuss who should be responsible for the data strategy in the business and why it’s so much easier to get it right these days.

Listen to more episodes…

---

Featured Links: Follow Jeremy on LinkedIn and Twitter | mParticle | INDOCHINO case study at Indicative | Jeremy’s ‘Analytics are the key to optimising the product roadmap’ feature at Mind The Product

Episode transcript

Lily Smith: 0:00

Hey podcast listeners. My poor podcast co host, Randy silver is currently hiding away with COVID. So I’m flying solo. Don’t worry, though, he assures me he’s unmanned. And we’ll be back next week. So today, you folks will have to help me out. What starts webinar ends with a P and has a lot of work in the middle of it. Yep, you all guessed correctly. It’s a roadmap. And this week, I had a great chat with Jeremy Levie CEO of indicative about roadmaps leveraging customer data and also data strategy. I found the talk super helpful and I hope you do too. The product experience is brought to you by mind the product.

Randy Silver: 0:49

Every week, we talk to the best product people from around the globe about how we can improve our practice, build products that people love,

Lily Smith: 0:56

because they mind the product.com to catch up on past episodes, and to discover an extensive library of great content and videos, browse for

Randy Silver: 1:05

free, or become a mind the product member to unlock premium articles, unseen videos, AMA’s roundtables, discount store conferences around the world training opportunities.

Lily Smith: 1:18

mining product also offers free product tank meetups in more than 200 cities, and less probably one. Jeremy, thank you so much for joining us on the product experience podcast. It’s so lovely to be talking to you. Thankfully,

Jeremy Levy: 1:34

it’s great to be here.

Lily Smith: 1:36

Sadly, we’re missing my co host, Randy. But I will do my very best today. So before we get started, it’d be really great if you could give our listeners a quick intro into yourself and your background and your kind of experience with product management.

Jeremy Levy: 1:55

Sure, happy to. So I’m a serial entrepreneur at heart. And I’ve started three companies. The first business I started was a company called Meatwad, back in 2006. And we were the first location based mobile dating company for iPhone and Android, we effectively pioneered the notion of using real time, location and proximity to find you matches within your within your criteria. We sold that business to match.com. I see. And then shortly thereafter that we started a company called x phi x defy was the first company to be an enterprise mobile CRM. And so what we did effectively was provide the messaging and the lifecycle messaging capabilities for mobile for enterprise businesses. So our customers included everyone from Disney, to Sephora, and so forth. And then we sold that business to IBM. I was the CTO and co founder of those businesses. And then after that, you know, we started indicative and indicative was really the culmination of my experience leveraging data in both the consumer business and b2b enterprise business. And our mission effectively was to enable product and marketing teams fail to more easily understand the customer journey, or in other words, how customers are interacting with their products. And what are the factors that drive outcomes that they care about things like engagement, reducing churn, understanding, definitely the why customers are doing things. And then recently, but last month, we sold that business to M particle, where today I’m a VP of product.

Lily Smith: 3:24

Amazing. Thanks, Jeremy. And today, we’re gonna talk about roadmaps, which is one of my favourite topics, and definitely one of the most favourite topics of mine the product in general, I think product managers get very excited. Well, all the nerdy ones that I know, really, very excited about talking about roadmaps. But it sounds like you’ve come from a more technical background with your kind of start in life being more on the engineering side and CTO roles. So tell me about your journey of discovery of roadmaps and what it looks like when you first started working in tech companies and kind of how that evolved over time.

Jeremy Levy: 4:07

Well, you know, my experience really starts from this notion of the business that we created. And all of those began, essentially not a product roadmap, or even a product of the matter, really just an idea with a dating company, we knew there was an opportunity around taking advantage of the iPhone and the Android coming into market. And we had an idea for mobile dating company. And so when we started building product, it really was let’s just start building something that allowed you to view profiles have, you know, exchanged messages and so forth. And by doing that, we got our product in market as quickly as possible. And it really was just a number of pivots. We were constantly experimenting with what worked, what didn’t work, we’re able to move very quickly to the point where we established I don’t even want to go so far as to say product market fit because we were this was even so early, but where we established what looked like the essence of Have a business that would make sense where people were engaged, we are growing. And that’s when we started really thinking about what does the roadmap look like? And then we started thinking about okay, well, now that we have some momentum, what do we want to do to optimise on this growth is that optimising acquisition was that focusing on engagement, entering in new markets, new channels, and so forth. And so that was really the first time we were proactive about thinking about what we were building and the inputs to that. And, you know, as I think about sort of how I build roadmaps, the inputs in those days were very much more an art than a science in the sense that there was no such thing really, as analytics platforms, the way we have today, there wasn’t an easy way to get metrics and data on what you’re doing. And so it really was for us, let’s we have an idea. Let’s see how quickly we can prototype it, let’s put it in market and see if it works or not.

Lily Smith: 5:50

That’s really interesting. So and it’s very similar to my experience of working in startups where I think when you’re first starting out, you’re you know, your focus is just to drive engagement, and really kind of find that product market fit. And I haven’t found a need for roadmaps in that very early stage in the same way that when you’re then optimising and getting into that stage of growth and, and kind of scaling a business, then it becomes much more important to have a roadmap. So in terms of getting to the best possible looking roadmap and the roadmap that your company needs, like, what does that look like for you today?

Jeremy Levy: 6:35

So I think, like, to your point, I think a lot of it depends on the stage. And so when appropriate, what the best possible roadmap I think is a little bit subjective, the way I sort of think about it is it really does start with understanding organizationally, what your objectives are, and even higher than that organise you what your vision is. From there. You know, in our experience, we’ve taken everything from qualitative feedback from customers from user sessions, to the quantitative feedback that we collect by way of product analytics, use tools like product board and JIRA, to help synthesise rank some of that feedback. And then honestly, it’s a combination of combining that into a roadmap that not necessarily is 100% Customer informed and not necessarily 100% vision form. But that melds those into something cohesive along with us organisational objectives. And those objectives are typically something along the lines of expansion into a new market, increasing engagement, revenue, obviously, a new line of business. And then, you know, to the extent that there is a like, perfect version, like, I don’t think such a thing exists, you know, every roadmap that we’ve ever put together, I think of as being a living document, there’s no such thing as like a roadmap planning exercise, and then you rubber stamp it and it’s good to go and you just go execute it. We look at roadmaps very much is something that is evolving, in the context of the business. Never have I defined a single roadmap, and then that ended up being the final product. But what I think is so valuable about the exercise is that going through that process of being thoughtful about what you’re going to build, and thinking through what the output is, helps you evolve it over time to have a quote unquote, perfect version of the roadmap. The reality is, everything changes and you get smarter along the path. As you break down tasks, requirements and milestones. You update that roadmap to reflect what you know, but I think as a planning exercise, it’s incredibly valuable, because it forces you to go through the motions of understanding what it is that you should be building, and what the impact of that could be.

Lily Smith: 8:40

And okay, so in terms of like gathering that data, we talked about qual data, and quant data as well, or kind of analytics data, and then also having a part of the the kind of the business goals as well. How do you get to a point where your data is sort of informing what’s going on to the roadmap? Like, what are the prerequisites, I guess, for having like a roadmap session and going okay, this is what we’re focusing on in this quarter? Like, what are the things that you need in front of you?

Jeremy Levy: 9:13

Well, you know, I think it begins with establishing at a high level, what are your objectives, because if you come into a roadmap meeting without some semblance of what we want to achieve on some timeline, then roadmap meetings tend to just be a brainstorming session of random ideas that don’t have any cohesion. And so I would say normally, when entering into a roadmapping meeting, you have to before stepping in or having that first session, you have to establish what the criteria is that you’re trying to achieve. And again, that might be when you double our revenue. It might mean proxies for that by way of we need to enter a new market or we need to solve a problem that we have, maybe it’s clear technical debt for that matter, but there needs to be at least a high level objective, that that roadmap is in further and stuff. And by the way, it’s not uncommon to have multiple roadmaps at our company, we have a feature roadmap, we have a technical debt roadmap, and all those eventually, you know, get merged to some extent into a single execution pipeline. But really, you can’t have the conversation and absence of knowing what your objectives are, as a business, or at least as a unit or product inside of a larger company.

Lily Smith: 10:24

So that’s really interesting that you have different roadmaps for technical debt, and for kind of feature building and presumably, like tied to driving business outcomes, where there are other types of roadmap and do you have then have, like teams that are aligned to deliver on each of those different roadmaps that you have?

Jeremy Levy: 10:44

Well, you know, my companies have always been relatively small. And so we’ve always struggled with a sense that, actually, to that, and even larger companies, there is a finite amount of resources that we have ability to execute on anything. And so, you know, generally the way I thought about it is there is a technical roadmap that aligns sometimes with you know, organisational objectives, and maybe just technical objectives. And then there’s the feature and value roadmap, and obviously, there can be others. But a lot of the work in formulating that roadmap is around figuring out what you can’t do, because there just inevitably are not enough resources to execute on all of those in parallel, or even serially to be able to get done everything you want to do. So a lot of that is an exercise and figuring out what actually is important, then I think there is a bit of an art and figuring out how you can make sure to clean out your technical debt, align your architecture with where you want to go from a technical perspective, alongside building new product. You know, I have a particular I don’t want to say sympathy. But I’ve have a particularly soft spot for making sure that we don’t maintain a lot of technical debt. Because having been someone who has built features for our product, I do understand the trade offs between making sure that we make sure that the technical environment under which we operate in is one that is as efficient, efficient and conducive for engineers to build quickly and efficiently. That allows us to play a little bit more fast and loose on the product in the future side by knowing that we can execute as quickly as possible.

Lily Smith: 12:12

Yeah, so important as well, in terms of attracting and retaining good engineering talent to know that they’re not just going to be sorting out a load of technical debt. Yeah, for sure. Great. So one of the things that I found really interesting from a blog post that you wrote, for me in the product was a quote, or a kind of stat from a McKinsey Report around organisations, organisations that leverage customer data and internal decision making outperform peer companies by 85% in sales growth, and 25% in gross margin, which is fantastic. I think everybody wants to do that. I would argue that that’s not a fantastic idea. And I think one of the things that’s really interesting, in your approach, as well, that you mentioned in that blog post is that you really kind of work to leverage your customer data in order to inform your roadmap. So how do you go about doing that? Who is doing that? Is that the product manager that’s doing that? Or is the whole team kind of responsible for looking at and interpreting customer data and understanding where their opportunities

Jeremy Levy: 13:34

are? You know, I think leveraging customer data specifically for a roadmap is really just one piece of the puzzle. Again, it comes back to it is an art and a science. And you know, there the data fallacy, or the quantitative fallacy is real in terms of you know, just because you have data does not mean that the data tells the whole picture. data, whether it’s for roadmaps, or for informing any decision is one part of the puzzle when it comes to informing decisions that businesses are making on a day to day basis from a roadmap and a roadmap basis. So you know, the way we’ve thought about it is, at least from a roadmap perspective, there are numerous stakeholders who participate in any given roadmap planning planning session, typically, it’s a head of product, certainly engineering, and other product resources, even potentially sales, marketing, and so forth. Bringing quantitative data to the table for those meetings is something that I believe in, but it’s not something I look at religiously, I think of data as one tool in a product leaders tool belt. And you know, I say this knowing full well that I’ve started data companies, I work for a data company. It’s not the tool, right? And so you have to make sure when thinking about building your product, there is as much a notion of a gut check in terms of I know what the market wants, or I know where the market should go or where I’m trying to take the market in so much as what the data I have that I’m collecting directly from my customers. Even what customers are telling me they want in terms of their features, you know, I would say, it’s important to remember that you shouldn’t follow any of these methodologies religiously, right? Like, customer feedback. But I don’t advocate for 100% customer driven roadmap, I advocate for a data driven roadmap, but I don’t advocate for an exclusively data driven roadmap, and exclusively data driven approach would be like pulling, you know, pulling teeth, right, it was a decision you make needs to be quantified and justified and tested. And I think that in, you know, in this environment, we can move much more quickly by using data to inform decisions not entirely to make every single decision. And you know, we talk about that, in terms of being data driven. But really what data driven means is using this information to inform the decisions.

Lily Smith: 15:49

Yeah, I think that’s a really important point, I feel like there was a period of a while ago, maybe a couple of years ago, where it was all about data driven decision making. And it kind of almost went a little bit far.

Jeremy Levy: 16:07

I think there are areas where that still makes sense. So for example, at Meatwad, the dating company that we had four features that we would build, a lot more of them were based on, this is what we think would be valuable to our users. And we would test that ask users and so forth. But, for example, conversely, on our acquisition channels, where we had landing pages, where we had onboarding, things that we could very directly measure and optimise that we tested religiously, that was literally like, let’s A B test the colour of this button, let’s move it up, move it down. And we had built a framework to allow us to do that very, very easily and efficiently and without a lot of friction. And with that thing’s like optimising a half a percent or so on a conversion path meant real dollars in terms of whether it’s saving it from the cost of acquisition, or higher ROI in terms of getting more customers in the door. So like, in that case, we love the pure data driven approach, because the the start and the end of what we were doing was really clearly well, well defined, whereas adding new features, it was sort of like the great unknown, like, we didn’t know what was gonna happen when we added this new, you know, what I’m thinking of is like a gifts feature in our product, we just didn’t know what was gonna happen, we hope users were gonna use it. But on the act was this path. We knew where step a and were Step B, and sub C and D were, and those we could optimise till the cows came home. And we did really, really well with that approach there. But it was again, in that case, it was really a case of the the right tool for the right job and the right situation.

Lily Smith: 17:40

Yeah. And it’s interesting, I think, with sort of bringing discovery into the picture as well, like, Where does discovery sit within your roadmap? Do you? You know, do you include it as like, part of each piece of work that you’re planning on doing? Or do you do discovery on a separate track and almost have like a separate discovery roadmap as well?

Jeremy Levy: 18:08

Well, I mean, answer your question slightly differently, which is, how do we get from a conceptual idea or an objective into like, an actionable roadmap? Yeah. And part of that process, the way I approach it is, I think it’s sort of like the altitude we have off the ground. And so you know, for example, when we start a roadmap discussion, we start that at, you know, 100,000 feet off the ground. And we come up with an outline structure for that, and then we break and then we go and research each of those. And then we come back, we try to break that down to the 75,000 foot level, the 50,000 foot level, to be able to refine it to something that we can actually execute against. And by continuing to refine, evolve and iterate on that, what we’re basically doing is we’re removing the complexity. And by removing the complexity, we can understand where those tasks are, and what’s involved with them, without having to start the beginning, building a complex spec and do a bunch of discovery and research and so forth. But by quickly iterating it and evolving it, we get down to an actionable roadmap that we can actually do something against, as well as a dates and projections in terms of timelines for what we think we can hit, because we’ve iterated all the complexity in the unknowns, added it over a period of time. So I think probably the answer your question is, it’s iterative. And it happens both in the context of a roadmap planning meeting, and then it also happens offline, when doing your research to contribute back to that meeting.

Lily Smith: 19:32

So one of the things that I find really interesting as well with, you know, really focusing in on the customer data side of things and making sure that you are, you know, as you’ve said, You need to have your sort of gut feel or your instincts about things and also your business plan and your business goals. But on the customer data side, I think One of the things that’s really important to get right and understand is like how all of your customer data connects. And in this day and age, it can be quite difficult when you have all of your different marketing stack, and all of the different tools and all of the different ways in which you kind of interact with your customers, and then your product, data and stuff like that. And I believe it can all come together in a customer data platform, although I have yet to see it myself. So what’s your sort of experience with this? Like, do you need in order to really, really leverage that customer data and get the most out of it? Do you need all to come together in a customer data platform or in a, you know, in a single, like, have that single view of the customer? And how much more valuable is that, then, you know, it having those silos of customer data and having to sort of piecemeal put things together to build a picture of the customer in a more manual way?

Jeremy Levy: 20:59

So, you know, I think we sort of benefit from today in the sense that we’re operating in a revolution of the way these products operate from a data perspective. And so the answer your question is CVPs, I think the smartest companies are leveraging CDP’s, because there’s a number of ways in which they help accelerate customers time to value for data. And the reality is that covers things like governance, like proper data collection, helping to fan out that data from activation protection to third party tools. And what I think is so interesting, in terms of the way I think about the data ecosystem, and CDP’s is that CDP’s to some extent, also help you future proof your data infrastructure, there’s a lot of talk about sort of the modern data ecosystem. CDP’s complement the notion of the modern data ecosystem, in the sense that if you want to have your data in a redshift, or a BigQuery, or snowflake, CDP’s pay really, really well, from that sense, in terms of helping you get data in there helping you activate it, and help maintain good governance, it’s become incredibly more difficult to operate these these data warehouses, because things like GDPR, data compliance, and so on, are things that every company has to some extent, implement from scratch, leveraging CDP’s, cuts all of that out of the way, and still provides your marketing team and your product team with the ability to self serve data in a really clean, governed way. And when I say cleaning governed, we’re talking about the data maturity scale, right? Like as companies become more sophisticated, as businesses become collecting more data from more types of channels, and more customers and so forth. Having data clean, govern properly and mature to the end allows teams to work more efficiently. But really, what I recommend to companies today is whether or not you need a data warehouse or not whether or not you you’re thinking of a CDP or so, the CDP is a way off the bat to allow you to leapfrog or accelerate your entire data stack. And to do that without having to really, as you mentioned, cobbled together, you know, two or three dozen, you know, half a dozen or so different types of products to get a full suite of tools. The way I think about it is a CDP combined, the data warehouse provides you with literally the universe of options of things you would want to do with data and the protection that you need as a company from a governance perspective, rather a compliance perspective, and the ability to have data that is clean and actionable without having to spend a substantial amount of time in terms of synthesising that, or cleaning up to figure out whether it’s product insights or prioritisation of that data,

Lily Smith: 23:34

which kind of sounds like the utopia from a, from a product managers point of view, you know, when you have a lot of customers, I guess for, you know, for some businesses in that sense, and probably particularly more DTC businesses and products. But

Jeremy Levy: 23:54

it’s still work though, right? I mean, like, I don’t want to make it sound like it’s, and all of a sudden, you just plug it into works. Fundamentally, even if it’s clean, even if it’s well organised, it’s still really hard to get value from that. But you know, the things that are the things that are most interesting about our ecosystem, is that the things that used to require engineering teams and data engineering teams to accomplish are now no longer things that we have to worry about. And that allows marketing teams product teams to own more and more of the portico data platform, because data really is a function of product, right? It’s not some third thing that happens in your business that you need, that you know, historically needs a separate team to manage and get value out of, if we look to the future. Its product teams and marketing teams who want to be able to easily activate and own that data not have to think about how do I run a Python script to join my you know, my user table and my events table and and push that out to Salesforce? You know, those days are over?

Lily Smith: 24:52

Yeah, and you guys can’t see me but I’m frantically nodding. Yeah, so if it’s productive Marketing teams, as you say, that are, you know, the ones that are really like wanting to own the, or benefit from a really good data architecture in the business? How do you see this kind of playing out in terms of like, who like who should own this within a business? If you don’t have, you know, if you’re not a very big business that has like a whole data team and a head of data or whatever? does it fall on, like the lead engineer? Or does it fall on the head of product or whatever, to really understand the sort of data requirements across the business and put together a bit of a data strategy? And what would a data strategy look like? Sorry, I feel like that’s about 10 different questions.

Jeremy Levy: 25:52

No, I got the gist of it. So I answered this question differently than I used to. And I used to say you do need like a centralised data team with a very, very thought through cohesive data strategy. And my answer has really evolved over time, as the barrier entry for data tools and affected the democratisation of Data Tools has become more available. So for example, I think today, and I mean, this truly that product teams should be responsible for data, I don’t see why product teams can own the data warehouse, I don’t see why product teams can own the CDP or the data stack. And that just means that consumers of that are another aspect. It’s another aspect of product. And you know, the reality is like setting up, a CDP certainly does require some engineering effort setting up data warehouse as well. But these tools have become easy enough and the glue between them. But we don’t really need data engineers, we still need them, but we don’t need them to the same level we used to. And the example I always think of is, which is frankly, the reason we started indicative is because when we were thinking about doing product analytics and meet wha we had to cannibalise our engineering team, literally to write code to get us any sort of metrics that we wanted to understand. Today, I need to do that today, you can plug and play with these solutions in an incredibly easy and quick way. And it’s like it follows the the evolution of most technology, initially, this technology is expensive, really hard to use, and therefore it’s only available, whether it’s to the enterprise, or the most well funded companies. And today, you know, you can have a powerful CDP you can have a powerful data warehouse, even if you’re just starting a company. And you know, the next question that people typically ask me lately is, when when’s the right time. And, you know, I, I think about that, also, in the context of how easy that technology is to use today, if you’re starting a business, and you don’t have a data strategy, or if you don’t have a plan to collect that data, it’s gonna be very hard for you to understand what’s working and what’s not. In the context of your business. I’ve heard, I used to meet companies who would say, we’re too young to worry about data analytics, we just have to build our product. And I get that. But the reality is, once you find something that’s working, you’re going to start asking questions around your data. And if you don’t have that already thought through, you’re going to be a little bit in trouble. And again, the barrier to entry for this stuff is basically zero today, that you can set up high quality data instrumentation, in the CDP data warehouse today, even if you’re one engineer, and you can have that in a way that allows you to scale it to the in the same way you would if you were IBM. That’s what’s so incredible about where we are today, that the barrier to entry for this technology is effectively been reduced to zero. So you can have the same quality of technology that the largest most successful companies have today, when you’re just starting out. And so my answer is, if you don’t have a data strategy today, you should have one.

Lily Smith: 28:47

Yeah. Hallelujah. Fantastic. So it’s so great to hear you say that I, you know, I, myself am at exactly this point in my journey of sort of understanding, you know, what, what we need in our business, actually, at our collective with a, you know, a data strategy, basically, because we have all of this data, and it’s not connected in ways that, you know, we can really, really deeply understand our customers, like we, you know, there’s high level stuff, and it’s, that’s fine. But yeah, I completely agree with you, but it’s difficult stuff, right? It’s like hard to, you know, that you’re right, the barrier to entry is much lower, and it’s much easier to get access to these really amazing tools. But really kind of understanding them and understanding what you need for your business is quite tricky, and it’s, you know, as product managers, a lot of us have kind of fallen into this role. Or, you know, if you’ve just started out you don’t necessarily have like skills or have have been trained in, in data in the way that perhaps like a data engineer or a data analyst may have is that, would you have any kind of advice for people in that situation where you’re thinking, Yes, this is all resonating with me, I totally want a single view of the customer. I need to be able to leverages customer data to build my roadmap. But I like literally have no idea where to start or, you know, I just need some help, like, what are your sort of top tips for getting started on thinking about what your data strategy should look like?

Jeremy Levy: 30:35

I have tonnes of tips. I think I think it comes back to sort of what you said a moment ago, which is, as a product leader, you’re not you but you know, you think about there’s all these things I could be looking at, help me understand where I should focus. And so I sort of have two thoughts around that. One is with data more is not necessarily better. And oftentimes, less can be more in that sense. So tracking everything under the sun, and having everything available to look at and experiment with doesn’t necessarily make your teams more productive. So when I when we talk about data, democratisation and indicative, I think about helping to expose the right sets of metrics and data to people from a democratisation perspective that goes along with this notion of proper governance. The second thing is, you know, there’s 1,000,001 Things you can track and you can think of, but helping to identify what the KPIs that matter to the business are, is really the starting point. Because like, you could look at registrations, you can look at conversions, and so forth, pageviews, and all these things. But a lot of those typically tend to tend to fall into things like the vanity metrics category. And so it’s really important to look at what are the key things that actually matter for your business. And so, you know, if you’re a SaaS business, we’re talking about retention, we’re talking about acquisition, we’re talking about conversion rates, you know, and again, it falls into the area that you’re focused on, but really identify what those key metrics are that define your business. So you know, at indicative, you know, we would talk about engagement. And from us engagement, we would define as only we only look engagement for people, for example, who had been tenured for at least three months, because we are a premium SAS product, people can start using our product and then drop off. So we only wanted to look at engagement, for example, for people who had been with the product for at least three months. And again, I mentioned that really just to think about be very particular about what are the metrics that matter, because there are a lot of vanity metrics that literally don’t make a difference. And you can throw them up on a dashboard on your plasma in the office. And you know, and look at them, but it’s really narrowing down to those one or two, or maybe three key metrics that actually matter, and from an impact perspective. And then once you do that, and this is sort of the Folsom that line, like if you measure it, it will improve. Once you identify what those KPIs are, that allows you then to start aligning your thoughts around what can I do to help improve it? And that begins sort of that ideation when it comes to that roadmap planning meeting, or when you’re just planning your next set of features, in terms of thinking about what can I do to impact those numbers? Because again, like data is critical. But more doesn’t always mean better. And less is more oftentimes, when you’re thinking about democratising it.

Lily Smith: 33:19

That’s great. And I, I also like what you’re saying about that kind of segmenting your customers or like understanding at what point you know what type of customer you’re targeting as well with retention, like it’s not just everybody, it’s after the three months like they’re the people that we want to measure the retention for. And I think that’s a critical piece, which is often not really spoken about when we talk about focus on what impact you want to have. And we talk about revenue and engagement and retention and churn and blah, blah, blah. But we never talk about for this specific cohort of customers or segment of customers. Because that really, really helps you narrow down your, the scope of what you’re looking at and be able to impact that more directly. Because then you can understand, what are that what is that particular segment doing? And why are they behaving that way.

Jeremy Levy: 34:15

And that’s also a tool that we tell our customers about, if you identify a segment of users who have a positive behaviour that you want to replicate, it’s great to be able to segment your user base for that KPI by different types of users. Maybe it’s long tenured users, maybe it’s, you know, people acquire from this source, maybe it’s that demographic, and so forth, to find those threads inside the KPIs that where things are really working, and then use that as a way to figure out what are those customers doing what’s unique about them, that you can then use to impact other users, like the traditional example that everyone talks about is the example of Twitter right like when Twitter launched. I’m sure this is a made up story at this point, but they had incredible drop off and they looked at They’re long tenured users. And they what they discovered was that users on Twitter who followed, you know, five or 10 other people ended up being the people who stayed around for a long time. And so that insight of finding that group of highly engaged users, and what are the common behavioural characteristics associated with them, allow them to go back and say, Well, you know what, let’s modify our onboarding process. Let’s modify our onboarding process, so that by the time you’re done, you’re following five or 10 people so that you look like those highly engaged users. That’s, you know, I love that example, because it shows how with a little bit of ingenuity, you can figure out things that are working within your user base that you can apply to the larger population to to help grow effectively those key KPIs.

Lily Smith: 35:46

Yeah, I suppose that’s a really classic example of having a very, you know, a roadmap or a plan that’s very aligned with like, or leveraging that customer data.

Jeremy Levy: 36:00

Yeah, we have a customer into Chino, who did something similar. They had their checkout process, only by examining their conversion checkout process, and looking at the individual threads of people who are having success allowed them to replicate some of those changes to their wider audience. And I think they ended up with something like a 20% lift on their checkout process, which is incredibly, incredibly large change from a, from a from a conversion perspective.

Lily Smith: 36:26

Wow, that’s amazing. Great win there. I bet they were very happy. So we are sadly running out of time, I felt like I could talk about this a lot longer. And I just have one more question for you. What do you think people get wrong with planning their roadmap or with try, you know, that sort of trying to be more data driven?

Jeremy Levy: 36:49

You know, I think following any one methodology religiously, is always a mistake. So you don’t want to be exclusively making decisions purely on data, you don’t want to exclusively be making decisions based on user feedback. I think what customers tend to do is is or what users tend to think about is finding the right blend of the type of data data. I mean, ambiguously, that helps inform their roadmap. And I think about this in the context of the right methodology, the right tool for the job, you know, following any of those religiously as if it is absolute, we have to do it this way. It’s usually a mistake in my opinion,

Lily Smith: 37:29

that I can wholeheartedly agree with. Jeremy, thank you so much for joining me today. It’s been fabulous chatting with you. And I think there’s lots of great advice in there for our listeners to take away and get inspired by.

Jeremy Levy: 37:45

Thanks so much Lily, it’s great chatting with you.

Lily Smith: 37:57

If you want to learn more about roadmaps then check out all the content are mine the product this week, we have a special focus on this tricky artefact and loads of great content to help you find to your own. And now from the art of the P it’s goodbye for me and see you next week. Haste, me, Lily Smith and me Randy silver. Emily Tate is our producer. And Luke Smith is our editor.

Randy Silver: 38:31

Our theme music is from humbard baseband power that’s pa you thanks to earn a killer who runs product tank and MTP engage in Hamburg and please based in the band for letting us use their music. Connect with your local product community via product tank or regular free meetups in over 200 cities worldwide.

Lily Smith: 38:50

If there’s not one Nagy you can consider starting one yourself. To find out more go to mind the product.com forward slash product tank.

Randy Silver: 38:59

Product Tech is a global community of meetups during buy in for product people. We offer expert talks group discussion and a safe environment for product people to come together and share greetings and tips.