The benefits of squad-led discovery — Jim Morris - Mind the Product

How can product managers get more work done by doing less? On this weeks’ podcast, we spoke with Jim Morris, Founder of The Product Discovery Team about the benefits of having a team-led approach to research, and how that can be an effective way to collaborate effectively and improve your product over time.

---

Featured links

Episode transcript

Lily Smith: 0:00

You know, Randy, I always wanted an excuse to do less work. And now I have one.

Randy Silver: 0:05

Why what’s happened has the singularity suddenly occurred? You know, the nerd Rapture?

Lily Smith: 0:11

No, I should be so lucky. Instead, we talked to Jim Morris, product discovery coach about how the team should be doing more discovery and not just leaving it all to UX and research. But

Randy Silver: 0:24

you run product and growth, you’re not a UX or researcher.

Lily Smith: 0:29

That is true. But then I tend to work in startups where a researcher is a luxury we don’t get to have so the idea of sharing the customer research is music to my ears.

Randy Silver: 0:39

You know, I tend to work in larger companies and research are dedicated to each team is a luxury to most of them. That sounds great, but it also sounds really challenging. So let’s hear all about it from Jim.

Lily Smith: 0:58

product experience is brought to you by mind the product.

Randy Silver: 1:02

Every week, we talk to the best product people from around the globe about how we can improve our practice, and build products that people love.

Lily Smith: 1:09

Because it mind the product.com to catch up on past episodes, and to discover an extensive library of great content and videos,

Randy Silver: 1:17

browse for free, or become a minor product member to unlock premium articles, unseen videos, AMA’s roundtables, discount store conferences around the world training opportunities and

Lily Smith: 1:29

more minded product also offers free product tank meetups in more than 200 cities. And there’s probably one near you. Hey, Jim Morris, really lovely to be speaking to you today. So we’re going to be talking all about discovery. But before we get stuck into our topic, it’d be really great if you could give our listeners a quick intro to who you are. And just a potted history on your background. And yeah, what you’re doing today.

Jim Morris: 2:01

Sure. So I’m a product discovery coach, and I work with product teams, and product leaders at Fortune 100 companies all the way to like tech startups, to entrepreneurs, and I help them work through their ideas. So my background is, as a software engineer, turned into a product person, you know, I did product management as the business minded engineer. And then as that field developed, I ended up managing a team of product managers, and then helping build the product of power reviews. And then we ended up selling to a different company. And then, after about 10 years in that business, I started out on my own as a coach, the last six years, I’ve been working with a variety of companies to help them work through their ideas, it gets smarter before they work harder.

Randy Silver: 2:54

We always talk about discovery and say, give advice to people about, you know, get your developers really involved to help that have them fall in love with discovery and participate in it. But you’re the first person I’ve ever met, who’s gone from being involved on the dev side and managing dev organisations to make discovery your primary purpose in life, how did you fall in love with it? What what happened?

Jim Morris: 3:19

Yeah, when I was looking to see what I would do next, it became really clear that building something has gotten easier with Amazon Web Services and frameworks to really consolidate a lot of functionality. You don’t have to build anymore. But what we build was still super immature, mainly because the people who were making decisions were super immature about their decision process, you know, basically relying on educated guesses, even all the way up until today. And engineers wouldn’t be doing that in their respective careers and professions. And so, at that time, I really thought this is this is a place where I can add value, where I can help people develop their ideas. And to me having engineers involved is crucial. Some of the most important companies in the world used by billions, whether we like them or not, were started by engineers. And nowadays, with agile, or any form of development, we often see engineers as implementing our vision. And this is super dangerous. It’s also a lot of stress for product teams, or product managers or designers to put all of the burden of finding the next idea on themselves. And if we share it with our engineering counterparts, it’s super great for morale, and it gives us that spark of innovation when we least expect it.

Lily Smith: 4:44

So one of the things we chatted about before, when we were talking about during the interview, was the difference between squad lead research and UX lead research. You kind of intimated this before, so Tell me what you mean by this? I mean, I’m, I’m assuming it’s got something to do with those engineers getting more involved in research.

Jim Morris: 5:08

Yeah. So one of my, one of my clients kind of nicknamed that squad lead research, and I just jumped right on it. Because people asked me, Well, do we need a professional user experience person on our discovery squad? And the answer is no, it can help quite a bit. But sometimes it’s an excuse not to do discovery. So I’d say, you know, a no excuses approach is this squad led research, and the expectation for all of my squads. And I think just for squats in general, is from a product manager, a designer, that discovery engineer, whoever else is on that that core team, that you will recruit and interview people, users yourselves. And that you can learn this. There are great resources out there, I teach people all the time. And with practice, you learn to ask good questions, you make really good solution tests so that you have a good experiment to ask questions about. And so the UX our lead research would be a user research person kind of leading the research and setting the hypotheses. The issue, of course, is that we don’t believe the things that other people say, or do, if we’re not there. It’s just like a human nature thing. And so you can have the greatest UX are in the world. But if they create research, make a great document present it, it won’t have the impact, as would these three or four folks actually interviewing the users directly. And of course, it takes more time. But insights stick more deeply in the stick over time. So you know, six months later, after interviewing three users a week, these, these folks on your discovery team are going to be incredibly knowledgeable about your customers. And they have direct control over which solution is technically feasible and viable. There’s a lot of pressures that the product team have in order to make a great product. Often the UX, ours don’t have that pressure. And so with that pressure comes the crucible that makes a really good product where you balance everybody’s needs. From the UX or point of view, are they really going towards an OKR key success metric, the same as yours. And so you know, being aligned is difficult with your in different silos and organisation. So I think we’re squad led, it’s really about the team doing it themselves, believing they can do it. And if the company’s perspective should be that, where I’m aligning all of my success metrics and expectations is where they should also be talking to users, it doesn’t need to be split out. And I would say the same for data and analytics, honestly.

Lily Smith: 7:37

So as a as a UX person, and that are the kind of research the person in charge of research in that sort of dynamic and that sort of situation. Do you become more of a coach for the rest of the squad? On how to do good research? Or just, you know, upskilling? The rest of the team?

Jim Morris: 7:59

Yeah, I would say, definitely a coach, definitely somebody who can provide some centralised services, such as maybe user recruiting, helping generate a panel from all the customers you have, or helping them actually recruit and find users, making sure incentives are paid, there are some logistics to recruiting, that can be friction and blockers for teams. So I would say, reducing that friction and impediments, certainly coaching people listening in to interviews, helping them improve what questions are asking what hypotheses are making. And if you can, you could allocate user research, user researchers to a team to be on that team. They get it, they’re doing the research together, interviewing folks creating the prototypes, if the user researcher has the same priorities, then that then that would be almost a form of squad led research with a UX researcher on it.

Lily Smith: 8:54

And I guess, kind of aligned with that, like, how do we measure how good we are at Discovery, you know, as a, as a squad lead team, if you’re not used to regularly doing research, then you’re gonna start off at a particular level of expertise, and then hopefully improve, but how do you measure kind of like where you’re starting from, and then see, like, the improvements over the time and kind of measure? Well, hopefully the improvements over time?

Jim Morris: 9:23

Yeah, this is the, you know, say like, this is what they asked for Twitter, and you know, wrong answers only. And it’s, you know, a number of people I’ve talked to and, you know, number of prototypes I made, and it’s like sort of counting stuff, right? That’s sort of the wrong answer. I do want to see that there is activity, there is no substitute for talking to folks. So you do need to have something like you’re talking to two folks a week, three folks a week, you know, see evidence would be that teams are rejecting some ideas with some evidence. So teams that validate every idea, honestly, like shouldn’t really bother with discovery. Right, what are they learning, they’re only getting usability feedback, meaning the person was a little bit confused. So I’m going to tweak the content, the location, the calls to action, okay, but I want to see value oriented changes, whole flows, reorganised. Whole concepts, tried and then failed. And I want to see the teams exploring the edges. So if you’re managing a team, and you see them, you know, having all this great success, you might think, Well, my team is so great, I would just get sceptical of that greatness and say, Look, I want you to actually push this because in design, there’s very low risk. I don’t want you to be unethical or illegal. But you should be able to push all these push people down sort of the spectrum of ideas without having to build it without all the engineering. So I would say failing, I want freshness to the insights. So maybe we don’t have to count how many people were talking to. But it’s, it’s about that user test that was three months ago, and you keep talking about it, I’m gonna get the sense that you’re not doing any more of it. So freshness, and you slowly move away from customer anecdotes to a sound a consensus, right, you’re not gonna get all every user to agree. But if I’m using one customer and using one customer, and you’re using one customer, or just battling over anecdotes, right, this is the sort of the another wrong answer. In discovery. So what I did consensus, well, five out of six people, six out of eight seminars I ate like, these don’t sound amazing to statisticians. Right. But it, it actually these are meaningful insights. If you do enough user testing, you’ll know that their ideas that people don’t find consensus in their ideas that are worth it. So I think that’s sort of, you know, the consensus concept of freshness, the rejection, it all ends up leading to humility, which is something I learned very late in my career, James that basically say, maybe I don’t know this, but I have the tools to find it out.

Randy Silver: 12:03

I like that it’s I’m not sure if Willie will get this might be too American a reference. But sounds like the Trident principle. You know, four out of five dentists and seven enemy.

Jim Morris: 12:13

Tried in principle. I just published an article on this. And I was trying to name it and consensus, the only word I came up with, and I’m sure there lots of people who won’t like it. But we’ll have a lot of great names for stuff and product. So

Randy Silver: 12:27

let’s go see where we go with it. So you heard it here. First, folks. Jim, what’s stopping most teams from working this way? Why aren’t they doing it already?

Jim Morris: 12:38

Live, it’s a natural? Well, there’s a natural aversion to taking your ideas and sort of being vulnerable, giving them to other people to actually let them click through it. I’ll see a lot of teams just show it on screen and they’ll narrate it like a sales presentation. Don’t you like my prototype? Don’t you like my idea? This is just indicative of. And of course, it takes about five minutes to make this thing clickable. Right? So this is not a matter of time or energy to do this. It’s a matter of understanding one, it’s a radically different concept to let somebody else go through your prototype. I also think just in terms of reading appstore reviews, quantitative surveys sitting in on call centre conversations, having one on one conversations with users, there might be an aversion for folks that are just used to working in their office environment. Right. And I think, really, there’s that just general aversion for like office workers to talk to outside folks. There’s finding users, sometimes people think, Well, I have to get them to sign these documents. And they’re really complicated. And, but there are lots of great online services in various countries in the world to find users. So I can find users in about 48 to 72 hours, right with one of these services. So I think blocked some of the blockers, just not knowing what these services are and having exposure to them. Some of it’s just a simple part about budget. But more than more than willing to spend the time and energy of of 10 people on something. But when it comes to finding $500 to talk to 10 users, it’s not there. It’s just mind blowing. So budget, previous experience, sort of an unwillingness or a fear. And then, in some b2b companies, there’s a cultural aversion, cultural aversion, meaning the salespeople don’t want you talking to their customers. They think that customers believe in a perfect world where everything is, you know, one way and all most customers, especially now in b2b software, really understand that. If somebody’s interested, in my opinion, that’s probably a good thing. Not just sort of listening to me dictate features, but working through ideas. So there’s some internal headwinds that block folks. And then the biggest one is just managing To fill the team’s time and their, their backlog by just dictating features, right? There’s just no little room. I’ve given you something to do, please go do it. Why are you telling me this may not be a good idea? I gave you a deadline. And so roadmaps, but deadlines are inherently incompatible with this bottoms up information finding journey that might make you pivot either a little or a lot.

Randy Silver: 15:27

When you say managers, are you talking about stakeholders, product managers, delivery managers? Or is it even possibly engineering managers who are also working on the refactoring or tech debt and things like that?

Jim Morris: 15:41

Yeah, I mean, it’s rare that I find engineering managers who want 100% of backlog, you know, I give it about 20, to 40%. So if I work backwards through your list, you know, good engineering managers will make the case to keep 20 40% of time fixing stuff as we go, you know, and, but then, outside of that you’re looking at direct managers say, you know, group product managers or heads of product, you know, line heads of lines of businesses, CEOs, who maybe might be managing in a more traditional sense, like, they don’t really understand software. So here’s the thing, here’s a deadline, let’s put it together. And that’s their management style. Whereas the other style of generating a good success metric or two, takes time could take 10 or 15 hours. To figure out what a good metric is, it could take another month or two to measure a baseline. And then, of course, you’ve got to wait for the team to make some progress against that baseline. That just sounds really unromantic the way I bought it. But this is building a long term scalable way to increase your probability of success. Otherwise, you’re looking for hero culture, where there’s someone in the organisation who is going to constantly be dictating and helping dictate and hope, dictate and hope. Whereas if you can stand up to that, at any of those levels, engineering manager, group, product manager, head of line of business CEO had a product and say, Look, in this 60 minutes, let’s talk 10 or 15, about success metrics, instead of all 60 solutions, let’s talk 10 or 15 minutes about problems. And I try to create this little conversational virus of data and problems on opportunities that my team members can talk to their bosses and their bosses and their bosses. So the design lead toxic design managers, engineering, the toxic engineer leader, because design leader says what about this great idea? Design folks are all full of ideas, right? But do they have the context of what the problem is? And what the metric is? Usually no. So the team, as they develop context gets confident that they’re going to get to the right place, they just need a way to represent that to the rest of the business. I don’t know if we got off track there. But But yeah, I get it kind of excited.

Lily Smith: 17:57

No, I think it’s a really, really good point. And I’d love that idea of, you know, the team being able to kind of influence upwards by sort of being more aligned in how they’re working together. I guess, kind of on that topic as well. Sort of a couple of questions, like just bringing it back to discovery, again, and the kind of the teams working through taking more ownership of the discovery process. How do you kind of build that culture of discovery within the business so that that becomes acceptable? And sort of as part of that, like, one of the things I’ve done in teams that I’ve worked in before is included some business metrics, or some of the success metrics around discovery activities in order to ensure that like, we get it, because, you know, managers love to see numbers and data and oh, yes, we’ve agreed to do this discovery activity, and you’ve done it well done, tick the box kind of thing. So is there something in that that enables and empowers the teams to take ownership of that discovery and kind of make it part of the, the outputs so that management teams who are slightly more output driven can can see some progress there or can kind of feel some something tangible that’s going on?

Jim Morris: 19:25

Yeah, I think that I could see something about customer touch points. You know, I talk a lot about solution testing. So talking to real people live, it’s important for discovery to be meaningful, no matter how much I read about surveys and give surveys, right, so there’s gonna be a balance to the input. So my first thought about metrics on discovery would be to a metric for my team, honestly, you know, was two customers or prospects a week. So I was in a b2b business, and we had 1200 clients and it felt very overwhelming. I said, To my product managers, designers, heads of engineering, a director of engineering, you will just like I will meet to customers or prospects a week from now, until forever. And I worked very tightly with my counterpart who ran customer success. I said, anytime you’re having meetings, I’d like to include these people. And what we did was kind of pair people up. Right, so one of my director of engineering was paired up with one of our particular customer success manager. Right, so then that person could just schedule the director of engineering in and there was no asking, everybody had access to each other’s calendars, and you just got on it. So to me, that was something I held them accountable to, and I held the other parts of the organisation accountable to like that one. That’s a long term play. I don’t know what the result of that is, right. But I knew that I’m not going to have these only educated guesses coming out of my team. And if I get them to talk to more than a couple people, I’m gonna get beyond the anecdote, I got two problems. I got anecdotes and educated guesses that I’ve got to really overcome, then, any particular discovery activity, whether it’s making user experience map or finding problems or making prototypes, these get harder to count, because they’re very lumpy. I might do a prototype one week, and I might just tweak it 20 times. You know, I think Marty Kagan talks about making 20 changes a week. Absolutely, you can do stuff like this, some of the changes are just me talking to you. Some of them are us talking to customers. Right? So the process of change if I don’t know if you can measure it in discrete things. But I do have at the end of each cycle of five or six users, my teams do a readout. And what did you discard? I want more than I want a nonzero number of discards. So there’s something you can count. If you’re not discarding anything, then you’re not exploring enough. I don’t think you’re gonna find innovation that way. So you may not be my independent team. If you only discard things, you probably have a decision making problem. Yeah, we have to have input, we have to make decisions with imperfect information, right, we’re not going to get perfect information. And once we get it, it’s can often be too late. So I would say those are two starting points, I want to see some failure. And I want to see that you’re really talking to folks in in different ways, or shapes or forms. And maybe there’s, you know, some way you can summarise all the non live interactions, actual reviews, you know, online surveys once a month, just just so I know that you’ve gone through them. And we’ve covered because some people will communicate to your company. And that’ll raise some red flags, and someone’s got to listen. And the product team is is a primary listener in any company.

Randy Silver: 22:49

Jim, it’s a lot of what you’re talking about makes sense. But I’m curious about the best structure of a team for all this. Now you’ve got your certainly your engineering, your UX and your product leads or meeting with customers, hopefully the rest of your engineers are doing on a regular basis as well. But should you have a dedicated user researcher on every team? Or is there an agency model? Do? Do teams get to the level of competency and expertise themselves doing is what what’s good look like for you?

Jim Morris: 23:23

Yeah. A dedicated user researcher is not required, I would say 85% of my team do not have one. And what I need on each team, though, is somebody to take responsibility for loving humanity. And that in bringing that humility, it’s sometimes people expect this to be the designer always. And I find this to be unfair. But if I have a set of personalities that I’ve associate, brought together in a in a core discovery team, and I find that they’re all a little bit hesitant about users, I might just need to break that up and take this team that’s got three folks that love users, and start to intermix them. And you’re thinking, Well, what if this person doesn’t know about this technology? What about this person doesn’t know about this domain? Honestly, if no one really cares about users, it just won’t happen. Right? There’s a we are people. And as leaders, we cannot create every desirable trait in every employee. So we can start with things that lead us in the right direction, whether it’s the pm or the designer, sometimes it’s the engineer who says, Look, I really want us to do our homework before we generate this work for the engineering team. So I’m looking for at least one person to be really strong in terms of getting out there and making sure they do the discovery activities talking to users. Another aspect is often people talk about engineering leads, and I might just back it off to be an engineer. In fact, I sort of prefer hands on keyboard engineers, mainly because their facility with the technology is direct. They know it so well. This is especially the case in AP And app engineers, iOS and Android because they often deal in the UI at the same time they’re programming. And then I think the one of the issues with lead engineers or solution architects, is they can get called into a lot of meetings when things happen. So in terms of attendance and time, someone who has purview over lots of engineering systems tend to be less available. So they may be the most knowledgeable person, we think, Oh, we have to have the most knowledgeable person on the team. In actuality, engineers are fantastic and asking their colleagues about stuff they don’t know. You know, engineers are constantly googling everything. This is sort of the joke on engineers, right? Like, well, what do they really know? Well, it’s because we don’t want to memorise all this stuff. Right? And we’re more than happy to not know something. Business people don’t actually get that luxury, right, you often walk into a room and saying, I don’t know can be incredibly vulnerable. engineers typically don’t have this problem. And so I’m more I’m almost, I would prefer to have a hands on keyboard engineer who doesn’t know every part of the system. But who knows somebody that they can ask. Yeah, that would be sort of some key information, concepts.

Lily Smith: 26:10

Just thinking about if you have multiple different squads, doing their research within their own kind of squad teams. And this might be an issue already actually with them, the way that research is done by the research team, assuming that that’s the other way that it happens. But what’s the best way for larger organisations to share their discovery learnings? Because I imagine that that’s, you know, there’s lots of people talk, if you end up in that great situation where you have lots of people across the business talking to customers, you’re getting lots of insights, but like, how do you share them across the whole business? And, or, you know, is that even necessary?

Jim Morris: 26:57

I think how you reach decisions as a culture is incredibly important. And if you have folks that are using a discovery based approach, where they’re trying to get external information, to me, being an advocate or evangelist for customer feedback, is great. And so large organisations that want discovery to take hold that want this external feedback engine to keep going, they will make sure there’s time for teams to report back about their findings. Now, let’s say that we have a decision we’ve made about a particular concept, and we show the before and the after that we’re gonna talk about the process, we might just think that there was an enlightened set of tech workers that that made this decision, right. And so what I want folks to add to this read out process, the sharing is a little bit of how you got to this decision. Right? And that means that for the PM, or the UX designer, it might be something like, Well, I thought it was this and it turned out to be that looks super vulnerable, like, whoa, you didn’t come up with this, maybe I shouldn’t have you as my employee. I mean, these are people’s jobs. And they want to come across as confident. We need a culture that says, Hey, you don’t have to have the answer. But you need to have the mechanism to find the answer. And if celebrating the mechanism, the process. And when I say process, I don’t mean overdoing it, right, I just mean that I found a non intuitive learning because I gave three solutions to the user. And I had them compare and contrast them. As opposed to, I told my designer to come up with one. And then we all kind of argued in meetings about where to put everything on this one design, and we showed one solution to a customer. That’s a very common process. But it’s not going to lead to the insights that that coming up with three solutions would, because you’ve got three different ideas because you’ve stretched yourself and your teammates. And it could be that this engineers idea beats the pm and the designers idea. And this happens all the time. And this is actually what drives long term humility is when you allow your idea to be backed off against other ideas in your team. As I often tell the designer in discovery, we will get into your business quite quite, quite far. It shouldn’t be less stress for you, because we’re not expecting you to solve the world’s problems. But we also expect you to incorporate other ideas, you know, later in the process, and you normally would

Lily Smith: 29:16

love that. And so one of the things that you mentioned before as well was that people pay far too much attention to what customers are saying, rather than what customers are doing. So what are your top tips for kind of understanding what customers are doing rather than what they’re saying?

Jim Morris: 29:39

In addition to getting a strong signal of demand, in a situation where the user believes are actually giving you something of value, we can make sure that we’re going beyond just having a conversation about something. Right? So we can we can talk about things that user feels really confident about, like we were talking about creaky chairs and how to fix it. You and you were describing the situations and the stories about creaky chairs to me, one with Randy one with you just now. The stories are 100% believable. The fact we could almost hear the creeks in the audio, then what users or stakeholders typically do is look, I want you to get the Aeron chair. The Aeron chair is like the number one chair. Right? And it has this cool mesh and all the other startups have the Aeron chair and I’m dating myself, but this was what happened. And people that we would literally go to failed startups and we would auction and buy the Aeron chairs, and if anybody’s ever had an Aeron chair, you realise it’s impossible to configure to your body, that the levers are impossible to use. It’s just, it’s baffling. It’s the most expensive baffling chair that everybody wanted. So that particular solution wasn’t that great. So what do you need? Well, here’s a foam chair, here’s the chair, we had your knees forward, here’s the Aeron chair, let’s get a couple solutions. Now those are perhaps expensive ways because have to make all those chairs. But in software, we can we don’t have the hardware physics limitations, we can make multiple versions. And as you experience, the multiple versions are just like you were sitting on different chairs, you’re going to be able to give me feedback about the farm chair, the Aeron chair, the one where you’re leaning on your knees, I’m going to believe that feedback more than you just hypothesising about these chairs without sitting in them, just like user interfaces without experiencing them. We try to get as close to that reality as possible. And the due portion of that is them experiencing that prototype rather than you just talking to them about a solution. Or Honestly, my my clients, just talk to folks about the problem, go into the hole of development, and come out with a solution. I call it the product discovery valley of death. Because I do just some discovery, talking to people open ended conversations, but they don’t get that do phase where someone gets to experience their solution. Why? Because it’s really vulnerable. And they say they don’t have time. Right? Yeah, well, you’re gonna build something. It’s super risky to build something that you never showed to folks, or show me the last minute where it’s just usability. And I’m not going to like to make any major changes. So that was my say versus do.

Lily Smith: 32:15

Jim, it’s been so great talking to you today. Unfortunately, we have run out of time. But thank you so much for joining us and imparting some of your discovery knowledge. I love the idea of Scotland discovery. And I’m sure lots of other people will as well. So yeah, thank you very much.

Jim Morris: 32:34

Thank you. It’s been been fantastic to hanging out with you too.

Lily Smith: 32:47

The product experience is the first and the best podcast from mine the product. Our hosts are me, Lily Smith, and me Randy silver. Louron Pratt is our producer and Luke Smith is our editor.

Randy Silver: 33:02

Our theme music is from Hamburg baseband pow. That’s P AU. Thanks to RNA killer who curates both product tank and MTP engage in Hamburg, and who also plays bass in the band for letting us use their music. You can connect with your local product community via product tank, regular free meetups in over 200 cities worldwide.

Lily Smith: 33:23

If there’s not one near you, maybe you should think about starting one. To find out more go to mind the product.com forward slash product tank