Viewing ChatGPT through the lens of product sense

Viewing ChatGPT through the lens of product sense

Senior product manager Todd Lewandowski evaluates the hot new trend with a method we’re all familiar with.

---

Right now (June 2023) ChatGPT and all generative AI is undeniably hot. It’s dominating headlines and water cooler discussions. On its own, it’s an amazing piece of technology. But we are all still figuring out what it can and can’t do, much less the long term implications.

Product managers, with hands in both business and technology, are right in the middle of it. Bosses and coworkers are confronting us with a simple question:

“Should we use ChatGPT in our own company’s product?”

That’s what I aim in to help you answer in this blog post.

Address the Emotional Concern

First, acknowledge the underlying concern behind this question. There is a whole lot of peer pressure in tech. It’s especially intense if the pressure comes from executives or investors. But what drives peer pressure is actually an underlying fear of “Are we being left behind?” or “Are we not being innovative enough?”.

Second, ChatGPT is quite an amazing tool. Just play with it yourself and see! Immediately taking a critical attitude (to any new technology) won’t match their positive emotions towards it and is likely to scare them away from further interactions with you.

So before you provide a logical answer be sure to address their emotional concerns first.

The Right Way to Make Decisions

However,  your internal alarm should be going off right now as both of those ways of thinking are inappropriate. As product managers, we don’t make product decisions based on authority (peer pressure or a rich/powerful person) …  because authority is subjective and can fade. Nor do we decide based on a tech’s own novelty or capabilities … because that’s working forward from the technology instead of working backwards from customer problems.

Of course, the “right” reasoning is:

Is this the right thing for our users? Does it further our company’s mission?

If you pose it this way, no doubt your coworkers will wholeheartedly agree.  Plus it’s easy to point to “Put customers first” as the #1 value for every company.

Our answer could still be “yes” or varying degrees of “yes”, but now our arguments are much more sound. Now we’re using logical/quantitative arguments like opportunity size or emotional/qualitative arguments like “Will this help the user.” Now our arguments are based on customers who actually pay us. Customers are numerous, and as a whole, have needs that are a lot more consistent than any single stakeholder.

Working backwards through product sense

Every product manager is familiar with “product sense.” We use it in verbal discussions,  writing PRDs, and of course when interviewing. Traditional product sense goes from general to specific:

1. Mission, vision, strategy
2. Users, segments, customers
3. Pain points, opportunities, use cases
4. User flows, mockups, MVPs

Sometimes an interview will switch things up. For example: “You’re a product manager at Apple or Google. An engineer approaches you with a new technology to detect smells. Should we include it in the next mobile phone?”

This is still product sense, but in the opposite direction. So we just reverse the framework:

• Describe the technology: How it works and its limitations
• Abstract to use cases: How someone would use it to solve a problem or gain something new
• Abstract to segments: What kinds of people have these use cases and the frequency and intensity of those uses.
• Abstract to mission-vision: How this gives overall value to society and any larger trends.

The only difference with traditional product sense is that along the way you are comparing it to your own company’s product-market fit. There will never be a 100% fit as it is new technology. If there was, your team was probably considering it before it went mainstream. But of course, the more overlap the easier it is to say “yes.”

(0) Limit the Scope

Product sense begins by asking questions and making basic assumptions. These limit the scope and simplify the problem.

• Limit to a single mode. So just text-input to text-output, as in ChatGPT in its first iteration. Exclude text-input to image-output (Midjourney, Stable Diffusion) and output to video, databases, or any structured format.
• Exclude AGI (artificial generative intelligence) ie a sentient robot brain. Impractical, fanciful, and too many philosophical, psychological, and social issues.
• Limit to American English, though other languages are possible.

(1) Describe the Technology

• Input: Freeform text area input. User inputs a question, request, or comment in natural prose.
• Output: Text response in natural prose. User consumes directly or copy-pastes elsewhere.
• Format: Web app for users to create and read past conversations. API for partners to integrate with their own software. With the right resources, companies can build their own large language models.

Nuances

• Memory: Remembers questions/answers in same conversation up to 50 interactions.
• Reactive: Only responds to a direct user prompt. Never proactively tells you of anything, ie a push notification.
• Non-deterministic: Doesn’t give the exact same answer every time. There is randomness, though the overall message is the same.

Limitations

• Limited knowledge based on its training data. Does not have the most current information like news and weather.
• Predictive based on paths of high probability. Does not inherently reason through logic, concepts, or proofs.
• Hallucinates sometimes to make claims that are not true. Does not share sources, so can make claims that seem authoritative but are based on spurious sources.
• Generic output. Simple prompts lead to simple answers. Better answers require more detailed questions.
• Lacks spirit. Can display facsimiles of emotions, but is not a human. Users may project human emotions onto it when it is actually just an algorithm.

(2) Abstract to Use Cases

The tech is text-in text-out which is language as a representation of information. User functionality is then manipulating information (the content the text represents) or manipulating language (the symbols that represent the content).

• Searching: Query for information like a search engine but returning in natural human prose. By extension, providing product support.
• Summarizing: Gathering main points and condensing data from a large source. By extension categorizing, classifying, or evaluating.
• Editing/rewriting: Modifying content to a different style or tone. By extension, translation to another language.
• Coding: Creating compilable computer code.
• Content generation: Creating fiction or non-fiction.

(3) Abstract to Segments

Some appropriate segments  (not mutually exclusive):

• Basic tech literacy: Limited to people with a smartphone/laptop and a stable internet connection.
• Knowledge workers: Those who discover and manipulate knowledge for their work. Teachers, scientists, software engineers, product managers, and similar white collar employees.
• Professional communicators: Those who primarily communicate with others for their work. Writers/creatives, media, support agents, content creators, and similar.

(4) Abstract to Mission-Vision

Automate Basic Tasks

In general, AI replaces the easiest, most common, and most rote knowledge tasks. It ‘cuts costs’ in a basic sense of time and effort. However, job displacement is not complete, and it depends on the repeatability of the task as well as the economy-of-scale of the organization.

For example, while automation is common in warehouses and factories, they still have a large amount of workers. While there are a ton of consumer packaged goods, we still go to farmers markets and local restaurants. So labor displacement is truly only marginal for those extremely rote knowledge tasks that are already machine-like.

Even knowledge workers doing very specialized tasks do some rote things. In this sense, AI acts as your writing and search assistant. It supercharges your workday to make it easier, faster, and more productive.

Springboard for Quality

AI’s output tends to be generic, which makes sense as it is based on the most common trends in language and thought. It can be refined with prompts (though you have to know how to prompt), but ultimately won’t produce anything as creative as James Joyce’s Ulysses.

On the flip side though, AI can create rapid prototyping upon which to see if a creative idea has potential … and then human experts refine it to make it better. AI thus helps you ‘grow revenue’ (happiness) in a general sense. As I like to say it:

“AI is the beginning of great writing. Not the end.”

Make a recommendation

Finally, it’s time to make a recommendation. Compare reverse-product-sense to your company’s product fit on those four criteria: (1) technology (2) use cases (3) user segments (4) mission vision. The more overlap (especially on the latter criteria), the easier it is to say yes.

But rarely is the answer black and white. Consider the degree of fit (very strong to very weak), the category of fit (better on some parts not others), and the perception of fit (how to spin it to execs or customers). For a start:

• Good fit: Yes, here’s how it super-charges our current product-market fit
• Lukewarm fit: Yes, but maybe later in the future after they improve X, best for a big customer, or we might use a vendor that uses X.
• No fit: No, but we are at the bleeding edge of doing something else very cool and innovative (addresses the underlying emotional fear).

Final thoughts

Finally, this method of   “reverse product sense” is applicable to any new technology, not just ChatGPT and generative AI. Feel free to use it for your next big thing or in your next interview.

Is this framework perfect? Not necessarily. But frameworks come easily to us product managers. They also provide some basic structure and are a great starting point for further discussions. Best of luck to you in the future.

Discover more great content on AI