From pandemic to pingdemic – Building through uncertainty

From pandemic to pingdemic – Building through uncertainty

In this #mtpcon London+EMEA keynote, Randeep Sidhu, former Director of Product at Babylon Health and the NHS Covid App, describes the challenges of building the UK’s NHS Covid app, a universal product with critical deadlines, limited discovery, and high-profile stakeholders.

Watch this video or read on for key highlights from the talk.

In brief

• Build from the bottom of the pyramid
• An easy-to-align framework or attitude, beats a more detailed model
• Create quality assets to allow your users and stakeholders to share feedback
• Use radical transparency to manage stakeholders
• Finding your user’s true priorities is key to achieve success

“A product you build will kill someone”

Randeep opens his keynote by recalling a conversation from early in his product career, where he was told that “a product you build will kill someone, a decision you make will kill someone. Your job in product is to make sure your conscience is clear if that happens.” He describes this as a reality of working in healthcare and health tech, and something even more acute when you work in government. The nature of that world and the nature of those environments, he tells us, is quite disruptive.

Randeep, describes his recent role as director of the NHS COVID App as “an abusive product relationship”. One which he started in June 2020, at a time of great uncertainty in the height of the Covid-19 pandemic. “It was a bit scary back then”, he says “we didn’t know how it was transmitted fully, how you caught it fully, or actually how you protected yourself”.

Three days into consulting on why an early iteration of the app was shelved after just three months (and the team disbanded), Randeep was asked to build its successor. He explains how the people who spoke with him “didn’t have that much healthcare experience, and particularly didn’t have experience at the bottom of the pyramid, [helping] the people who are most impacted”. As a result, he felt compelled to take on the role. With experience building healthcare technology in Rwanda, and having worked with underrepresented populations in the UK his decision seemed clear. “I thought, you know, my communities are the ones being killed by this, we need to do something.”

Reassessing goals

Randeep needed to focus on building a product that would break the trends of transmission. He needed to focus on making it anonymous, building it universally, and using data science, but he first chose to better understand the emotional drivers of risk, which would lead to users downloading the app and changing their behaviours.

Unlike the first team who assumed people would use the app like a public healthcare service, one that “everyone would do out of the nobility of their soul”, Randeep’s team’s research indicated that people peoples’ priorities were different. They found that people would, in fact, prioritise the concept of protecting their loved ones, then themselves, and only then the public, which allowed them to better focus their development goals.

Building a universal product

“The NHS is kind of universal” Randeep says, “it’s for everyone […] to build healthcare, you genuinely have to help everyone. And that’s quite a tough challenge.” He describes how the team had a six-week deadline to release the pilot product, which meant there was not have time for a full discovery process, instead, Randeep devised a product framework borrowing from Maslow’s Hierarchy of Needs.

Rather than focussing on the key survival requirements, they looked at the needs of deploying an effective product:

• Physical needs: The app must be small enough to be downloaded across all devices and should not require a permanent internet connection for those at the bottom of the pyramid who do not have access to that.
• Cost: The app should be free so everyone can use it.
• Safety: Minority communities may risk being exposed to the police or feel unsafe within the home based on tracking abilities of the app, for example looking to the LGBT+ community, domestic abuse survivors, asylum seekers, so it should be anonymous.
• Purpose: Protecting loved ones, then oneself, then the public.
• Practical: Supporting 12 languages that represent over 96% of spoken word in the UK, using a low reading age to allow for all literacy levels and adhering to Web Content Accessibility Guidelines (WCAG), allowing for audio, visual and mobility differences.

The framework illustrated how the core needs of users are accessibility-related, and how the most important factors are often overlooked. Randeep describes how, “as product people, myself included, I often focus on the top of the pyramid, I focus on driving behaviour change and what I’m doing, forgetting that I need to get it in the hands of people often.” This framework allowed the team to proactively work to meet the major needs of users and better prioritise bugs and problems based on how it impacted these larger goals.

The fight for quality feedback

“COVID is a disease of inequality,” Randeep says, “if you’re poor, or if you’re an ethnic minority, you’ve got twice the death rate, and if you’re black it’s four times”. He describes how the team had these statistics but, like any medical product, needed to test and validate that what they were building was effective. To do this they looked to increase feedback making “parallel processes for getting feedback and identifying risks”.

The Isle of Wight was used as the first pilot location as it’s a Conservative Council and therefore easy to test, it also, says Randeep, “actually lives up to its name racially […] not only that, it’s atypical even for white populations. You’re testing a contact tracing app about people connecting in an area where people are quite far apart and don’t kind of interact very often.”

In contrast, the London Borough of Newsham was densely populated, had the highest impact of COVID deaths, and a diverse population. It was here Randeep wanted to get a new pilot underway but the process of doing so was, he says, “an actual personal fight against some political pressure”, on which led him to pitch to the mayor of Newsham directly, highlighting how his team were working to better serve people that were “overlooked and not heard”.

He describes having major successes, gathering feedback from proxies like charities and organisations — as they couldn’t do focus groups in all these different areas, they spoke to the people who spoke for them. Proxies would actively go out and solicit information from staff or users and filter it so that Randeep’s team could quickly identify where to their energy. What was also important in gathering feedback, says Randeep, was creating quality assets and multiple ways for people to share their insights.

Pre-release testing

Due to the speed of development, the team opted to test during the design phase, learning with each iteration. “The key thing here is we didn’t always change every single thing that was highlighted […] the unicorn interface is almost unachievable, but we always made sure we documented what we didn’t build”.

While testing-in-design was largely a success, Randeep acknowledges that the development of the app was, at many points, “a dumpster fire”. He gives some examples, such as bad translation agencies which led to instances where a negative Covid test would tell Polish-language users they were in ‘a good village’ or when a submenu called ‘moist’ would appear on the Welsh-language version of the app. This forced Randeep to get family members involved to support proofreading and checking.

Stakeholder management

Launching a “universal” app means that “everyone” is a stakeholder and Randeep describes how problems came about when “somebody would announce a roadmap publicly in Prime Minister’s questions” or tell the public about features that were still in development or, worse still, hadn’t been built yet. This led him to move to a place of “radical transparency”.

Randeep built a Change Review Board, getting people to share feedback via a form and removing the assumption that things were being hidden. “When someone expects you to move back to prepare for a punch, you just lean in” he says.

The outcome

Ultimately, says Randeep, “I do think it was successful”. He describes how the app was the fastest downloaded app in UK history. There were 28 million downloads and more than 250 million people checked in. “It did help and it was proven to reduce the spread of COVID” but, he admitted, it has its issues. He cites data published in science journal Nature, which details how at the time of launch there were 1.9m infections in a 3-month period. “We stopped up to 900,000 of them, we reduced the peak in December hospitalizations by half.”

To close his talk, Randeep highlights his two regrets from his time working on the COVID app:

1. That he didn’t fight harder against “political reasons” to get vaccine responsiveness into the app — mapping and being able to share how variants are changing.
2. That he didn’t take MNEK up on an offer of a gold press-on nail manicure before demonstrating the app to Matt Hancock (you’ll need to watch the talk in full for more on that )

“The first app worked or didn’t work because of the kind of emotional intent,” he says. “So when we built this app, we were trying to make sure we focused on the thing that mattered the most, and that was protecting your loved ones.”

Gain Access to More Content Like This

Ordinarily, premium content like this can only be accessed by Mind the Product members. However, we felt the messages in this talk were so important that we decided to make it available to everyone in the community. Upgrade now to discover more amazing content like this!