This week, the Defense Innovation Unit (DIU), the division of the U.S. Department of Defense (DoD) that awards emerging technology prototype contracts, published a first draft of a whitepaper outlining “responsible … guidelines” that establish processes intended to “avoid unintended consequences” in AI systems. The paper, which includes worksheets for system planning, development, and deployment, is based on DoD ethics principles adopted by the Secretary of Defense and was written in collaboration with researchers at Carnegie Mellon University’s Software Engineering Institute, according to the DIU.

“Unlike most ethics guidelines, [the guidelines] are highly prescriptive and rooted in action,” a DIU spokesperson told VentureBeat via email. “Given DIU’s relationship with private sector companies, the ethics will help shape the behavior of private companies and trickle down the thinking.”

Launched in March 2020, the DIU’s effort comes as corporate defense contracts, particularly those involving AI technologies, have come under increased scrutiny. When news emerged in 2018 that Google had contributed to Project Maven, a military AI project to develop surveillance systems, thousands of employees at the company protested.

For some AI and data analytics companies, like Oculus cofounder Palmer Luckey’s Anduril and Peter Thiel’s Palantir, military contracts have become a top source of revenue. In October, Palantir won most of an $823 million contract to provide data and big analytics software to the U.S. army. And in July, Anduril said that it received a contract worth up to $99 million to supply the U.S. military with drones aimed at countering hostile or unauthorized drones.

Machine learning, computer vision, facial recognition vendors including TrueFace, Clearview AI, TwoSense, and AI.Reverie also have contracts with various U.S. army branches. And in the case of Maven, Microsoft and Amazon among others have taken Google’s place.

AI development guidance

The DIU guidelines recommend that companies start by defining tasks, success metrics, and baselines “appropriately,” identifying stakeholders and conducting harms modeling. They also require that developers address the effects of flawed data, establish plans for system auditing, and “confirm that new data doesn’t degrade system performance,” primarily through “harms assessment[s]” and quality control steps designed to mitigate negative impacts.

The guidelines aren’t likely to satisfy critics who argue that any guidance the DoD offers is paradoxical. As MIT Tech Review points out, the DIU says nothing about the use of autonomous weapons, which some ethicists and researchers as well as regulators in countries including Belgium and Germany have opposed.

But Bryce Goodman at the DIU, who coauthored the whitepaper, told MIT Tech Review that the guidelines aren’t meant to be a cure-all. For example, they can’t offer universally reliable ways to “fix” shortcomings such as biased data or inappropriately selected algorithms, and they might not apply to systems proposed for national security use cases that have no route to responsible deployment.

Studies indeed show that bias mitigation practices like those that the whitepaper recommend aren’t a panacea when it comes to ensuring fair predictions from AI models. Bias in AI also doesn’t arise from datasets alone. Problem formulation, or the way researchers fit tasks to AI techniques, can also contribute. So can other human-led steps throughout the AI deployment pipeline, like dataset selection and prep and architectural differences between models.

Regardless, the work could change how AI is developed by the government if the DoD’s guidelines are adopted by other departments. While NATO recently released an AI strategy and the U.S. National Institute of Standards and Technology is working with academia and the private sector to develop AI standards, Goodman told MIT Tech Review that he and his colleagues have already given the whitepaper to the National Oceanic and Atmospheric Administration, the Department of Transportation, and ethics groups at the Department of Justice, the General Services Administration, and the Internal Revenue Service.

The DIU says that it’s already deploying the guidelines on a range of projects covering applications including predictive health, underwater autonomy, predictive maintenance, and supply chain analysis. “There are no other guidelines that exist, either within the DoD or, frankly, the United States government, that go into this level of detail,” Goodman told MIT Tech Review.

For AI coverage, send news tips to Kyle Wiggers — and be sure to subscribe to the AI Weekly newsletter and bookmark our AI channel, The Machine.

Thanks for reading,

Kyle Wiggers

AI Staff Writer

This article is part of a Gaming Insights series paid for by Facebook.

When Wildlife Studios, one of the largest mobile gaming companies in the world, was ready to capture a new international audience for one of their largest games, “Zooba: Battle Royale,” they looked toward Pacific America — and specifically, Thailand. Thailand is a key country for growth, especially for multiplayer online battle arena (MOBA) games, and an area where influencer marketing can be particularly powerful.

Partnering with Facebook Gaming and performance-based influencer marketing agency A2Z Influencers, they created a strategy that combined the creative power of influencers with a diversified marketing strategy  — and that yielded extraordinary results, says James Koh, director of user acquisition at Wildlife Studios.

“Our goal was to, first, see if we could have better conversion, second, double or triple our daily active user counts, and third, see if this improved retention is maintained in the long run,” Koh says. “In fact, the creatives that we used with the videos of these influencers, outperformed all the other creatives we were using in Thailand.”

Their influencer marketing strategy increased daily active users (DAU) by several orders of magnitude, he says. Plus, they not only achieved 22% higher installs per 1,000 impressions and a 17% higher click-through rate compared to their business-as-usual creatives, but saw 42% lower cost per incremental conversion and 10% lower cost per install.

There’s also been a long-term effect on retention, Koh says. At almost nine months since the campaign ran, a significant number of players who joined back then are still playing today. In fact, today Thailand is still running at an elevated DAU count compared to the baseline.

“We were able to prove that influencer marketing does increase the effectiveness of your marketing campaigns,” Koh adds, “and when you mix that with your other marketing buys it can definitely have some compounding effects, and some really good results.”

Here’s a look into how they conceived, developed, and ran such a successful influencer marketing strategy.

Why Thailand?

MOBA or multiplayer online battle arena games have historically struggled on mobile in the western markets, particularly in the U.S. But in other parts of the world like Asia, where Arena of Valor is one of the top-grossing MOBA games today, there is an opportunity for games in this genre to gain a foothold. And unlike western markets where the influencer space is significantly more fragmented, individual influencers — or key opinion leaders (KOL) — have a tremendous amount of penetration in Thailand.

“By working with just a handful of people, you suddenly have a huge impact on your audience,” he explains.

Why influencer marketing?

Influencers in Thailand have a potentially massive reach, but what Wildlife was particularly interested in is the buzz they can create. Getting a surge of users into the game has a compounding effect, and is beneficial to the game’s health. Rather than a small group of players, even if they’re regulars, these battle games most need a considerable audience playing simultaneously.

They were also looking for a strategy that would let them enter a new territory and engage authentically. It’s essential to respect cultural nuances, and regional attitudes and sentiments. The people who know this audience best, and can speak to them most sincerely, are the local creators who are embedded in that community and lively participants in local gaming trends and movements.

“Partnering with creators that are endemic to the gaming space — they’re going to be able to convey the messages in the most genuine and authentic way,” says Peleg Israeli, Director of Global Gaming, Creative Shop at Facebook. “That’s a big part of what sets influencer marketing or creators apart from just regular paid ads.”

How Facebook Gaming and Wildlife built the campaign

The first step in creating an influencer campaign was identifying the well-known gamers who could deliver the most impact for the budget proposed: mid-tier, with strong production quality, open to a collaborative production process, and most importantly, with broad appeal. That’s because with influencers you only have one shot, Koh says, since there’s diminishing returns on subsequent campaigns.

“I can’t stress enough how important audience fit is,” Koh says. “So a lot of time goes into watching their videos, trying to understand who are the type of followers that watch this particular personality.”

Two motivations that would resonate with the local audience stood out: the group play element, and Zooba’s adorable characters, which sets the game apart from other, more realistic MOBAs like Fortnite. Those motivations became central to their approach.

They tapped five different creators who produced about 20 assets. After Facebook Creative Shop briefed them, they co-scripted each ad, and built out an internal production process with Wildlife Studios to finalize the assets.

The media strategy, from app install objectives to app event optimization and value optimization, was diversified, leveraging a number of Facebook measurement and marketing best practices to reach a broad group of gamers of a variety of value. The campaign involved a mix of both organic content that the creators posted on their own channels, but also paid UA ads that had edited cuts, or a combination of their organic content with gameplay and other Zooba content that was produced specifically for this campaign. Each ad was built to be easily adapted to the length and format that worked best for its placement.

Because of the language barrier, they partnered with a talent agency that helped manage the communication directly with the creators — and that actually ended up being an unexpected benefit, Israeli said. They had to make sure the strategic brief was very clear about objectives and direction, but at the same time gave local creators the space to do their magic and bring their creativity into the mix.

“It created a very good opportunity for everybody to do what they do best,” Israeli says, “and the results speak for themselves.”

Key learnings, best practices, and great ideas

The Facebook Gaming and Wildlife teams came away not only with a wildly successful influencer marketing campaign, but a number of lessons in what makes the difference between doing well and knocking it out of the park.

Partnering up: One of the most important takeaways for both companies is finding the right intermediary, creator agency or Facebook Business Partner – it’s non-negotiable. The local partner is responsible not for overcoming the language gap, but for sourcing the right creators, managing the communication with them, and taking care of all the contracts and logistics.

“Allowing teams to focus on the strategic brief and giving clear direction, and then also focusing on the output that comes back from the creators is something that the intermediaries can really help with,” Israeli says.

Measurement KPI alignment: Influencer marketing, perhaps more than any other kind of strategy, requires all the key stakeholders to be aligned on the measurement strategy and KPIs from the start.

“These types of campaigns involve quite a lot of different parties,” says Israeli. “Having everybody very clear on what we’re trying to achieve and then representing that in the media plan, having clear KPIs that everybody understands are the objective of the campaign, all that really helps drive toward success.”

Follow up with live marketing. The increase in DAU was even more than Wildlife anticipated, Koh says, and to really leverage the influx of daily active users, it’s important to prep more events, and more integration with your influencers. For example, an on-air drop to present a free offer to players.

Let creators create: The creative people know their audience the best and know how to engage with them in the best way, Israeli says. Let them read and understand the brief, play the game, see what resonates with them, and then come back with their ideas on taking that brief and making sure the brand is represented well and the content is relevant to the followers they know best.

Dig deeper: Get more insight on the impact of this influencer marketing strategy here.

VB Lab Insights content is created in collaboration with a company that is either paying for the post or has a business relationship with VentureBeat, and they’re always clearly marked. Content produced by our editorial team is never influenced by advertisers or sponsors in any way. For more information, contact [email protected].