Lakera launches to protect large language models from malicious prompts

Large language models (LLMs) are the driving force behind the burgeoning generative AI movement, capable of interpreting and creating human-language texts from simple prompts -- this could be anything from summarizing a document to writing a poem to answering a question using data from myriad sources.

However, these prompts can also be manipulated by bad actors to achieve far more dubious outcomes, using so-called "prompt injection" techniques whereby an individual inputs carefully crafted text prompts into an LLM-powered chatbot with the purpose of tricking it into giving unauthorized access to systems, for example, or otherwise enabling the user to bypass strict security measures.

And it's against that backdrop that Swiss startup Lakera is officially launching to the world today, with the promise of protecting enterprises from various LLM security weaknesses such as prompt injections and data leakage. Alongside its launch, the company also revealed that it raised a hitherto undisclosed $10 million round of funding earlier this year.

Data wizardry

Lakera has developed a database comprising insights from various sources, including publicly available open source datasets, its own in-house research and -- interestingly -- data gleaned from an interactive game the company launched earlier this year called Gandalf.

With Gandalf, users are invited to "hack" the underlying LLM through linguistic trickery, trying to get it to reveal a secret password. If the user manages this, they advance to the next level, with Gandalf getting more sophisticated at defending against this as each level progresses.

Lakera's Gandalf. Image Credits: TechCrunch

Powered by OpenAI's GPT3.5, alongside LLMs from Cohere and Anthropic, Gandalf -- on the surface, at least -- seems little more than a fun game designed to showcase LLMs' weaknesses. Nonetheless, insights from Gandalf will feed into the startup's flagship Lakera Guard product, which companies integrate into their applications through an API.