AWS brings Mistral AI’s open-source LLMs to Amazon Bedrock

AI

Amazon Web Services Inc. today announced plans to make two artificial intelligence models from Mistral AI, a high-profile machine learning startup, available through its Amazon Bedrock service.

Introduced last April, Bedrock provides access to managed foundation models via an application programming interface. Developers can use the API to test which of the available neural networks is most suitable for a given project and then integrate it into their software.

Paris-based Mistral is backed by more than $500 million in funding. The two AI algorithms that it’s making available via Bedrock, Mistral 7B and Mixtral 8x7B, are both open-source large language models. They target similar use cases, but their design and capabilities diverge significantly.

Mixtral 8x7B, Mistral’s most advanced LLM, debuted in December shortly before the company announced its latest $415 million funding round. It can power chatbots, summarize documents and generate code. The model outperformed OpenAI’s ChatGPT 3.5 on most of the benchmark tests Mistral evaluated during an internal comparison.

Under the hood, Mixtral 8x7B is based on an LLM design known as the mixture-of-experts architecture. The model comprises eight different neural networks, or “experts,” that are each optimized for a different set of tasks. When Mixtral 8x7B receives a user prompt, it identifies the two neural networks that are best equipped to generate an answer and activates them.

LLMs based on a mixture-of-experts design require a relatively limited amount of hardware to run. Because Mixtral 8x7B only activates two of its eight neural networks when answering a user prompt, the remaining six don’t use any processing capacity. The result is a reduction in infrastructure costs compared with traditional LLMs that activate all their software components when processing prompts.

Mixtral 8x7B features 46.7 billion parameters across its eight neural networks. Mistral 7B, the second LLM the company is making available to AWS customers via Bedrock, features a simpler design with only 7 billion parameters. It’s geared towards applications that place a particular emphasis on hardware efficiency.

“Mistral 7B is the first foundation model from Mistral AI, supporting English text generation tasks with natural coding capabilities,” Donnie Prakoso, a principal developer advocate at AWS, detailed in a blog post. “It is optimized for low latency with a low memory requirement and high throughput for its size.”

Mistral’s two open-source LLMs will become available alongside the more than half a dozen foundation models already accessible through Bedrock. Some of those models, namely the Amazon Titan series, were developed by AWS. Bedrock also provides managed versions of neural networks from Meta Platforms Inc., Anthropic PBC and other major market players. 

Image: AWS