Distributed² Machine Learning notebooks with Elixir and Livebook - Launch Week 1 - Day 2

Welcome to the second day of Livebook Launch Week! 🎉

Today we will discuss all the new Machine Learning capabilities in Elixir and Livebook. First, we will discuss some of the new Neural Network tasks in Livebook v0.9. Then we will show how easy it is to make an existing model run concurrently and then distributed across multiple machines and multiple GPUs.

Let’s get started!

Distributed² Machine Learning with Elixir

This new Livebook release has many new Machine Learning models you can use with just a few clicks. But there is more exciting news about the Machine Learning story in the Elixir world.

Not only can you do Machine Learning with Elixir, but you can also do Distributed² Machine Learning!

But what does that “Distributed²” means?

It means that when executing a Machine Learning model with Elixir’s Nx, the distribution can happen on two dimensions:

• it can be distributed to multiple machines
• inside each machine, it can be distributed to multiple GPUs

This can lead to increased performance and better utilization of your hardware resources. And it only takes very few changes to your code.

Do you want to see an example? Watch this part of our video.

We are very excited about these features because we are starting to marry the benefits of Nx with the capabilities for building fault-tolerant and distributed software from the Erlang Virtual Machine. Case in point: adding distribution support to Nx took only 400 lines of code, including docs and tests!

New Machine Learning models available

Neural network tasks were initially added to Livebook version 0.8 with the idea of giving you the power to use modern Machine Learning models with just a few clicks.

This 0.9 release comes with more models. Here they are.

Speech-to-text

Conversation

Question answering

Language detection

Zero-shot text classification

Punctuation

What now?

We encourage you to go ahead and install Livebook’s latest version to start playing with all of those Machine Learning models!

And if you have any comments or want to share what you’ve built using Livebook, you can tweet using the #LivebookLaunchWeek hashtag.

Stay tuned for the following announcement of the Livebook Launch Week!

More Launch Week

• Day 1: Deploy notebooks as apps & quality-of-life upgrades