Generative AI has taken the world by storm but until now it took a while to generate an image from a text prompt with the typical 50 Steps on a GPU. The fastest generally available solutions on Windows start at 5 seconds or higher unless you want to start copying DLLs by hand to upgrade the torch libraries. There has also been a wide variety of accuracy-degrading performance optimizations like Xformers and Flash Attention, which are great tools if you are open to trading accuracy for performance, however we wanted to unlock maximum performance without any of the accuracy degrading optimizations.

The Nod.ai team is pleased to announce Stable Diffusion image generation accelerated on the AMD RDNA™ 3 architecture running on a beta driver from AMD.  Nod.ai has been optimizing this state-of-the-art model to generate Stable Diffusion images, using 50 steps with FP16 precision and negligible accuracy degradation, in a matter of seconds.

Here are results from one of our end users running on a Windows 10 system with the Ryzen™ 7950X and Radeon™ RX 7900 XTX graphics card with this KB driver and the WebGUI. 2.63 seconds.

Everyone makes tall claims about performance, so we don’t want you to take our word for it. We want you to try SHARK on your system and report the performance you see.

We believe that Generative AI should be accessible to everyone irrespective of their technical background. So we made our Stable Diffusion WebGUI  easily accessible and usable. Today you can download a single file and get started on your Generative AI endeavor. The community has reported that it is able to run on older generation hardware dating back five years.

Here are images generated by the SHARK community on AMD RDNA™ architecture-based devices in the #ai-art Discord channel.

an oil painting of a rat wearing a christmas sweater – empty headed

Cthylla

empty headed [RX 480 8GB]

empty headed [RX 480 8GB]

Ian2400

Denbe and Not Quite Denbe

cstueckrath

Crache

MDuica

Cthylla

Give it a try at http://shark.sd, share and show off what you can create with Generative AI. We are not done with performance, ease of use or feature requests – so stay tuned for more over the upcoming weeks.

SHARK is an open source cross platform (Windows, macOS and Linux) Machine Learning Distribution packaged with torch-mlir (for seamless PyTorch integration), LLVM/MLIR for re-targetable compiler technologies along with IREE (for efficient codegen, compilation and runtime) and Nod.ai’s tuning. IREE is part of the OpenXLA Project, an ecosystem of ML compiler and infrastructure technologies being co-developed by AI/ML industry leaders including AMD, Google, Nod.ai and many more. OpenXLA aims to let ML developers build models in their preferred framework (TensorFlow, PyTorch, JAX) and easily execute them with high performance across a wide range of hardware backends (GPU, CPU, and ML accelerators).

It was fantastic to see the Nod/AMD collaboration produce the great results it has. Beyond the numbers, I am really proud that we were able to create an engaged community that is empowered to make this kind of project happen. That was a key reason I started IREE and was ultimately behind the decision to become part of the OpenXLA project. As part of OpenXLA, we’ll work closely with our community to carry this momentum forward.

Stella Laurenzo, IREE co-founder, OpenXLA community leader, Google ML Compilers

Nod.ai is hiring: [email protected]. Join us on Discord