Neural Magic takes aim at simplifying the optimization of machine learning models in production

AI

With hardware innovation somewhat stagnated, the next frontier of technological advancement seems to be putting computers to work on intricate tasks through artificial intelligence.

With AI and machine learning playing inextricable roles in the next generation of enterprise computing, enabling capabilities to ease developers’ jobs is a topic worth discussing. And Neuralmagic Inc. is tackling that unique problem area.

“It’s not good enough just to work on models; you’ve got to put them into production,” said Brian Stevens (pictured), chief executive officer of Neuralmagic Inc. “So, what we do is we make it easier to optimize the models that have been developed and trained and then try to make it super simple when it comes time to deploying and managing them in production.”

Stevens spoke with theCUBE industry analyst John Furrier for a CUBE conversation ahead of the AWS Startup Showcase: “AI/ML: Top Startups Building Foundation Model Infrastructure” event, They discussed Neural Magic’s value proposition and how it sets the company apart. (* Disclosure below.)

Building on existing foundational models

What usually drives the cascading collapse of complexity in technology is the gradual build-up of resources and repositories upon which successive iterations can develop. This same trend has been instrumental in pushing machine learning forward by negating a lot of the hitherto-required heavy lifting, according to Stevens.

“It means that people don’t have to start from scratch,” he explained. “The avant-garde now starts with an existing model that almost does what you want but then applying your data set to it. So it’s really the industry moving forward. And then the best thing about it is open source plays a new dimension — but, this time, in the realm of AI.”

The logic behind how Neural Magic simplifies the operationalization of these ML models, as they go through production, has a lot to do with an idea known as “sparsity.” Applying this technique makes the model a lot smaller while still retaining all of its foundational elements, according to Stevens.

“In many cases, we can make a model 90 to 95% smaller, even smaller than that in research,” he said. “We do that in a way that preserves all the accuracy of the foundational model. So now, all of a sudden, you get this much smaller model that is just as accurate.”

Subsequently, the “sparsified” model is paired with Neural Magic’s inference runtime called DeepSparse, and because it now occupies a fraction of its original size, the compute resource requirements are also greatly reduced.

“You now can actually deploy that model anywhere you want on commodity hardware — so x86 in the cloud,  x86 in the data center, or even ARM at the edge,” Stevens stated. “It’s like this massive unlock that happens because you get the state-of-the-art models but on the IT assets and the commodity infrastructure. That is where all the applications are running today.”

The outlook for growth in AI and ML fields couldn’t be brighter, with a barrage of investment and human effort pouring in from all swathes of the industry to tackle a myriad of current and potential use cases, Stevens pointed out.

“What’s gone from nascent eight years ago is now the Wild, Wild West out there. So there’s a little bit of everything right now, and I think that makes sense because, at the early part of any industry, it becomes really specialized,” he concluded.

Here’s the complete video interview, part of SiliconANGLE’s and theCUBE’s pre-event coverage of the AWS Startup Showcase: “AI/ML: Top Startups Building Foundation Model Infrastructure” event:

(* Disclosure: Neuralmagic Inc. sponsored this segment of theCUBE. Neither Neuralmagic nor other sponsors have editorial control over content on theCUBE or SiliconANGLE.)

Photo: SiliconANGLE