From data to insights: Vast Data's journey to empower data developers

NEWS

The data storage and artificial intelligence sectors are witnessing a significant transformation, led by innovative companies that are redefining industry standards by merging advanced scalability, performance and technology integration. This narrative reflects a broader trend in the industry, moving toward more efficient and intelligent data management solutions.

Early on, Vast focused on AI, which has become crucial in today’s data-driven world. The company also had a vision to eliminate traditional hard drives and promote flash storage. It recognized that as data volume increased, the advantage shifted from quantitative to qualitative, unlocking new capabilities. Consequently, its platform was meticulously engineered to handle not just exabytes, but extend beyond that scale. All of this has paid off, with Vast raising $118 million in late-stage funding, driving its valuation to $9.1 billion.

“We saw that with more information, you turn from a quantitative advantage to a qualitative advantage, and new abilities emerge as you give these systems more data, and so we built this for, not just exabyte scale but beyond that,” said Renen Hallak (pictured), founder and CEO of Vast Data, during a recent conversation with theCUBE, SiliconANGLE Media’s livestreaming studio. “Four-and-a-half-years after we launched the company and have a lot of exabytes on the platform, we’re seeing that that scale is not just theoretical — it actually works.”

Before founding and serving as the chief executive officer of Vast Data, Hallak worked at XtremIO, where he served as VP of research and development and led the development of an all-flash array from its inception to over $1 billion in revenue. He also served as chief architect at Intercast, where he developed the company’s content distribution system from inception to deployment.

This feature is part of SiliconANGLE Media’s ongoing series exploring the latest developments in the data storage and AI market.

The challenges of AI development

During his discussion with theCUBE, Hallak talked with theCUBE’s John Furrier and Dave Vellante about how the core inspiration for Vast Data has always been artificial intelligence, even before embarking on the development of the company’s storage system.

“We didn’t have confidence. We didn’t know that it would work,” Hallak said. “We didn’t know that this path was the right one, but we thought that if it was possible to do the things that we presented in this keynote, it was worth a big bet.”

The goal was to enable fast access to extensive information to support emerging algorithms and applications. The company faced many challenges and uncertainties when initially starting its journey in 2016, with AI still in its early stages.

There is a big difference differences between Vast Data’s early vision of AI and the current state of AI, which often relies on pattern matching and statistics. Regardless, Hallak remains optimistic about AI’s potential to generate new ideas and solve complex problems.

“I don’t know that we have the answer right today, but we’re definitely in the right direction.,” he said. “What we do, our process, is a very simple one. We talk to our customers, and we ask them, ‘What do you see? What do you need?’ They help us see around corners, because they know where their challenges are.”

Revolutionizing data storage with flash technology and pioneering AI integration

Even before Vast Data’s development of AI in relation to data management, the company was big into upcoming innovations, especially as pertained to flash storage and hard disk drives. In a past CUBE Conversation with theCUBE’s Dave Vellante, Hallak talked about implications for the future of data storage and data centers.

The motivation behind founding Vast Data was essentially to sound the death knell for hard drives in data centers. The company’s vision was to eliminate hard drives and make flash storage the primary medium for accessible data, according to Hallak.

“I think what we bet on in the early days is the fact that the pyramid doesn’t work anymore and that tiering doesn’t work anymore,” he said. “That was the driving force behind us building this new type of architecture. We’re seeing every single day when we talk to customers how the old architecture is simply breaking down in the face of these new applications.”

Even back then, AI was of interest to the company, as its customers were already using AI and deep learning to handle workloads that accessed the entirety of their data, reading it over and over for insights. This fast access required storage media, such as flash, to handle it, according to Hallak.

“Once you have that, you have the foundation to step into this next-generation type work where you can actually make money off of your information,” he said.

The emergence of data developers: A paradigm shift in application development

In today’s ever-evolving landscape of technology and innovation, data is increasingly taking center stage in the realm of application development. The traditional paradigm is undergoing a profound transformation, with data now assuming a pivotal role, according to Hallak. He envisions a future where every developer becomes a data developer, and applications are intricately woven around the principles of data manipulation, analysis and insight generation.

“I think all developers will be data developers relatively soon,” he said. “The paradigm is flipped on its head. We used to have applications and they were at the center, and they were writing to storage systems and reading from storage systems and from databases … we’re seeing that happen in these new workloads that we discussed. I think it’ll happen across all the workloads.”

As the significance of data continues to grow, industries are crafting their own specialized clouds to cater to their unique requirements. Vast Data perceives itself as a catalyst for these industry-specific clouds and has forged strategic partnerships with technology leaders, including Nvidia Corp., to provide the essential infrastructure.

The company is confident that, with applications spanning diverse sectors such as finance, healthcare and beyond, it is exceptionally well-positioned to manage and support the data needs of these industry clouds, according to Hallak.

“We partner with companies like Nvidia to leverage the latest and greatest of what they have to offer,” he said. “Then you need that software infrastructure layer that makes it easy for the various verticals to focus on their applications. And that’s where we come in.”

Photo: SiliconANGLE