Harvard researchers clone supercomputer on Google Cloud

CLOUD

Researchers at Harvard University have used Google LLC’s public cloud infrastructure platform to create a clone of a supercomputer that was used to perform a heart disease study.

They claim it’s a highly original use of cloud computing resources that can help other researchers who are struggling to access powerful supercomputers to complete their studies.

Harvard professor Petros Koumoutsakos told Reuters that the study was intended to simulate a new therapy that’s designed to dissolve blood clots and tumor cells in the human circulatory system. His team required enormous amounts of computing power that are typically only available with supercomputers.

According to Koumoutsakos, the research team was only able to reserve enough supercomputer time to carry out one full simulation, but it was not able to repeat that exercise in order to refine or optimize any aspects of the test.

It’s a common problem for scientific research teams. In the U.S., there are only a small number of supercomputers available to scientists that have enough power to perform the billions of calculations required for a study like Koumoutsakos’s. As a result, there’s a long waiting list for those wanting access to these machines.

To get around this challenge, Koumoutsakos and his team turned to their partners at Citadel Securities to see if they could instead replicate a supercomputer within the public cloud, where there’s no need to wait to access resources.

The public cloud is not a straightforward solution, as platforms such as Google Cloud are not designed to handle the kinds of tasks researchers usually perform. Rather, cloud instances are designed for millions of much smaller computing tasks, such as serving web pages, hosting applications, streaming video and database access. On the other hand, the cloud is generally very reliable and resilient, and there are no waiting lists for access.

Koumoutsakos and the team from Citadel Securities, along with researchers from ETH Zurich in Switzerland, demonstrated how they used thousands of virtual machines on Google Cloud to replicate a supercomputing platform. They used “extensively tuned code” to leverage these distributed cloud resources to achieve an impressive 80% of the efficiency provided by dedicated supercomputer facilities.

Bill Magro, chief technologist of Google Cloud’s high-performance computing, said the cloud has unique potential to solve problems around technical scientific engineering computing. He explained that to modify cloud infrastructure to behave like a supercomputer, changes need to be made in the software, networking and physical design of the hardware. “Google Cloud’s high performance computing technologies and solutions are purpose-built to both simplify and scale the largest, most complex workloads, enabling researchers to dramatically accelerate time to discovery and impact,” he added.

The research is a nice discovery that can possibly result in alternatives for researchers and organizations that need massive amounts of compute power, but for industry insiders it is not surprising Koumoutsakos and his team were able to pull it off, said Holger Mueller, an analyst with Constellation Research Inc. He points out that Google’s cloud has always been highly configurable because Google’s internal workloads have always needed that kind of configurability. “An example is Google’s translation models, which have been running for many years,” Mueller said. “They need high-end instances and a fast network, which are the hallmarks of supercomputers, and this is exactly what Google Cloud provides too.”

Mueller added that it’s unlikely the few supercomputer providers will be too worried about the public cloud emerging as a rival in the high-performance computing industry, as cloud platforms are equally in high demand. “Just about every cloud platform is seeing capacity constraints now with the interest in AI workloads, and it will remain that way for the foreseeable future.”

Photo: svstudioart/Freepik