Vinay Gaonkar is the co-founder of Kmesh.io.
In this video, Vinay Gaonkar from Kmesh.io presents: Vinay’s presentation detailed the ‘ins and outs’ of Multicloud Lustre-as-a-Service.
Vinay reviewed the three main drivers behind the need to have a high-performance file system in the cloud and, as you likely already know, Lustre is the highest performing of them all. The need for cloud-based Lustre, he explained, is driven by both technological and business factors. On the business side, many enterprises need to comply with a wide array of new data sovereignty (data localization) regulations that compel organizations to decentralize their data, having country-specific or region-specific data hubs. At the same time, the increasing adoption of multi-cloud strategies and edge applications (think Kubernetes) further push organizations to decentralize their cloud data. In the end, this all adds up to a trend in which the cloud world is moving from heavy use of centralized data lakes to a much more flexible and responsive architecture of many smaller, distributed data ponds.
Highlights:
- Dataplane architecuture. Vinay then showed the Lustre users how Kmesh developed its dataplane architecture in response to this new world of distributed data ponds and the need to orchestrate data across them.
- High performance file system. As part of the service overview, Vinay highlighted the need for a highly performant file system. In essence, once an HPC user recognizes the need to make data accessible and moveable across multiple clouds and locations for HPC applications like EDA, the choice of Lustre as the core of the Kmesh architecture seemed an obvious choice to make. In fact, the entire Kmesh engine is built on Lustre.
- Removing the File System and Cloud Management Burdens for HPC Users. Vinay went into detail regarding what it takes to perform HPC in the cloud and demonstrated the steps Kmesh took to make Lustre in the cloud a turnkey, enterprise-ready solution. Lifting up the covers from the Kmesh architecture, Vinay explained how it was made enterprise-ready by creating a unique HA model for Lustre. Kmesh figured out how to make multi-cloud data cloning and sync a reality. This lets HPC users take data created in cloud and bring it wherever they want, such as for hybrid HPC such as ML, data analytics, and EDA. One of the key features that makes it all possible is the use of Lustre-based data nodes spread throughout the customer’s choice of clouds and locations.
- Easy deployment. Vinay further highlighted the massive effort HPC teams will save when using Kmesh SaaS rather than trying to develop their own Lustre-in-the-cloud solution. Instead of having to figure out how to install Lustre, prepare AWS instances for every single use case, and figure out what storage strategy is best-suited to each different cloud, it all comes turnkey with Kmesh. All the customer has to do is make data orchestration selections within the Kmesh portal. They never need to deal with calculating which underlying technologies or capacities are required. Kmesh handles that in the background.
- Lustre-based data nodes within Kmesh. There are node options for performance, balance, and costs. So, a performance node special for an AWS EC2 instance can be picked to match the customer’s specific size and performance, along with granular control over how to manage storage, snapshots, tiering, etc. with all the bells and whistles.
- DataFlows Make Numerous Use Cases Possible. Kmesh DataFlows are, essentially, a data sync technology between two Lustre instances. They can also sync object storage (like NFS) to Lustre instance, or even sync from a traditional Lustre instance to a Kmesh Lustre instance. As a result, Vinay showed, HPC users can use Kmesh for multiple use cases, including real-time sync between Lustre clusters; sync between a data source (like object storage) and a Lustre cluster; and remote data access using the Kmesh intelligent caching.
Vinay Gaonkar is the co-founder of Kmesh.io. He has over 20 years of experience in enterprise storage, virtualization, networking and cloud. Prior to founding Kmesh, he led product and solution strategy for EMC/XtremIO, led data management software at Violin (IPO), and drove VMware’s storage strategy.
