High Performance Computing (HPC) and Data Analytics (HPDA) are coming together as the techniques that have been developed and used for HPC can now be applied to investigating and making sense of large amounts of data. Even by stringing together a high number of existing servers cannot handle the demands of high performance analytics. A new paper from IDC discussing the challenges and a possible solution is now available from the InsideHPC White Paper Library.
The key to advanced analytics is to create a system that balances compute, memory access, I/O and storage, all in an efficient and easy to use package. Rather than a one off device, it is important to integrate systems that can excel at data analytics into an existing corporate infrastructure. As the technologies for HPC and enterprise computing merge, new opportunities exist for robust data analytics across a wide range of industries. Key verticals that can use highly tuned HPDA systems include financial services, manufacturing, on-line retailers, telecommunications and healthcare/bioinformatics, and energy among others. As data analytics becomes more mission critical, hardware and software need to evolve to handle both historical data (batch) and real time streaming data. This combined ability to manage different types of data is critical for a wide range of organizations.
The HPDA market is forecast to grow at over 20 % CAGR from 2013 to 2018. This can be attributed to the knowledge that “to outcompute is to outcompete”. Factors for this include:
- Complexity – organization can use more complex inquiries to their data.
- Time to value – businesses need to make decisions faster in order to gain a competitive advantage and can use highly tuned systems to do so.
- Variability – data can be wide or deep and tools are needed for both.
While clusters of traditional x86 servers are used for much of the data analytics market, the main challenge is how to deal with the limited memory on a node. Using standard servers for HPDA requires tremendous amounts of communication, as the data has to be moved around from node to node for environment where the data cannot be partitioned and stored on one node.
The Ryft ONE platform takes an innovate approach to the I/O bottleneck challenges. The compute, storage and I/O are tightly integrated in a compact 1U system. The Ryft ONE combines the Ryft Analytics Cortex (hardware) and the Ryft Algorithm Primitives library (software) to optimize the overall performance of the system. Ryft ONE uses the company’s long experience with predecessor platforms that were primarily used for military use.
Each 1U Ryft ONE device gives users the ability to simultaneously ingest and search up to 48 TB of locally stored data. This data can be either static or streaming, and can be structured, unstructured, and in many forms. Benchmarks have shown that using a single Ryft ONE appliance gives the same performance of 100 to 200 nodes for a Spark cluster equivalent.
This paper gives a great overview of the HPDA market and the Ryft ONE system. It includes information about the market, the solution and opportunities and challenges for the Ryft ONE data analytics platform. Download it now.
