Alluxio, a developer of open source cloud data orchestration software, is creating a solution in collaboration with Intel that offers an in-memory acceleration layer with 2nd Gen Intel Xeon Scalable processors and Intel Optane persistent memory (PMem).
The solution eliminates performance degradation of analytics clusters that are increasingly built on disaggregated compute and storage architecture.
“Today’s disaggregated cloud storage lacks efficient file system semantics support like ‘rename’. Additionally, disaggregated cloud storage typically can’t leverage compute side storage media such as DRAM and SSD for use as buffers and page caches,” said Haoyuan Li, founder and CEO, Alluxio. “Adding Alluxio Data Orchestration System and Intel’s Optane persistent memory solves both issues, enabling maximum benefit for cloud storage and achieving competitive and even better performance than traditional on-premises configurations. This is particularly helpful for hybrid cloud environments when data is remote.”
Using Storage over App Direct, a feature of PMem App Direct mode, allows Alluxio to access high-performance block storage without the latency of moving data across the I/O bus to provide the data acceleration and reduction in query runtime.
With Alluxio and Intel Xeon Scalable processors, an I/O intensive benchmark delivers a 3.4x speedup over disaggregated S3 object storage and a 1.3x speedup over a co-located compute and storage architecture2.
In addition, Alluxio has joined into a strategic collaboration with Intel aimed at improving their joint customers’ experience with managing and processing their data, such as optimizations for Intel Deep Learning Boost, the AI acceleration technology built into Intel Xeon Scalable processors.
Together, Alluxio and Intel will bring solutions to market to help fuel next-generation data, analytics and AI applications and use cases.
“With this new collaboration we bring to market a solution that enables enterprise-grade shared storage and faster time to insights to solve for the challenges we see around bounded storage and compute resources on Hadoop,” said Rowan Scranage, chief business officer, Alluxio. “Together with Intel, we plan to disrupt the advanced analytics and AI status quo with an in-memory data accelerator layer to accelerate intermediate data access and ease data bottlenecks that many of our customers are highlighting as key challenges with their increasing big data requirements.”
For more information about this news, visit www.alluxio.io.