Pivotal has announced the release of Pivotal HD 2.0, a commercially supported enterprise distribution of the Apache Hadoop stack, now rebased and hardened on Apache Hadoop 2.2, and the general availability Pivotal GemFire XD, an in-memory database integrated with Pivotal HD 2.0.
The combination of Pivotal HD 2.0, the HAWQ query engine, and GemFire XD are the foundation for the Business Data Lake architecture, which aims to provide data scientists, analysts and developers with a big data application framework for enterprises. Pivotal is partnering with Capgemini on the Business Data Lake.
The approach combines big data volumes from new sources with legacy data to provide business with relevant analytics capabilities. The objective is to enable a broad base of business users to create their own personal perspectives on all data – including structured and unstructured, stored and streamed and from inside and outside their organization.
Also new within Pivotal HD is an enterprise integration of GraphLab, an advanced set of algorithms for graph analytics that enables data scientists and analysts to leverage popular algorithms for insight, i.e. page rank, collaborative filtering and computer vision. Pivotal released Pivotal HD with HAWQ last year to increase the speed of analysis for Hadoop queries; it was designed from the ground up as a massively parallel SQL processing engine optimized specifically for analytics. With Pivotal HD 2.0, new improvements to HAWQ include the MADlib Machine Learning Library to help unlock deeper predictive insights faster with over 50 in-database analytic algorithms; Language Translation to help users benefit from the full power of R, Python and Java to enable business logic and procedures that are otherwise cumbersome with SQL; and beta Parquet Support to read/write Parquet files.
"When it comes to Hadoop, other approaches in the market have left customers with a mishmash of un-integrated products and processes,” said Josh Klahr, vice president, Product Management, Pivotal. “Building on our industry-leading SQL-on-Hadoop offer, HAWQ, Pivotal HD 2.0 is the first platform to fully integrate proven enterprise in-memory technology, Pivotal GemFire XD, with advanced services on Hadoop 2.2 that provide native support for a comprehensive data science toolset. Data driven businesses now have the capabilities they need to gain a massive head start toward developing analytics and applications for more intelligent and innovative products and services."