Gluster, a provider of open source storage solutions, has just released a version of its storage system that includes support for Apache Hadoop. GlusterFS 3.3 beta 2 employs standard file system APIs available in Hadoop to provide a new storage option for Hadoop deployments.
The company claims that GlusterFS is the first open source, POSIX-compliant file and object storage solution that works with Hadoop. Compatible with existing MapReduce-based applications and with the ability to co-exist with HDFS, the new functionality opens up data within Hadoop deployments to any file- or object-based application, providing a new option for big data deployments.
A differentiator with the technology is that there is no centralized metadata server within GlusterFS, Tom Trainer, director of product marketing at Gluster, tells 5 Minute Briefing. "And not having the centralized metadata server really speeds things up tremendously."
GlusterFS offers faster access to data, as well as N-way replication for synchronous and asynchronous replication to ensure data stays synchronized, regardless of latency or potential network interruptions. Gluster also enables high availability with built-in failover, self-healing and no single point of failure; and provides increased flexibility in sizing, since with Gluster, Hadoop deployments are no longer limited to large-block sizes or 16 terabyte volume sizes. Gluster provides POSIX-compliant NAS (NFS, CIFS) and object access to files within Hadoop deployments as well as easy access to both file and object storage, simplifying management and controls storage costs.
The new compatibility presents an opportunity for users from a number of perspectives, says Trainer. Companies that have been involved in doing analytics and managing big data growth environments - major global conglomerates - have been building a silos of servers and storage, but, he says, in this big data analytics world that silo concept can only last for so long because organizations wind up with unique specialized hardware that becomes very expensive. Gluster he says, provides an alternative. "You can use GlusterfFS and commodity servers - and Hadoop was designed to use commodity - and commodity storage, and build out this repository that can hold Hadoop files but can also consolidate some NAS requirement and also some object storage requirement as well."
GlusterFS 3.3 beta 2 is available now. To participate in the public beta, visit the 3.3 beta resource page at http://bit.ly/nUtssL. General availability is expected in the fourth quarter of 2011. For more information about the company, go to www.gluster.com.