Calpont Corporation, a provider of analytic databases has announced a data connector between InfiniDB and Hadoop. The new connector is bi-directional, enabling organizations to determine where they want to do the analysis of data.
More organizations are using Hadoop for large-scale structured and unstructured data, but the sheer volume of data collected often results in a Hadoop implementation that is no longer performing at the desired speed for analysis, according to Calpont. "For doing high-performance analytics, a high-performance database like InfiniDB is the proper database to be doing the analytics in," Nick Ochoa, vice president of marketing, Calpont, tells 5 Minute Briefing. MapReduce environments tend to be geared more toward building a repository of data and doing some kind of transformation, like ETL processes, but because it is a framework, it doesn't inherently have an out-of-the-box capability of doing large-scale analytics very quickly, he explains. "It is really a custom-build environment based on the hardware you have, so we have enabled InfiniDB to connect to the Hadoop environment to pull data across into InfiniDB to do the analytics very rapidly."
By providing a bi-directional connector for InfiniDB customers, Calpont says, it eliminates the need to custom code Hadoop for high performance analytics since InfiniDB's columnar architecture is optimized for rapid and easy access to highly dimensional data and has the flexibility, scale and horsepower to work with massive data sets.
While Ochoa acknowledges the most likely scenario the company sees for using the connector is moving data from Hadoop to InfiniDB, the bi-directional connectivity provides flexibility for situations where data is brought into an InfiniDB environment from another data source and might eventually need to get back to the larger Hadoop data framework.
According to Calpoint, the InfiniDB architecture is optimized to do faster queries and scale as processing demands grow. InfiniDB is built for big data demands, with a massively parallel processing (MPP), column-oriented architecture that is designed specifically for read-intensive analytic applications and data warehouses that consume large amounts data. The company says InfiniDB scales performance linearly with hardware, is easy to deploy and maintain, and requires no special database tuning, indexing or materialized views, making it suited for big data analytics.
The InfiniDB-Hadoop Data Connector will be made available to InfiniDB users at no cost on www.calpont.com and www.InfiniDB.org in September 2011.