Endeca Technologies, Inc., an information management software company, has unveiled native integration of Endeca Latitude with Apache Hadoop. "This native integration between Endeca Latitude and Hadoop brings together big data processing from Hadoop and interactive search, exploration and analysis from Latitude," says Paul Sonderegger, chief strategist of Endeca Technologies.
According to Endeca, Latitude is based on the Endeca MDEX hybrid search-analytical database, and is particularly well-suited to unlocking the power of Apache Hadoop. The engine has no predetermined schema for the data, and instead, it derives its model of the data based on the incoming records and documents with all their diversity, says Sonderegger. As a result, the combination provides flexibility and agility in merging diverse and changing data, and extreme performance in analyzing that data.
When organizations want to bring together tweets with CRM data about their customers, with customer satisfaction data from panel surveys, for example, it is much too time-consuming and laborious to build a traditional single pre-determined schema in a data warehouse for that data, Sonderegger explains. While Hadoop excels at bringing all of that data into one place, it is not designed to provide interactive analysis, and it is not intended to help brand managers or someone who owns a marketing budget with questions unless they have access to real technology expertise to investigate the data that is in Hadoop, he notes.
According to Sonderegger, while big data offers the promise of improved daily decision making because of its great variety, in order to fulfill that promise, there has to be an interactive analytic layer that allows people without training and technology to ask and answer their own questions. "It really takes both of these pieces, Hadoop to do the big batch processing on all of these great sources of diverse and really large-scale data, and then Latitude to provide the interactive analytics piece on top."
Sonderegger says that Endeca, which has more than 600 customers, has been working with intelligence agencies on big data analysis for quite a while, but is now also seeing increased demand in this area from companies in the private sector as well. "That's why we felt it was time to productize the connection between Hadoop and Latitude."