Such a 360-degree view can have an immediate impact on many areas of the business, agrees Steve Weiss, manager of strategic development for UC4 Software. “For example, combining web log files with ecommerce transaction data, combined with shopping cart abandonment data, can help retailers understand how their promotions translate into both revenue and profitability for their products. This enables the development of a highly optimized sales channel.”
The bottom line is that enterprises that successfully tackle comprehensive data integration are at a competitive advantage. “They can take advantage of contemporary techniques such as sentiment analysis and extremely targeted marketing which improves their success rate,” says Goodson. “Those who don’t will be at a severe disadvantage as they will not to have the same power to analyze key data sources—whether it is customer data from a cloud CRM database, marketing data from Marketo, or Eloquoa in the cloud and on-premise finance data.”
Goodbye, Warehouse?
Why aren’t more established approaches to data integration, such as data warehousing, batch loading, and ETL, up to the task? While data warehouses will remain parts of enterprises for a long time to come, they don't offer the real-time, turn-on-a-dime flexibility that may be more attainable with more virtualized approaches. “Traditional data warehouses will not go away as large stores of information,” Steve Gold, executive vice president of Opera Solutions, tells DBTA. “But data warehouses are not flexible. The key is flexibility to viewing information, and we’ll see that with big data that is hosted in the cloud.”
For the ETL process, expect to see greater automation, as enterprises seek to speed up the process of moving data from source systems into a warehouse or analytical engine. “The future of ETL is ‘just-in-time ELT,’” says Akred, emphasizing reversal of ‘transformation’ and ‘loading’in the acronyms. “Data is extracted and loaded ahead of time, but it is transformed fit-for-purpose when the question is asked. Traditional data warehouses will be replaced with scale-out MPP versions for structured enterprise data, and large NoSQL instances for less structured data.”
Ultimately, data warehouses will need to evolve, Yves de Montcheuil, vice president of marketing of Talend, argues. “Like the data warehouse, which is now extended to become the logical data warehouse, data integration is a lot more than just ETL,” he tells DBTA.
Goodson agrees that virtualized environments are changing the way data is delivered, but they don’t necessarily ease some of the pains of data integration. “Organizations are now building mobile and SaaS applications first—and integrating from those environments is even harder than doing it via on-premise applications,” he points out. At the same time, he predicts that new approaches will emerge over the next few months that will better address cloud integration problems.