Image courtesy of Shutterstock.
Moving data to the cloud is oftentimes a surprisingly complex and slow process. Likewise, data transfers on-premises can require a large investment of time, money and maintenance of infrastructure, as well as intensive monitoring by staff.
Regardless of how it’s done, when the data does finally get to the right place, cleansing and sorting it for analysis using the traditional ETL (Extract, Transform, Load) method can be time consuming, requiring additional expert resources to initiate, manage, and monitor that effort.
As a recent report by Gartner suggests, bulk data movement remains the dominant delivery mechanism for data integration. In fact, their research shows that more than half of all data integration is accomplished through bulk loads. To manage the process of loading data into data warehouses, ETL has traditionally been used. This method is ideal when complex data transformations are needed.
However, many data integration use cases only require simple transformations. And others don’t require any transformations at all. For these scenarios, traditional ETL tools are overly complicated and can’t deliver data updates as rapidly as business users need. The priority isn’t on the T, but rather on the L—expediting data delivery—so users have fresh, accurate data to make business decisions.
Thankfully, there’s a new approach that is better at addressing the growing data integration requirements for business intelligence and analysis, known as ELT (Extract, Load, Transform).
Below are four data integration use cases where loading the data takes precedence over transforming the data—optimizing the ELT approach:
1-Enabling faster reporting – offloading data.
In many organizations, large numbers of users access the data warehouse for reporting purposes. This puts unnecessary load on the server and degrades performance, and business users end up experiencing unacceptable delays as they wait for reports to complete.
One solution is data distribution – moving data from the data warehouse to other servers, which are used for reporting. In this case, no transformations are required. For data distribution to be effective, however, rapid synchronization updates from the data warehouse to the reporting servers are essential. With an ELT approach, business users have the best of all worlds—constant access to up-to-date information and fast reporting.
2-Widespread access to mission-critical data: replication for database synchronization.
Today’s business information landscape involves numerous systems and large volumes of operational data. The challenge that companies face with this scenario is in figuring out how to access and manage this data volume efficiently to make it available for mission-critical decision-making. Not a day goes by that we don’t hear about “big data,” as companies capture growing amounts of customer, supplier, and operations-related information. A global survey by the Economist Intelligence Unit found that over the last year, 73% of respondents felt that their collection of data had increased “somewhat” or “significantly.”
Data replication is a technique that can be used to keep operational data current in multiple systems. It is well suited for situations where users must have rapid access to large volumes of mission-critical information. In this scenario, any required data transformations are minimal, and therefore ideally suited for the ELT approach.
3-Supporting geographically dispersed business locations: synchronization over a WAN.
In many industries, like retail and hospitality, for example, the company headquarters must pull up-to-date information from remote sites, such as stores or hotels. This is typically done to facilitate company-wide reporting and analytics, and no data transformation is needed between systems. But data synchronization is often done over the Internet or another form of wide area network (WAN) with low bandwidth and poor performance. To facilitate business at multiple locations, data replication must be completed quickly.
Since retail chains usually operate within very thin profit margins, timely access to data can have a meaningful impact on the bottom line. Based on store-specific sales information, headquarters may decide to transfer inventory from one location to another or make price changes for all stores. Alternatively, store employees may need access to information about stock levels at other stores to improve customer service and satisfaction.
4-Cloud-based analytics: overcoming low bandwidth connections.
Like big data, cloud computing is a major trend in business and IT circles. Cloud-based analytics is often an attractive option for midsized companies. As organizations move information from a data center to the cloud for BI and analytics purposes, the data is usually replicated and no transformation is required.
Because the link between data centers and the cloud is usually low-bandwidth, like an Internet connection, it is essential to use a data integration tool that can send information rapidly. The InterContinental Hotels Group, for example, is using the cloud for many different applications, ranging from data centers to its reservation system. As the company’s CIO, Tom Conophy, commented in a Computerworld article, “If your employees and users can’t access data fast enough, then the cloud will be nothing more than a pipe dream.”
What do you think? Is your company looking at implementing more of an ELT approach instead of ETL?