Data Integration for the Modern Enterprise - How Cloud Shifts the Balance

Bookmark and Share

The rise of cloud computing has changed the whole concept of data integration. Previously, it was an information technology department concern, with IT and data staff sweating it out with manual scripts, product connectors, and middleware brokers in efforts to cobble together relevant applications, or surface data trapped in legacy system silos to deliver it to modern front-end interfaces. Top-level efforts also consisted of endeavors to bring data into a common place through master data management, or organizing information through ontologies.

The blood, sweat, and tears that go into enterprise data integrations may not go away anytime soon, but lately, it has become easier to make data from any and all sources more available to people and applications that need it. The cloud—and more specifically, database as a service (DBaaS)—has shifted the challenge of data integration. Clouds and DBaaS offerings are gaining traction as an online means to manage and process data. The cloud offers a major venue for big data solutions, since they are faster to deploy and easier to operate and scale than with typical on-premises systems. At the same time, the rise of cloud and DBaaS as an information management environment has shifted the balance of responsibility for enterprise data integration away from the confines of data centers to the enterprise as a whole. Suddenly, the entire business has a stake in identifying, managing, and leveraging the way data flows through IT systems.

To access more articles on the new world of database technologies, go here.

A recent survey of 300 DBAs and IT professionals, conducted by Unisphere Research, a division of Information Today, Inc., finds growing interest in DBaaS as a viable approach to serving their enterprises’ needs for greater agility and faster time to market with cloud computing. Many of the early hurdles in delivering enterprise capabilities for security and availability in the cloud become more evident with the reliance on hybrid cloud approaches and the need to move enterprise applications to the cloud and back on-premises based on the business requirements of the organization, their legacy investments, and regulatory requirements (“Database as a Service Enters the Enterprise Mainstream: 2016 IOUG Survey on Database Cloud,” April 2016). DBaaS is taking off, with adoption expected to triple over the next 24 months. There will be a significant amount of enterprise data shifting to the cloud over the next 24 months as well, as enterprises rethink data management in the cloud. Seventy-three percent of managers and professionals expect to be using DBaaS within their enterprises by that time, versus 27% at the present time. What does it take to construct and sustain a viable DBaaS strategy?

Here are some considerations for constructing a successful DBaaS strategy:


Moving to DBaaS is more than simply making data more accessible, it also opens new paths to innovation. Data is a tool, a means, to better engage customers and better understand markets. Plus, as new ideas and requirements arise, DBaaS—especially if delivered by a cloud provider—serves as a testbed to help accelerate innovation and experimentation, since cloud providers will likely have all required features and services in place. Shifting more activities to cloud providers or shared service environments frees up enterprises and their IT staffs to provide higher-level support to the business.


Data governance has long been a challenge for enterprises, and cloud or DBaaS doesn’t make things any easier. Essential concerns such as data security, quality, and relevance will need to be dealt with at the enterprise level. In addition, there is a need to wrap governance around disparate data sources, which often were external to enterprises and therefore not under their purview. There are a range of business requirements that must be addressed, from real-time data streaming to analytics to customer relationship management. And, organizations must to be able to move large and varied datasets at a high velocity through their systems—which raises issues for the handlers of this data, such as who owns it, who has access to it, and how and where it should be stored.


From a planning/spending perspective, the future belongs to more hybrid approaches. Many organizations continue to maintain an abundance of legacy or on-premises assets, and this is likely to be the case for some time to come. As long-standing legacy assets, these systems have proved their worth and resiliency, and continue to function well for their organizations. Mainframe systems, in particular, continue to be refreshed by IBM and are capable of supporting the largest cloud and DBaaS workloads and the latest protocols. As a result of this sizable legacy base, the largest percentage of organizations in the Unisphere/IOUG survey, 44%, see the establishment of hybrid cloud as their most important priority as they enter the cloud space.


Data security isn’t just about securing data from hackers, but also entails access control, as well as avoidance of any potential for third parties to mishandle the data. Half of the managers and professionals in the Unisphere-IOUG survey indicate that security and privacy concerns are the greatest inhibitors to their cloud initiatives. Data ownership and retention follow closely behind as the second-ranked concern. Often, trusting outside cloud providers with sensitive or mission-critical corporate data is seen as risky, not only in terms of potential breaches, but also in terms of the potential need for a relationship between a cloud provider and consumer to be modified or terminated. The fate of data held by a cloud provider may not be clear-cut.


The beauty of cloud and DBaaS is that multiple standards, devices, and interfaces are supported. However, it’s still important that all parts of the enterprise be on the same page. On a high level, standards are emerging to help simplify cloud-based integration. For example, the Open Data Protocol (OData) promises to replace the web services standards REST and SOAP to enable greater interoperability between enterprises and across the cloud. In many ways, enterprises are becoming API-driven, meaning applications, functions, and services can be interconnected, on-the-fly, to share data as required by the business demands of the day.


For the private cloud, a robust enterprise infrastructure is essential. The internal systems—particularly storage— need to be highly adaptable and elastic for unpredictable workloads. Public cloud services also offer compelling storage options. Services such as Amazon S3, OpenStack Swift, Microsoft Azure Storage, and Google Cloud offer virtually unlimited storage resources that can serve as total storage sites or as bursting services for spikes in enterprise workloads. On-premises solutions—from open source frameworks to commodity hardware— also provide scalable storage solutions.


Not all business cases may be suitable for DBaaS or cloud. To determine the cost/benefit of cloud sites, enterprises need to weigh the traditional on-premises costs of maintaining hardware, systems, networks, and storage, versus that of a shared, multitenant enterprise private cloud, versus subscribing to a public cloud provider. There are also management and labor costs that pertain to project management, development, oversight, monitoring, and quality management—costs that are likely to be incurred regardless of whether the data is managed on-premises or by an outside cloud provider. Even with the public cloud, there are time and expense requirements related to migration or integration between various systems, or between cloud and on-premises systems. Other expenses that need to be measured include the cost of software licenses versus subscriptions, or the costs of upgrades and maintenance versus pay-by-the-sip monthly fees. In some cases, it may make sense just to leave things as they are; in other cases, the savings may be significant. But, ultimately, the true value comes from the enhanced opportunities for innovation and growth.

To access more articles about the new world of database technologies, go here.

To access industry research reports on  enterprise technology, IT, marketing and media trends, go to