Today, it’s not enough for companies to collect as much data as possible in the hope of gleaning insight. Much of this information is useless and only serves to bog down corporate networks. Instead, companies need to focus on data integration and the ability to make the right data available to the right users, regardless of origin or platform. Here are 6 technologies to improve data integration:
- Discovery - The data integration landscape is changing, in large measure thanks to the cloud. Over the next few years, it's likely that startups and enterprises alike will adopt hybrid and multi-cloud services at scale, making the effective integration of data a top priority.Chief among the technologies needed to make this integration possible are discovery tools able to map data services on both internal and cloud-based networks. This will help eliminate the problem of redundancy so common in big data efforts today — by creating a “catalog” of available data assets, companies can avoid the issue of over-spending time and money on identical integration services.
- Virtualization - Virtualization is not a new technology, but will rapidly become a necessity for advanced data integration. In many respects, virtualization addresses the problem of legacy systems by creating new data structures which can be placed “over top” of existing databases, in addition to changing the way systems are made visible and accessible. In effect, virtualized integration tools allow companies to control how data is seen, how it is used and who has access.
- Movement - The movement of data between multiple sources is also critical. This ties into the notion of “fast data” — information and insight must arrive on demand to provide the best benefit for enterprises. As a result, expect to see a sharp rise in technologies which focus on the speed of data integration across multiple platforms and operating environments. In the long term, this focus will likely shift to quality as speed of movement becomes an expectation rather than an emerging force.
- Identification - Where does data live, and who owns it? Data identification technologies aim to answer this question and provide a starting point for integration. First is location identity: Does data reside primarily in the cloud, on local stacks or in virtualized environments? Next is type: What kind of data is being stored — this might include trade secrets, employee records or the personally identifiable information (PII) of consumers. Defining this identity allows the development of access: Who should be able to use this data, for what purpose and from which source? Identity tools will become critical in managing compliance expectations while enabling integration.
- Specificity - As noted by a recent Healthcare IT News article, one major hurdle for data integration efficacy is specificity. Health agencies in particular have access to a wealth of unique and confidential information; off-the-shelf data integration tools simply aren't designed to handle the volume of this data and its accompanying access controls. As a result, data integration solutions will trend toward “tooling that effectively and naturally understands and validates industry coding, provides meaningful data profiling, componentizes processing for reuse, and can handle the sheer volume of healthcare data.” The same is true for other industries — specificity will soon trump general usability.
- Gravity - Last is the notion of data gravity. Dave McCrory proposed the idea a few years ago — the “mass” of big data tends to attract other pieces of data along with services and applications. For example, consider the number 13. In isolation it has little value but when more data is added, context and value emerge — consider that 13 is a prime number, is the number of original American colonies and is considered bad luck in some religious traditions. In effect, data tends to fall together and create ever-more-massive objects. New data integrations able to manage and curate this gravity will help companies avoid the problem of data black hole, where information enters but insight never leaves.
Use the Right Tools for Sustainable ROI
Data integration tools are evolving to meet the emerging demands of cloud-based and on-demand businesses. Expect to see a host of new solutions emerge that focus on key areas of integration: Discovery, virtualization, movement, identification, specificity and gravity. The right tools, used effectively, should help companies navigate the sea of big data and only collect information that's relevant, actionable and provides sustainable ROI.