How are cloud and container technologies enhancing enterprise data management capabilities? “With the ability to transparently scale computing across the data center and public cloud, without changing applications, developers can be much more ambitious and productive,” said Singh. Consider a large financial institution with 22 new use cases across marketing, HR, sales, support, and other areas, said Singh. “In a world of static data center technology, it would have to lift and shift to the public cloud, which would create friction for the deployment of new use cases.” With hybrid cloud data management, the IT department can satisfy demand without having to lift and shift petabytes of data.
At the business level, managers “will not have to choose between the data center and the cloud,” Singh added. “Their investments in the data center will continue to accrue benefits while the public cloud will be integrated more seamlessly into their data management strategy.”
The challenge for data managers is providing direction and oversight for the explosion in activity and innovation that cloud services and containers are enabling. “As soon as the business side can allocate an arbitrary amount of computing for queries, IT teams will be faced with workloads that are runaway, with lax security, and in silos,” Singh said. This could result in serious adverse consequences for cloud computing itself. “In this world, the DBA has to be able to predict usage patterns, or data management will be chaotic. Another problem with transient cluster computing is that admins, architects, and developers lose context, history, and security when the dynamically allocated compute disappears. The technology teams will have to learn how to audit, secure, and tune in the world of dynamic computing.”
Master Data Management
When it comes to scaling to the enterprise, there’s a need for consistency and quality. Master data management (MDM) has taken center stage as the means to keep data aligned with the enterprise. “Data quality has been understood as managing consistency within a dataset and fixing individual bits of data, but more and more, it’s being understood as managing consistency across datasets and fixing data as it relates to other datasets,” said Jake Freivald, VP, Information Builders. Even though people don’t typically master the big data itself, there’s a renewed interest in knowing how to connect sets of less-governed big data to highly governed master data. He predicted that while still nascent, MDM adoption will grow as data scientists look for higher-quality data for their analytics. “They are still suffering from the ‘garbage in, garbage out’ phenomenon, and integrating MDM with big data helps avoid it.”
MDM brings more flexibility than previous efforts to centralize data management, such as data warehouses, Freivald added. MDM “doesn’t require all of the transactional data to be fully modeled, but provides much more governance and consistency than using big data on its own. Because of that, using MDM is allowing people to have the same level of flexibility—the data is still stored in relatively free-form ways—but with sufficient governance to allow them to connect the dots reliably in more analytical situations.”
The business also benefits from MDM, Freivald continued, due to “better data inputs for AI and machine learning, which will lead to better outputs and greater adoption for these potentially transformative technologies.” There will be better analytics in general, as well as an “improved ability to house big data-related content in data catalogs, which will increase awareness and usage of big datasets to solve a wider variety of business problems more reliably.”
‘Alternative’ Data
A big part of big data is, of course, data coming from outside the enterprise. While this “alternative” data has been on the scene for many years, there is a new wave of significant adoption, said Gary Read, CEO and founder of Import.io. This may include hedge funds that need extra guidance for investment decisions, as well as retailers that require rapid competitive pricing analysis or data on consumer reviews. The result is that companies are increasingly relying on substantial amounts of alternative data from sources, such as the web, in conjunction with the traditional data sources for business decisions, said Read.
“Currently, the majority of companies are spending internal resources to manage alternative data,” Read said. “However, looking forward, data extraction and integration technology will be better equipped to help businesses collect and harvest the value of alternative data, and demand for such solutions will continue to increase.”
Technologies that enable the sourcing and collection of alternative data “provide huge efficiencies to the enterprise,” Read noted. “Not only are internal resources no longer wasted on writing code and maintaining scripts to manage such data, but there are now an increasing number of services and platforms with the capability to extract data automatically so that the data can be quickly integrated with other data sources to drive decisions.”