The wealth of data now available to organizations from internet-scale applications and the growth of the Internet of Things, is fueling the use of data lakes, artificial intelligence (AI), machine learning (ML), and predictive analytics solutions.
These and other technology initiatives for managing and analyzing data were explored in a recent Unisphere Research survey. In the report, analyst Joe McKendrick identified key trends that are shaping the way enterprises leverage their data, as well as the evolving priorities of data managers. Data lakes, a place to store diverse datasets without having to build a model first, are perhaps the most mature technology initiatives seen among enterprises in the survey.
According to the report, sponsored by Oracle, data lake adoption continues to rise as data managers seek to capture and store data from a wide variety of sources in various formats. Overall, 38% of organizations are employing data lakes as part of their data architecture, up from 20% documented in a 2016 survey.
Another salient point raised by the survey, is the trend of Spark moving to the mainstream. “While it remains to be seen what role cloud computing will play in propelling Hadoop engagements, the cloud appears to be a natural setting for its newer sibling, Apache Spark,” observed McKendrick,
who also noted that as “Hadoop implementations have decreased, it appears some data managers are opting to favor object storage as an alternative to the Hadoop File System.”
According to the report, object storage, which is designed to contain identifiers, metadata, and files within single units and be deployable across devices and systems, is also relatively new, “but already has a sizable base of adherents.” ML is making its presence felt as well. About one-quarter of respondents’ organizations are using ML as part of their data architecture, and another 20% are considering adoption. Highlighting its newness, the report reveals that a majority of companies with ML, 51%, have been using it less than a year. Moreover, most respondents are either facilitating or are likely to facilitate ML applications in the cloud—33% of respondents with ML in place have implemented it in the cloud, and 21% intend to do so.
Identifying new and disruptive technologies, as well as evaluating when and where they may prove useful, is a challenge in the fast-changing big data market. To contribute to the discussion each year, Big Data Quarterly presents the “Big Data 50,” a list of forward-thinking companies that are working to expand what’s possible in terms of collecting, storing, protecting, and deriving value from data.
We encourage you to explore these solution providers by visiting their websites. You can also gain insight into trends in how data is being managed and consumed by accessing Unisphere Research’s survey reports at www.unisphereresearch.com as well as an extensive library of best practices reports and white papers at www.dbta.com/DBTA-Downloads/WhitePapers.