We stand at the start of a new year and on the precipice of a new decade—the 2020s. For data managers, these will likely be the “Roaring ’20s” with data at the heart of every key business initiative, accented by a growing sophistication in technologies and methodologies focused on increasing the intelligence of the enterprise.
To provide insight on emerging trends for data-driven enterprises, DBTA reached out to industry leaders for their perspectives on not only what’s ahead in the year 2020 but also what they see developing as the next decade unfolds.
THE YEAR AHEAD: FOCUS ON THE DATA THAT MAKES THINGS RUN
In the year ahead, organizations will intensify their efforts to manage and process the data that underpins many of today’s cutting-edge initiatives such as AI and machine learning. “The competitive edge goes to the organizations that understand and treat data analytics as one discipline,” said Gavin Day, SVP of technology at SAS. “Organizations are forgetting the critical nature that timely and fit-for-purpose data plays within training, model development, and model deployment. The use of AI within data management technologies will change the role and job function of our data workers—including data scientists. The days of data workers spending time configuring data quality and data integration jobs is behind us. Data analytics platforms use AI to do the routine, heavy lifting so we can focus on what we’re good at—creativity and solving analytical challenges that move our business forward.”
In the process, AI and analytics teams will merge into one as the new foundation of the data organization. “As the importance of data grows, a multitude of ways to get insights has emerged,” said Haoyuan Li, founder and CTO of Alluxio. “Yesterday’s Hadoop platform teams are today’s AI teams. It’s no longer just about managing your data lake.” AI takes a new approach to deriving value from structured and unstructured data, said Li. “What used to be statistical models has converged with computer science, becoming AI and ML.”
AI and Machine Learning
As a result, it no longer makes sense to keep data, AI, and analytics teams “siloed from one another,” Li added. “In fact, they need to collaborate to derive value from the same data that they all use by building the right analytics and AI data stack. Multiple storage silos and a multitude of computes, deployed on-prem and/or in the cloud, will be the norm. In 2020, we’ll see more organizations building dedicated teams around this data stack.”
At the same time, AI and machine learning will emerge from the research stages and move into engineering. “This will bring an increased focus on managing the AI and machine learning lifecycle in production,” said Joe Hellerstein, co-founder and chief strategy officer of Trifacta. “Machine learning models are only as good as the data used to train them, so we’ll see significantly increased focus on data preparation and data quality. We’ll also see an increased focus on monitoring AI and machine learning pipelines to track the quality of prediction serving in production.”