Data-driven business decisions “require smart human minds,” Roberts added. “People shouldn’t waste their valuable time worrying about table structures.” Data management has focused on the “static structure of data at rest, classically within a database management system,” added Naz Quadri, head of enterprise data science and quant development at Bloomberg L.P. Thanks to AI and machine learning, “people are starting to look at the dynamic structure found within data without imposing such static up-front restrictions. This should allow us to begin to harvest information from data sources previously thought unsuitable.”
AI and machine learning are providing other effective data management capabilities as well. For example, these technologies can “predict table growth to improve capacity planning,” said Thomas Fredell, chief product officer for Merrill Corp. Additional capabilities enhanced with AI and machine learning are “self-management, healing, resiliency through auto-tuning, and identifying anomalies,” he added. Ensuring data quality is another area, as AI can perform the mundane tasks of looking for bad data and offering suggestions for fixing the content.
Configuration and workload management are also candidates for AI-driven data management. “Individual configuration settings can be augmented by—or even completely driven—using machine learning algorithms,” said SAP’s Kazmaier, who stated that recently AI and machine-learning techniques—including mathematical optimization, deep learning, and reinforcement learning—have come to the fore. In addition, AI and machine learning “are driving the detection of security breaches based on abnormal data access, such as reading large amounts of data or atypical selection criteria.”
Query management is another potential area for AI, as it can be “used to identify potential root causes of long-running queries,” said Fredell. AI can also flag and potentially throttle users placing a disproportionately large load on the database, he added. Load patterns can be identified and preemptively scaled at certain times of the day, or days of the week, to provide a better user experience and effectively control costs.
In addition, AI and machine learning “can help catalog the massive amounts of data and metadata being produced today, and use that data to forecast outcomes and prevent failures,” said Kiran Chitturi, CTO architect at Sungard Availability Services. “AI and ML can also improve customer experience and support by quickly and intelligently answering queries from data lakes, while also analyzing customer sentiment. It can also help ensure smarter data governance, maintain data privacy, and monitor for privacy regulations as well.”
IMPACT ON JOBS
Industry experts are divided on the ultimate impact of AI and machine learning on data-related jobs, particularly among data professionals. “AI and machine learning will definitely impact data management jobs, but not in the way that many people are fearful of,” said Carr. “Instead, by complementing the skill sets of existing employees, AI and machine learning will help to ‘upskill’ the processes of these employees,” As a result, Carr continued, “we are going to see more diverse backgrounds and skill sets in data architects and IT staff, coupled with backgrounds in everything from hard sciences to social science, law, and many other fields.”
The role of the DBA will be changing as well, Fredell predicts. “There will be fewer traditional DBAs in the future, as many of these tasks will be automated through AI and machine learning,” he said. “The new DBA role will be expected to utilize AI-and ML-powered tools to simultaneously manage a much larger number of database instances than they do today. Data analysts will spend less time on troubleshooting and data cleanup and increase their focus on data management, stewardship, quality, delivery, and creating value from the database content rather than managing it.”
Traditionally, data management departments and staff have been tasked with reporting a historic view of the business and data for management to learn and make predictive decisions,” said Ed Macosky, senior vice president of product, UX, and solutions at Boomi, a Dell Technologies business. “Data management departments and staff are now being brought in more strategically to assist with a forward-looking view of the organization and to provide guidance on data management to predict the future.”