CHALLENGES AHEAD
As these trends develop in the year ahead, many challenges will keep data managers, CIOs, and chief data officers up at night. “The question is no longer if we have the data, it’s ‘should we use it, and if so, how?’” said David Lloyd, chief data officer at Ceridian. “AI isn’t always the answer. Regulations will rain down on AI from all corners of the world—in addition to data regulations already in place like GDPR, the California Consumer Privacy Act, and others—creating a complex regulatory maze that will be challenging for companies to navigate.”
Successful AI or generative AI needs a data-first mindset: It takes well-managed data to produce accurate results.
“Simply put, generative AI requires that businesses have data in order,” said Upchurch. “That means good data management practices, including data access, hygiene, and governance. Organizations that don’t step up their data literacy game and have good data management practices will find generative AI isn’t as productive for them.”
Generative AI may gobble up large volumes of data, as it is dependent on large language models. “Companies will need to train these models effectively,” said Varshney. “They need to ensure that the data used to train these models is of high-quality, accessible, available, and compliant with applicable compliance frameworks. Ensuring that the vast volume of data required for training meets these criteria will be a major challenge for data teams.”
The siloed and distributed nature of data will surely continue to vex enterprises during the next 12 months—along with the legacy environments that are staying in place. There is also the related challenge of “orchestrating the huge volume of data that grows daily, as well as extracting useful business insights out of that data and making them actionable,” said Waid. “These are difficult problems to solve, and they are compounded by digital transformation that is an unforgiving, Herculean task.”
Data distributed across diverse systems and formats “is leading to issues in data quality, integration, and accessibility,” Gupta agreed. “Furthermore, the exponential surge in data volumes and varieties adds to the intricacy of data platform infrastructures. The proliferation of data privacy and security regulations places stringent demands on organizations, requiring meticulous handling of customer data.”
The migration to cloud platforms—which will only accelerate over the coming year—will require new approaches to managing data, namely, “the continuous preparation and optimization of data for use in both native-cloud and hybrid-cloud applications,” said Bob Brauer, founder and CEO of Interzoid, and founder of StrikeIron. “Cloud data management introduces new imperatives: enhancing performance, ensuring data scalability, controlling storage costs, maintaining availability, and integrating many kinds of data while keeping up with evolving data security standards.” In addition, Brauer added, “data quality issues will also become more complex.”
Data security challenges won’t go away any time soon either, unfortunately. “Ransomware will continue to be a significant threat to organizations of all sizes, worldwide,” said Jim McGann, VP of strategic partnerships at Index Engines. “Many organizations will still go into 2024 under- or unprepared for ransomware.”
Bad actors will continue to ramp up their nefarious activities, McGann continued. “These groups are introducing new names and distinctive signatures to their ransomware. They manipulate data through encryption so it cannot be used, alter file extensions, and implement system changes, such as disabling security features and removing access to essential systems.”
Along with ransomware, another “potential dark cloud on the horizon will be the ability for maliciously wielded AI-powered cyber agents to probe data security mechanisms for flaws, identifying and exploiting security vulnerabilities,” Brauer warned. “This could possibly enable data heists and data breaches, dwarfing those of previous years. This will unfortunately materialize in the headlines throughout the year.”
The threat caused by “the escalating sophistication of AI-powered cyberthreats poses a significant challenge for data managers, CIOs, and chief data officers,” agreed Aron Brand, CTO at CTERA Networks. “These advanced threats, capable of adapting and evolving rapidly, extend beyond traditional IT environments, targeting smart devices, IoT networks, and critical infrastructure.”
Moving data at the speed of light to where and when it is needed is another challenge of the AI era. “Technologies like AI, specifically large language models or GPT-type frameworks, will drive enterprises to create purpose-built solutions … for their enterprise while also pulling in external data to augment training and deployment—not the other way around,” said Allardyce. “To move at the speed of AI and establish policy, usage guidelines, and environments, these data-focused leaders will be forced to try and evolve at the pace at which technologies are rolling out.”
Business demands for more real time data will create new kinds of headaches for data managers. “Obtaining and ensuring consent, collating data for the same entity, maintaining privacy, and bringing it all together for real-time decisions will push many to re-evaluate how they streamline decisions,” said Zoldi.
Data and privacy laws are another challenge shaping up in the year ahead—especially as they affect AI. “AI regulations are quickly emerging,” said Reiskind. “There are hundreds of regulatory initiatives in this space, so it’s challenging to keep up.”
In addition, “many people are excited and eager to use tools like ChatGPT, Bard, and others,” Lloyd cautioned. “Without proper education around what is or is not OK to input into these systems, there’s a chance that employees are creating a ‘leaky boat,’ feeding sensitive and proprietary information.”
Organizations across the board “need to make major changes in the way we approach data management and technology strategies,” DeSimone advocated. “We are not equipped to field data, analytics, and AI-enabled capabilities at the pace and scale required. In response, we will start to see increased use of data curation and advanced analytics to ensure data is consumable by both humans and machines. We need a data-driven culture across organizations, which requires a well-trained, knowledgeable, and data-aware workforce.”