Big data has been around in one form or another for a long time, but lately, due to current events and intensified pressure, there has been greater attention focused on data-driven approaches to manage operations and understand customers. Recognizing that value has shifted to the digital realm, businesses have been looking to technologies that will take them to the next level. To explore this mass movement in more detail, we asked a number of leading industry experts and solution providers to describe what they see as the most impactful technologies shaping today’s big data world.
THE CLOUD
Why it’s hot: “The cloud is changing the game for analytics and for any enterprise’s ability to compete on data,” said Zdenek Svoboda, vice president of platform at GoodData. Cloud services enable companies to use data to get products and services to market faster, they’re less costly than on-premise solutions, and they make data more easily available and more secure, he said, also noting that cloud services are fundamentally disrupting traditional on-premise installations. “The analytics stack is now available as managed services in the public or private cloud.”
Emerging or widespread? “Cloud data management and analytics is in widespread use, is still growing in double digits and will continue to grow,” said Svoboda. “Enterprises need fast and smart access to the ever-increasing volumes of data they’re collecting. For a long time, business intelligence was incapable of scaling to accommodate either the growing volume of data or the increasing number of users. The cloud has changed everything. It makes the process significantly cheaper and much easier to scale. This is enabling strategic decision making based on actionable analytics.”
Gotchas: Country-specific regulations. “Many companies operate beyond borders, collecting and analyzing data from customers and partners in an effort to provide better, more personalized services,” said Svoboda. “But the internet is becoming balkanized with different data and privacy requirements, such as GDPR in Europe and CCPA in California and other country-specific rules. Companies have to rise above the regulatory basics and see data as an asset, not a liability, and as a way to improve their services.”
In 5 years: “The cloud will be the default deployment option for data management, analytics, and data science solutions,” said Svoboda. “Analytics is about creating value for every customer, every partner, every user, and the cloud makes doing that easier, faster, less costly, and better.”
ARTIFICIAL INTELLIGENCE
Why it’s hot: “AI-augmented analytics is having the most visible effect on organizations’ ability to compete on data,” said Stanley Zaffos, senior vice president for product marketing at Infinidat. “It enables organizations to transform data into information that improves the quality and timeliness of decision making, increases staff productivity, and expands the breadth and scale of tasks that an organization or individual can take by compensating for human limitations.” Big data “is like oil in our new era, and machine learning is the technique to turn raw oil into gas and many powerful products,” said Jian Pei, chair of the ACM Special Interest Group for Knowledge Discovery from Data (SIGKDD) and professor at Simon Fraser University. “Enterprises need machine learning technology to turn their raw data into business edges, revenues, and profit.”
Emerging or widespread? “Very few companies can have the advanced capa-bility to train large-scale, deep-learning models that match their business needs, which require both a huge amount of data and massive computational infra-structure,” said Pei. AI usage is becoming more commonplace in areas like security information and event management, as well as AI operations—AIOps—use cases like application performance monitor-ing, added Zaffos.
Gotchas: Skills shortages, budget constraints, corporate cultures, and market chaos. “Chaos in the market takes many forms,” said Zaffos. “Vendors blinking in and out of existence, evolving market segment definitions, and competition between various approaches that are still evolving rapidly.” In addition, “enterprises need to access a large amount of data related to their business,” said Pei. “This requires that their business is well-digitalized and that the extensive data infrastructure is in place, such as data lakes. Enterprises need to access massive computational resources, which may be available on the cloud for some and may have to be set up by themselves for the others.”
In 5 years: “AI augmented analytics will become a must-have technology in widespread use,” said Zaffos. “Without it, large organizations will simply be unable to compete in the areas of productivity and customer satisfaction.” Pei predicted that “the more profitable large-scale consumer markets will see more extensive applications of big data and machine learning in the next 5 years. Many traditional enterprises may face the tough challenge of being replaced by new, data-driven, machine learning-?enabled startups born with the novel business models and culture.”