Real-time analytics is increasingly being utilized in many industries, said Oracle’s Lehmann. Examples range from manufacturing and factory environments to online commerce clickstream analysis to real-time fraud detection in financial services, among many others. “To start, organizations need to develop an expertise in streaming data in real time to bring the data for analysis first,” said Lehmann. “Next, the data needs to be analyzed, in motion, in real-time. And the requirements for that analysis are growing. Organizations need to be able to deploy trained and tested machine learning models against real-time data streams to derive more sophisticated insight in real time—
it’s not enough to do simple rule-based analytics on real-time data anymore.” And, finally, he said, real-time analysis needs access to the real-time data as well as historical or contextual data in databases, in data lakes, and in cloud-based object storage.
AI and ML Enhance Human Decisions
The combination of artificial intelligence (AI) and ML is seen as a way to incorporate information about opportunities and risk into the decision making process with a level of speed that surpasses human capabilities. In areas as diverse as banking and finance, marketing, and healthcare, AI and ML can inject intelligence to improve the efficiency of processes.
“There’s a vast movement toward machine learning, and ultimately autonomy, happening in many industries, including manufacturing, automotive, healthcare, information technology, and others,” said Tim Hall, InfluxData VP of products. But, he noted, it is important to remember that this is a journey that originally began with business intelligence [BI], data warehousing, and reporting—“essentially, the desire to understand what happened based on information that had been gathered.”
Now, the physical world is being instrumented via sensors embedded in an increasing number of consumer products and a very rapidly growing variety of industrial products, Hall continued. In manufacturing, he said, “Industry 4.0” represents a new industrial revolution that utilizes robotics, automation, and data exchange of cyber-physical systems to create a smart factory that includes machines that learn with the objective of being able to make decisions for themselves based on the information, scenarios, and goals they are directed to achieve.
“In today’s highly digital and fast-moving world, there’s simply too much data, in too much complexity, for people to extract what they need through all of the noise,” said John O’Brien, principal analyst at the research and advisory firm Radiant Advisors. “The promising use of AI/ML exists for companies to tackle their existing operational challenges where the problems are too complex to solve with traditional rules-based analytics.” However, he noted, “One obstacle for training AI/ML models has been a lack of quality training data.” The data has often been too scarce and difficult to generate objective real-world training datasets; the existing data has had too much historical bias; or the data has been the product of bad processes. A trend for AI and ML for the foreseeable future, said O’Brien, will be in “human assistance”—using AI and ML where possible to assist humans to be faster and more accurate.
AI and ML are becoming prevalent in the enterprise, but the area of most opportunity is the integration of those technologies with BI-style analytics, agreed Priyank Patel, co-founder and chief product officer, Arcadia Data. For example, he said, “You can use BI and visual analytics to present AI/ML outputs in a human-readable form. Visualizations like heat maps and flow charts can help non-technical users quickly navigate to insights that were uncovered by the complex algorithms built by your data scientists. And, since BI tools are designed for non-engineers to build visualizations, you create a self-service model for analyzing the outputs of your advanced analytics.”
AI and ML processes can also be integrated into BI tools to help business analysts be more productive with their analytical work. You can think of AI for analytics as being similar to GPS navigation for business users “who have to deal with tons of information and big data, and need some help navigating to find useful insights,” said Patel. This leaves the power of decision making in the hands of humans, but it accelerates the process.
Microservices and Containers Support Hybrid Clouds
Cloud implementations are seen as an important element for providing easy access to advanced technology to companies of all sizes. With the adoption of cloud there is also concern about vendor lock-in, however. This makes hybrid and multi-cloud scenarios that incorporate a mix of deployments more appealing, despite the risk of greater complexity.
“The primary purpose of cloud is to get on-demand IT resources and elastic scalability, which enables companies to avoid building their own and focus on their core competencies while allowing them to easily meet demand without overprovisioning,” said Eric Holzhauer, principal manager, strategy and product marketing, MongoDB. “Cloud also levels the playing field, so to speak. Whether you’re an individual developer, a small startup, or a Fortune 500 company, you have access to first-class, elastic, globally available, and high-performance resources.” The majority of MongoDB users are deploying in the cloud, either by installing it on cloud infrastructure themselves or using MongoDB’s managed cloud database, he added.
“We’re seeing a higher need—now more than ever—for hybrid cloud strategies among customers wary of lock-in with a single cloud vendor,” said Scott Clinton, VP of product marketing at Hortonworks. And, he said, for developers building the next generation of cloud-native data applications, the use of containers and microservices lets them move fast, deploy more software efficiently, and operate with increased velocity in the DevOps environments in which more data applications are being built.