To gain competitive advantage, “enterprises need to ingest, analyze, and act on data in as close to event-time as possible,” agreed Dheeraj Remella, chief product officer at VoltDB. “However, this can be challenging to do when data must first be sent to a centralized database, which increases latency and reduces the value of the data.”
There’s still a lot of work ahead, as the adoption of edge analytics and real-time inferencing “has been slower than expected,” Gude cautioned. “This is due to the bespoke nature of the solution stack and the integration required to deliver the desired business outcomes. The fragmentation of device, infrastructure, middleware, and application and services options available has made it difficult for customers to scale deployments beyond a limited set of use cases.”
Edge computing will gain ground in the years to come, Gude added, with “broad adoption of edge analytics to derive real-time decision making. Low-latency use cases such as predictive maintenance, quality improvements in manufacturing and process technology, and advanced safety and security will be powered by edge computing that will leverage the services available in the cloud and be delivered as a service at the edge.”
In addition, the use of 5G shows great promise to achieve more capabilities at the edge, Remella noted. “Edge computing is proliferating to create new businesses that can’t be imagined with today’s 4G networks. 5G and IoT are pushing the Industry 4.0 agenda forward and toward a massive machine-to-machine communication enterprise. Hundreds of thousands of IoT devices located throughout warehouses, trucking fleets, and production plants will generate data that needs to be acted upon near the edge, where it’s most valuable, and provides meaningful insights that power decisions and actions.”
Ultimately, Remella expects every enterprise organization to implement edge computing into their everyday processes. “Without it, they won’t be able to compete with the types of efficiencies and insights their competitors achieve.” If an organization has not acted on an event within 20 milliseconds, then that event data will already be part of their big data, or worse, their dark data, Remella said.
This level of speed requires deploying processing capabilities along a spectrum extending from centralized systems to edge networks. Remella advised dividing these requirements and using the cloud “for big data processing for machine learning and model retraining purposes, while the near-edge compute capacity is used for ingestion, filtration, aggregation, KPIs, monitoring, decision making, and action invocation.”
Even though IoT edge devices are already widely deployed, the most powerful use cases are still emerging, Manley added. “Today, most of the 26 billion IoT devices record telemetry and generate alerts. As the devices become more powerful, edge audio and video processing will transform how organizations use technology.”
DATAOPS
DataOps—the automation of data delivery across the enterprise—is another emerging technology focus helping to shape data environments. This is a powerful trend since the tools available for DataOps were previously only available to companies such as Amazon, noted Chris Bergh, CEO of DataKitchen. Bergh described DataOps as the “automation of data integration, preparation, transformation, analysis, visualization, deployment, testing, maintenance, and monitoring.” There are currently more than 75 vendors offering products and services targeting various aspects of DataOps automation, he noted.
Now, with DataOps applied, “the most innovative companies will be those that can quickly adapt to rapidly evolving market conditions.” Bergh predicted that DataOps will gain acceptance in the same manner as DevOps and agile methodologies have succeeded in the software development side of the house. “DataOps will help companies replace caution with confidence,” he stated. “They will automate everything that can be automated and enable their data scientists to focus on those tasks that require human creativity.”
SELF-SERVICE SYSTEMS
Self-service analysis is another trend reshaping data environments. “We’re going to see more and more self-service technology in use in ever-expanding sets of graphs, dashboards, and more,” predicted Grant Fritchey, product advocate and data privacy expert at Redgate Software. “Think of tools like Power BI. Organizations that are putting this to work and enabling individuals to leverage data at a tactical level are going to have the edge.”