<< back Page 2 of 2

Connecting Insights from the Internet of Things


While the volume of edge-based data is flowing, in many cases, it’s not delivering usable insights. Edge data tends to be fractured, often from third-party sources, even when it is successfully transferred. “There are real insights derived from edge data, but they can be very narrow in scope,” Russell Reid, founder and CTO of LightTrail, explained. “The most commonly identified use cases for edge data insights are monitoring, anomaly detection, and real-time alerts. Beyond these use cases, edge data insights tend to be too complex to implement well and therefore tend to fail at the first attempt due to not preserving dependencies and context in the ingest data pipeline.”

Data quality is the other constant that needs to be addressed as IoT builds out. “It’s not just sensor drift—it’s firmware updates that silently change telemetry schemas, devices that reconnect after hours offline and flood your ingest pipeline with backfill, MQTT brokers buckling under topic sprawl,” Guiu said. “If your downstream infrastructure can’t handle late-arriving, out-of-order, and schema-inconsistent data gracefully, your analysis is compromised before it starts.” Meanwhile, “expectations have evolved,” said Banthia. “Collecting data isn’t enough anymore. The real edge belongs to those who can act on it immediately.”

Manufacturing has long been the natural home for IoT, “but the reality is messy,” said Culwell. “[Manufacturing/industrial plants or warehouses] have been collecting data from devices and sensors for a long time, but getting that data to actually mean something—across systems, across shifts, across sites—is still a real challenge for a lot of companies. The broad insights people talked about early on are starting to show up, but it has taken longer and required more work than the initial hype suggested.”

Skills and expertise also emerge as issues, while data managers seek to wrangle many types of devices with differing data formats. Expertise is needed to “know the tags [and] setup, and how to properly interpret the data,” said Dag Calafell, director of technology innovation at MCA Connect. “Every machine is different, and situations vary, which leads to a high cost to reverse engineer these systems. Frequently, the company or individual who wrote or set up the system is no longer available, and the documentation is too general or not applicable to how it is being used today.”

The bottom line is, “if your architecture can’t keep up with hundreds of thousands of sensors writing every few seconds while simultaneously serving analytical queries, you don’t get insights—you get a storage bill,” Guiu said. “What comes next is much less about scale and much more about having discipline when working with edge data,” said Reid. “Smaller ingest pipelines, more timely and earlier validations, and stronger connections between the data collected and the decisions made using that data will separate winning teams from those that do not give IoT insights much value or credibility. Winning teams will not have more IoT data, but data they can actually rely on.”

WHAT’S AHEAD

Look for a rise in agentic AI capabilities that will increase the value of IoT and remedy many of the issues discussed above. This means “building solutions [that] can respond more dynamically to the real-time nature of the data,” said Martin. “Being able to respond to signals, with guardrails as opposed to a tightly defined ruleset we had previously or merely reporting, is going to be a game changer.”

Expect to see more AI processing “within the four walls, without sending the data to the cloud,” said Calafell. “The application of AI to understand these systems, prepare insights, and build digital twins should reduce the ROI, or increase the quality and speed of IoT projects.”

Expect to see more inference moving to the edge, which will boost intelligent IoT. “This will happen in a way that’s actually practical now, not just a conference slide,” said Guiu. “Smaller, more efficient models combined with better inference hardware mean you can run anomaly detection and classification on-device without round-tripping to the cloud.”

An AI-driven IoT “only works if the data infrastructure handles both the real-time edge pipeline and the historical analytical workload in the same system,” Guiu cautioned. The architecture built around IoT will also evolve to meet this challenge. “The central data platform becomes less about [the] raw ingest of everything and more about aggregated insights, model retraining, and cross-fleet correlation,” said Guiu. “Physical AI takes this further, with sensor data feeding directly into agents that can adjust, reroute, or shut down processes without human intervention.”

Along with effective data management, security is rising as a top concern to be addressed in the coming months and years.

“As device network segmentation matures from static configuration to continuous, intelligence-driven control, the underlying data it depends on—device identity, behavior baselines, and communication patterns—is also what feeds analytics,” said Somasundaram.

The rise of AI will shift inference to the edge “as a default requirement, not just a nice-to-have, letting systems decide locally and send only what’s essential upstream,” Banthia predicted. “Time-series and streaming workloads will move from afterthoughts to first-class citizens in core data platforms.”

IoT efforts are also being touched by regulations. “Directives like CISA’s edge device directive are setting firmer expectations around asset visibility and incident reporting,” said Somasundaram. “State-level IoT laws—like California’s SB 327—continue to raise the floor on connected device security. In Europe, NIS2 [Directive] has expanded cybersecurity obligations across critical sectors, and the EU [European Union] Cyber Resilience Act is pushing security and update requirements directly onto connected-product manufacturers.”

Ultimately, the most successful IoT efforts won’t necessarily be seen in organizations with the largest fleets of devices.

“They’ll be the ones that can move, structure, and act on data with minimal friction,” said Banthia. “The true promise of IoT was never about gathering more data. It was about making faster, smarter decisions. That only happens when the data actually works for you.”

<< back Page 2 of 2


Newsletters

Subscribe to Big Data Quarterly E-Edition