As 2015 draws to a close it's time to look back on the year's big changes in data management and reflect on some of the most insightful observations by leading data experts.
Here, Big Data Quarterly presents a round-up of thought-provoking articles that explore some of the key advances of 2015, how organizations are harnessing what big data has to offer, and the challenges they face as they seek to compete on analytics.
How big data is changing the Internet of Things and future technology:
As we enter a third dimension of big data it is important to understand what areas have seen growth in the big data space and what it all means. Ray Kingman, chief information officer at Semcasting, Inc., asks that if first dimensional data is about highlighting a causal relationship and second dimensional data is about inferring intent, then the third dimension of data may be about putting all variables into play and what does that look like? Read it here.
A clear sign of an increasingly digital world:
In February 2015, the White House appointed DJ Patel as the first U.S. chief data scientist. In an interview, Patel discussed what big data means for the government and what it’s all about. Read it here.
Mapping a path to the data lake:
No lake in the natural world can be defined as being exactly the same as another, nor does the path to a lake’s creation follow a one-size-fits-all approach, writes Radiant Advisors’ John O’Brien in a Big Data Quarterly article. In fact, he notes, it is this uniqueness and differentiation that paves the way for variety and diversity. The same can be said for the data lake, O’Brien observes: “There is no one single path to the data lake within the data architecture of the organization. Likewise, each data lake is unique, with inputs and decisions from the organization contributing a variety of essential elements in organization, governance, and security." Read it here.
How to overcome the shortage of data scientists:
According to Big Data Quarterly columnist Bart Baesens, the shortage of skilled talent and data scientists in Europe and the U.S., as well as the competitive pressure to reduce time to market and lower costs, may lead some companies to consider outsourcing analytical activities. In a recent article, Baesens covers what companies need to consider as they choose between building the analytical skill set internally, either at the corporate or business level, outsourcing all analytical activities, or going for an intermediate solution whereby only a portion of the analytical activities are outsource. Read it here.
Ways big data will change every business:
The era of big data is here and with it brings questions and concerns about how this influx is affecting enterprises. Bernard Marr, business and big data expert, explains how he believes big data and its implications will change the way enterprises will work together and interact with customers. Read it here.
Big data can pose challenges to data quality and governance:
According to FICO’s Scott Zoldi, as many organizations find their big data initiatives originating outside existing data management policies, the concepts of formal data governance are being either intentionally or unintentionally omitted in order to speed the process ingesting huge streams for increased insight and new analytic value. But, Zoldi points out, there are a whole host of problems that can result from ignoring established data governance rules - including unapproved access, improper use of data for personal gain, or decisions based on poor data quality or misapplied analytics - that can ultimately expose companies to litigation or poor operational performance. Read it here.
Crunching big data in the medical field:
Data is changing the way doctors are treating patients and only relying on basic science and human anatomy is becoming old hat. Now medical students at the NYU School of Medicine are taking on a project called “health care by the numbers” which aims to teach future doctors and nurses how to spot trends in certain datasets to better help patients care for themselves, assist patients in making important decisions about the cost of treatments, and diagnose major illnesses earlier. Read it here.
Using big data to help make cities smarter:
Inrix, a firm specializing in vehicle and traffic data, is using its crowdsourced traffic network and built-in two-way vehicle connectivity to help with urban planning and transit investment. The new services give agencies a more scalable, cost-effective and immediate way to predict, plan and prioritize investment in roads and transit across their entire transportation network infrastructure, the company said. Read it here.
Has Hadoop been overhyped:
On the one hand, Hadoop has been touted by mainstream media as being a cure-all for all big data woes, but it has also been speculated that the open source data management system has been overhyped due to issues such as ease of use and that as projects move from pilot to production, challenges around manageability, security, and performance can arise. DBTA columnist Guy Harrison tackled these thorny questions in an article titled “Decoding the Mixed Messages of Hadoop.” Read it here.
Understanding software licensing as it relates to cloud:
As organizations move larger portions of their critical infrastructure into the cloud, it is essential that they have a firm grasp of the “convoluted labyrinth of software licensing,” write Big Data Quarterly columnists Michael Corey and Don Sullivan. It’s important to know what they are getting, what they are not getting, what they are responsible for, what the cloud provider is responsible for, and who is paying for the many licenses they will inevitably need. Read it here.
Image courtesy of Shutterstock