Image courtesy of Shutterstock.
Data is being amassed at rates unimaginable just several years ago. Not just collected, this data must be stored, secured, shared and analyzed in order to be useful. But with all this data flowing fast and furiously throughout their organizations, how can users be sure the data is rock solid and accurate? Without that assurance, all the effort to store, secure and analyze information will just be an exercise in futility.
The challenges of big data quality were discussed at Data Summit in NYC, in a session presented by Paula Wiles Sigmon, program director, product marketing, IBM, and Elliot King, who has just authored a Unisphere Research report, sponsored by IBM, on how “governance moves data from hype to hope.”
Big data introduces unique data governance and data quality challenges
Until recently, most data was created as a result of internal business processes. Now, data from a diverse range of sources not possible before. The famous three V’s of big data with volume, velocity variety, includes a whole new range of data that can be explored for new insight. For example said King, who is also a communication professor at Loyola University in Maryland, colleges now routinely seek to conduct “learning analytics” to assess how students are engaging with online material and what they are producing themselves.
Companies need to know their data is accurate
But when data is flowing in from so many new sources, said King, what worries people is, is it reliable, can it be trusted, can it really be used? In addition, said King, IT teams are spending more than half their time devoted to maintaining the status quo, leaving them with little time to address emerging issues relating to data quality and security, or working to make all this data more accessible to users and open to analysis.
Increasingly, organizations across industries are adopting formal data governance strategies to get a grip on their data lineage, quality and protection, said Sigmon. This is driven by concerns that they are spending too much time looking for information rather than actually exploring it.
Sigmon gave examples of customers that have adopted beneficial data governance strategies, including a healthcare company that put in place a master data management system to be able to unify information from 17 different data repositories and provide a consistent, deduplicated view to practitioners and patients, and a fashion company that was handling its data in spreadsheets and put in place an automated, integrated system with BI on top that now allows it to predict merchandise requirements for different areas of the world four months in advance and avoid the risk of over-producing while making smarter decisions.
Emerging role of Chief Data Officer
In order to implement these new approaches, advocacy is needed at a high level, necessitating a new C-level role within organizations, said Sigmon. Increasingly, she said, chief data officers are the ones championing new approaches to use data more strategically, and looking at ways to advance the business with data.
Watch videos of Sigmon's and King's Data Summit presentations here - and for more Data Summit videos, go to www.dbta.com/videos.