As the linguistic power couple of the data industry, “big data” has spurred an incredible amount of attention during the past decade.
It’s for good reason. Not too long ago, large volumes of information were available only to the select few organizations able to afford the expensive IT infrastructure to collect, store, manage, and analyze it—the big-budget companies with seemingly bottomless pockets, or the professional research services that built empires on collecting and interpreting data.
But now, through the realized effects of Moore’s law, along with the consumerization of BI tools, this data is increasingly available to everyone—and without having to dig deep into budgets or employ robust IT departments. This is especially important for small (and newer) businesses. Widespread access to data is empowering these organizations to easily and affordably access, collect, and analyze data to create new opportunities with their markets and customers. That is the quiet momentum of “data democratization” that has emerged in the wake of the big data revolution. Through the emergence of web-based data collection and analysis and richer self-service technologies, this data democracy is enabling fast access to actionable insights. As a result, organizations, from the massive to the minuscule, are racing to get a handle on their big data.
Race they should. Big data equals more information, and information, as we all know, is power. Access and insight into data drives innovation and competition. Likewise, it’s the ability to index and integrate information to improve performance and create opportunity. It’s the grease that oils the gears that spur innovators to innovate, creators to develop, and marketers to market—what GoodData CEO Roman Stanek termed “the oil of this century.” More importantly, with a greater emphasis on mobile engagement and a mainstream willingness to share data online, individual consumers are becoming the new market research companies—and they’re ready to capitalize on their data. Beyond the sum of their data points, consumers are willing to share their behaviors too. Actually, it is the users who are generating a huge chunk of all the new data that we’re trying to use in our advanced analytic efforts today.
However, the continued proliferation of big data does not simply offer promise and opportunity. It has also raised concerns about privacy and the guiding principles surrounding the collection, storage, distribution, and use of sensitive information. While big data presents a convergence of technological and strategic capabilities that provide significant potential for data-driven organizations to squeeze the value out of information, it also brings the potential to use this data unethically. After all, data is a very valuable commodity. And today’s explosion of data is not just about collecting data; it’s about the opportunity to discover actionable insights from that data.
Therein lies the rub: Now that we’ve all had the chance to get our hands on all kinds of new and exciting data, what do we do with it? And more importantly, what should we do with it? The aftershock of the big data revolution is the world of ethical dilemmas.
During a session at TDWI Boston: The Analytics Experience in July, I spoke with data industry professionals about the emerging technological, commercial, and moral issues surrounding big data. We enjoyed an open dialogue focused on the key areas in which big data may impact the lives of people, from protecting their social relationships to preserving individual privacy in a public world. At the end of the session, the attendees and I engaged in a brief experiment in which I created a mock situation that any analytics officer might encounter in the world of big data. I invite you to play along in this experiment, and consider how you might respond in the following scenario:
Imagine you are the head of analytics at a major insurance provider. Your CDO asks you to create a social media presence to connect with customers’ (and their networks’) publically shared data. So, you do. Your CDO then asks you to mine social activities of customers to identify those who engage in extreme sports. Is it ethical to use customer data to adjust policy rates?
As you can see, big data opens up opportunities to use customer information in ways not previously considered and poses the potential for exploitation. With big data comes big responsibility.
Image courtesy of Shutterstock.