These days, everybody talks about big data and analytics but does anybody really know how to tackle it? Everybody thinks the others are doing it and hence also claim to do it as well, but it’s likely that only the fortunate have had (positive) experience(s) with it so far.
What is big data actually? Let’s start by narrowing down the perspective to size only and consider more than 1TB of data. Do we know of any successful business cases that create added value by storing, analyzing, and managing more than 1TB of data? Of course we do, but they are too often limited to domains such as astronomy, bio-informatics (e.g., genomics) and only rarely seen in business applications such as risk management, fraud detection, marketing, or supply chain management.
Two Main Issuess in Leveraging Big Data
There are two main issues involved in successfully leveraging big data. One is the organizational structure and the other is the economic challenge that comes with big data and analytics.
A first issue concerns the organizational aspect. How can this new technology be successfully embedded into a company’s DNA? A first option would be to set up a companywide analytical center of excellence, and staff it with data scientists handling all big data and analytics requests from the various departments. The problem is that such a centralized approach simply doesn’t work. To fully leverage and compete on analytics requires business knowledge, implying that the data scientists should be close to the business. The ideal skill mix of a data scientist are quantitative skills, ICT skills (e.g., programming), business knowledge, communication and presentation skills, and creativity. In other words, a the role of the data scientist multi-disciplinary, and to fully exploit this unique skill set another organizational approach is needed based upon the principle of subsidiarity.
The main idea is that a centralized unit should only manage the issues that cannot be successfully managed by the local business units, such as managing the ICT environment (both hardware and software), privacy rules, model governance, and documentation. A substantial amount of data scientists should be directly embedded into the individual business units, such that the analytics projects can be well-focused and fed with the right business knowledge. Business ownership for every analytical project is essential, but also the cross-fertilization between data scientists across business units is important. A well-focused, centralized analytics unit can play a key role in evangelizing, stimulating and communicating good practices and lessons learned (and vice versa, in order to prevent repeated rookie mistakes).
A closely related attention point concerns the sharing of (different types of) data across business units, since this is precisely where the added value is to be situated. Hence, it is preferable to not only consider size when talking about big data, but also to take into account the new insights that can originate from coalescing different data sources, both structured such as transactional data,as well as unstructured, such as server logs, click streams, or social media feeds.
A second consideration is the economic value of a big data and analytics investment. Firms only invest in a new technology when a positive return is anticipated.
Although the expenses, such as acquisition and post ownership costs, of an analytical project are fairly easy to grasp, this is far less obvious as far as the benefits. Firms primarily invest in big data and analytics due to competitive pressure, rather because of their belief in its positive return.
Benefits of Investing in Big Data and Analytics
There are, however, tangible benefits to be realized. Just think about the new strategic opportunities that emerge by better targeting customer segments, identifying new product needs, or anticipating customer behavior. These benefits are hard to precisely quantify upfront. The fruits of the investment are harvested about 3 to 5 years after the initial investment, although reaping some low hanging fruits in the initial stages of a project is also an explicit concern, if only for the sake of management buy-in. As a result, it is important to adopt a long-term perspective when making investments.
Unfortunately, due to both internal and external pressures for immediate results, companies are far too often short-sighted, thereby, impeding the adoption of new technologies such as big data and analytics to foster sustainable growth.
Most companies are still only at the earliest stages of being able to leverage big data. For big data analytics projects to be successful, they will ultimately have to overcome these two main challenges. They will need to create both optimal organizational structures and commit to the investment in time and capital that will be required for big data and analytics projects to bear fruit.
The authors, Bart Baesens, Wilfried Lemahieu, and Monique Snoeck, welcome any shared experiences (both confirming and contradicting)and they are also happy to share information about their Master of Information Management program at KU Leuven in which the above topics are covered into more detail.
Image courtesy of Shutterstock