Back in the 1990s, business intelligence (BI) initiatives often consisted of software tools applied to structured datasets (like corporate ERP) to enable more accurate analysis of past events as a means of preparing for and anticipating the future. For example, analyzing past customer purchasing data in CRM systems could help guide the next round of marketing – sending mail catalogs and targeting promotions to past or repeat customers.
The Changing Landscape of Enterprise Data
Today, the business environment is so fast-paced and competitive that reactive marketing intelligence strategies based on historical data will no longer suffice. Marketing intelligence must be real time, if not predictive, to be successful. A large part of the problem is prospective customers have limited attention spans. Social media, the web and mobile devices have resulted in human attention spans now being officially shorter than those of goldfish. With all this over-stimulation, the most successful companies will be those that can grab customers’ attention in the moment via the medium of their choice and deliver as promised.
Ironically, while social media, the web and mobile devices present challenges in terms of shrinking attention spans, they also present a tremendous opportunity. The volumes of unstructured data generated can fuel a whole new level of marketing and sales. The key is to harness and analyze this data, in concert with structured data sets as part of a larger, enterprise-wide big data initiative.
Examples in Action
The problem with “post-mortem” approaches to information analysis is that they rely on historical information, which is not always a clear indicator of the present or future. The “Cabbage Patch Riots” in the 80’s – when crowds got extra rowdy and trampled displays – and even the “Black Friday Riots” of the 2000s exemplify the shortcomings of this approach. Even more recently, several retailers were caught off-guard by the sudden spike in demand for hand sanitizer, which resulted from the news of the S1N1 flu outbreak. Retailers could not detect growing demand in real time, let alone anticipate and plan for the future – resulting in lost sales and revenues.
The bottom line is that traditional BI just moves too slowly; by the time data is gathered, analyzed and made actionable, it is too late and the opportunity is missed. Big Data analytics and the ability to analyze live, fast-changing data and provide immediate feedback takes BI to the next level and creates amazing new opportunities.
Consider the example of a modern day retailer that wishes to become the sales leader for a hot up-and-coming electronics product. In the past, all a retailer could do was ensure they were positioned to react to demand as quickly as possible. Today, this paradigm has changed and the retailer is now using Big Data analytics to predict trends and prepare for future demand, which may be months away.
By analyzing unstructured data, including web browsing behaviors, forums, onlinereviews and social media sentiment, combined with structured data such as customer loyalty and purchase history, as well as several other relevant factors retailers can predict when, where and for which customer segments demand will be the greatest. This allows retailers to stock up and deliver the right number of product units to the right channels. This is a stark departure from the days of the “Cabbage Patch Riots"; for successful, data-driven retailers, those days are over for good.
Another example to consider is the gaming company, which recently had tremendous success by aligning an unusual form of semi-structured log data (frequency of taps on a mobile device screen) with structured customer churn data. The company was able to draw a correlation between how quickly and repeatedly gamers hit the bonus round button on their device screens and when these gamers would churn. Declining fast bonus round button-hitting, so it proved, was a one month advance leading indicator that gamers are losing interest, which then allowed the gaming company a window of opportunity to retain the gamer.
What’s Causing Hesitation?
Beyond social media and web browsing data today there are a slew of additional unstructured data sources like voice mails, images and signal data from connected devices, all of which can be mined for greater operational intelligence. Unstructured data volumes worldwide are exploding, according to Wikibon, on Facebook alone, 30 billion pieces of content are shared every month, and brands and organizations on Facebook receive almost 35,000 “likes” every minute of the day.
It’s natural for organizations to feel as if they can’t possibly harness all of this data but competitors will, and those that do will quickly gain a competitive edge.
What’s holding the hold-outs back?
- Unwarranted reticence about scrapping existing BI investments: As the examples above show, legacy BI investments have a very big part to play in modern Big Data analysis initiatives. Choosing to analyze structured versus unstructured data is not an either/or proposition, and in fact, the greatest insights are often derived when these data sets are analyzed in parallel. However, traditional BI approaches *will* need to evolve and grow.
- False fear of big up-front investments: Businesses today can use the cloud to securely set up discovery environments incorporating all types of data – structured and unstructured – in just one or two weeks. This means organizations can start quickly and with minimal new hardware expenditures, which represents a huge industry advancement and helps avoid technology obsolescence.
- Lack of “Data Scientists”: It’s true that there is a lack of so-called data scientists, or individuals who can analyze data of various types and spot trends, particularly among small businesses. While data scientists are important, the goal of incorporating unstructured data into a Big Data analytics strategy should be to make *all*workers more agile, regardless of the type of information they’re working with. Successful innovators have this mindset and encourage creativity and experimentation at all levels of the company through broad availability to data and advanced analytics.
The Real Fear Should Be About Missing Untapped Opportunities
Today’s organizations don’t need to be hesitant or “scared” of unstructured data. In actuality, the real fear should lie in not tapping these resources. Fortunately, there are tools available on-demand to help manage unstructured data well.
In addition, services can help companies get off to the most successful start possible in this new territory of information management and analysis. BI professionals have been ingesting and processing data for decades, and these new sources and volumes of data can unleash significant insights and opportunity.
The truth is, it’s a wonderful time for companies of all sizes to start analyzing and leveraging their semi-structured and unstructured data sets. Many companies are doing this and having “happy accidents” – unlocking unanticipated correlations across data types that are helping them make smarter decisions and drive the business forward. The key is to get started now – it’s ok to experiment – and see what can happen when unstructured data is put to work.
The magic really starts to happen when all datasets – structured and unstructured, inside and outside the enterprise – are analyzed in parallel. This can now be done in hours as opposed to traditional BI and enterprise data warehouse projects which often took months and years. The time is now to stumble upon your next insights that will drive your business.