Big Fragmented Data: Making Sense of Fragmented Search

The search landscape is changing, and the days of desktop search engine dominance are waning. An emarketer study predicts desktop search to decline $1.4 Billion as Google users shift to mobile. People are moving away from because internet search can now be performed in multiple ways - on different channels and device types - and today’s internet users have an unprecedented amount of choice.

As search becomes more fragmented, search advertising success will come to those companies capable of delivering well-targeted, prescient results. The future of search advertising will not depend wholly on our ability to match keywords, but instead on our ability to utilize disparate intent data to give people what they want before they even ask for it.

I worked at AOL during the mid-1990s. At the time, AOL was very successful at getting people online for the first time, and a lot of this success stemmed from AOL’s ease of use. AOL’s home screen displayed a collection of icons called “channels” that represented general interests like weather, travel, sports, finance, love and romance, and shopping. Today, when I look at my iPhone and iPad, I see icons that look very similar to the original AOL homepage. Apps are bringing back curated content and replacing the need for traditional search.

In the late 1990s, as the internet experienced rapid expansion, search engines rose to prominence out of necessity, and curated content fell out of fashion. The rate and volume of new website creation yielded an unprecedented amount of information. We needed search engines to navigate this vastly expanding library of information, because we didn’t have the technology to give people what they wanted before they asked for it. Now, the mobile revolution has revitalized the need for searchable curated content. Mobile internet use and mobile search advertising spend are both predicted to overtake desktop usage and spend this year, and as people change the way they access the internet, they change the way they search.

This shift away from search engines marks the biggest digital advertising challenge of the decade. provided the best performance advertising channel in the history of marketing, but as search fragments into specialized, disparate pieces, marketers are struggling to keep up. This disruption presents a huge opportunity. There are currently hundreds of display advertising companies trying to make sense of mounds of user data through cookies. However, display ads have never performed as well as SEM, cookie data is constantly on the verge of disappearing, and iOS has already made cookie tracking impossible on many mobile devices. The new search climate calls for a new advertising model.

Every app, and every web publisher, is essentially a mini-search engine. We chose our specific sources and then trust these sources to provide us with the information we want. We use Kayak to search for travel deals, Yelp and YP to search for local restaurants and activities, and Twitter to search for trending news stories. Because search is now divided between thousands of different sources, search advertisers must aggregate this data and use it accordingly.

Ultimately, search marketing will increasingly rely on more than keywords to determine user intent. As we start to add intent data from user behavior outside keyword search, our big data problems are going to get exponentially bigger - this is scary, but it is also exciting. Several companies like Drawbridge, TapAd, Adtruth, and Flurry have raised tens of millions of dollars to find new ways to identify and track users across multiple device types, and they haven’t come up with much besides complex and imprecise heuristic algorithms. Everyone sees the forthcoming change, but no one has quite figured out how to tie all the various data and intent signals together.

The “Post Google Era” is here. Future digital advertising models, search or otherwise, will inevitably be more complex than their existing counterparts. Effective advertising is, and has always been, about aligning ads with consumer intent. Our search behavior is now fractured, and the clues we reveal about our true intent will continue to propel innovations in ad tech. Companies that are able to use big data effectively match ads targeted to user search intent -- across multiple devices and channels at scale - will be the real winners of the $30B direct response advertising market.


Michael Yudin is CTO of adMarketplace, a leading programmatic marketplace for search intent advertising. adMarketplace’s predictive algorithms adjust advertiser keyword bids by traffic source and device type based on performance data. Yudin has 20 years of engineering experience including AOL and Ubique, one of the very first successful internet startups. He received a Masters in Computer Science from the Weizmann Institute in Rehovot, Israel, and is an expert on information theory, data retrieval and cryptography. @mikeyudin