Image courtesy of Shutterstock.
Recent venture funding and acquisition announcements from key Hadoop and NoSQL startups are drawing even more attention to the Big Data space. Hadoop and NoSQL companies are growing in numbers, in size and public awareness; and as a result, the buying IT community is quickly realizing the opportunity of NoSQL database technology.
The database market is heating up again, but not because of its incumbent players. According to Wikibon, “the market for Hadoop/NoSQL software and services topped $540 million in 2012 as measured by vendor revenue. Over the next five years, Wikibon forecasts this market to grow to nearly $3.5 billion, a compound annual growth rate (CAGR) of 45%.” Many believe that the biggest technology sea change in terms of data will be the move to Hadoop-based systems for storing and processing vast quantities of data. But Wikibon also states, “Of the two market sub-segments, NoSQL is the slightly larger. Revenue for NoSQL software and services made up $286 million in 2012 and is forecast to reach $1.825 billion in 2017.” I believe the market has a lot to look forward to in the next three years.
These findings suggest that, indeed, the database industry is currently going through a shift, where the database is, again, the killer app.
So, what’s happening here? This NoSQL boom is a result of the upsurge of data in a variety of data formats and from a variety of sources (corporate, machine-to-machine, social, mobile, sensor, digital, etc). Increasingly innovative technology in the hands of users has contributed to an explosion of new data. Businesses struggle and ultimately fail to keep up with the growth curve when they stay with a strategy based upon relational database rows and columns, because that model is not as accommodating for all of these new data sources.
To enable flexible application integration in the enterprise, a database must support a wide variety of data types. In most enterprises, the typical data formats include:
- Tabular data from relational databases (Oracle, Sybase, SQL Server)
- Common desktop file formats (Microsoft Office, PDF, CSV, text files)
- Binary (audio, video, and image formats)
- Formats used for interaction within and between systems (JSON, XML)
- Semantic data (RDF)
- Geospatial (GeoJSON, KML, shape files, XML geospatial formats)
Today’s database must be able to natively support each of these data formats so that they can be leveraged to implement powerful applications using data in existing infrastructure with limited ETL and data normalization required. Another hurtle enterprises struggle with is that their developers are used to legacy relational databases. But NoSQL development does not have to require a new, arcane skillset. If the right NoSQL product is chosen, developers can stick with their preferred programming language. So along with being able to support all the different data types, the modern database must work well with all the programming languages that developers are using today. The most common programming languages are:
- JavaScript
- Java
- Python
- Ruby
- SQL (remember, NoSQL means “not only SQL”)
- C and C++
- Perl
- Microsoft .NET languages including, C# and VB.NET
- XQuery, XPath, and XSLT
- SPARQL
Customers are executing proactive data management strategies to stay ahead in the race to store, manage and mine data more efficiently. These new strategies leverage NoSQL database technology, a more flexible paradigm for data management.
NoSQL brings schema flexibility so data models can easily change as data types evolve. NoSQL reduces the engineering time and expense typically associated with typical 3-tier RDBMS applications. It cuts out the analysis and engineering costs associated with data modeling. NoSQL reduces the cost of application development and re-factoring cycles needed to marshal data rows into data objects and the overall maintenance of a brittle database and application architectures that slow customers down and add cost, when all they want to do is move faster and do more with less.
But not all NoSQL database software is the same. The flexibility and rapid time-to-results are just the first major steps to entice prospective customers to move from their traditional database architectures to NoSQL. NoSQL has to scale, it needs to be secure and highly available. Enterprises want their data systems to support and guarantee transactional consistency. They can’t afford to lose data or endure the tribulations that follow the theory of ‘eventually consistent’ databases. Enterprises today need to support real-time business needs and their database needs to be fast, highly functional and open before they pick a database for mission critical applications. In short, NoSQL, just like relational has for years, needs to be ready for the Enterprise.
Database solutions today must be able to support extremely diverse application use cases that cross all major verticals for core business and mission applications.
Data is king, and the demand to find new ways for storing, processing, analyzing, and mining data continues to provide green fields of opportunity for non-traditional relational database vendors.
About the author
Jonathan Bakke, senior vice president, Global Services for MarkLogic, is an industry expert in data management and software development, having worked at companies that include Oracle, Hewlett Packard (EDS), Northrop Grumman, and Sprint/Nextel.