Perhaps no technology is more aligned with the new world of big data than Hadoop.
The Apache Hadoop framework allows for the distributed processing of large datasets across compute clusters, enabling scale up from single commodity servers to thousands of machines for local computing and storage. Designed to detect and handle failures at the application layer, the framework supports high availability.
Initially developed by Doug Cutting, a Yahoo! engineer at the time, the Apache Hadoop open source framework was famously named after his son’s toy elephant, with Facebook, Twitter, LinkedIn, Yahoo!, and Amazon leading the way as the technology’s earliest adopters.
But today, with the explosion of big data, the interest in the technology has expanded well beyond that initial group. For a wide range of companies, data is seen more than ever before as a valuable enterprise resource and Hadoop holds is seen as being one of the key technologies to being able to store and analyze more data than ever before for longer periods. An important aspect to the framework is that it that allows for the distributed processing of large datasets across clusters of commodity hardware and can scale from single server to thousands of machines, each offering local compute and storage, with highly available service on top of clusters.
Along with a rich and growing assortment of projects flourishing as part of the main Apache Hadoop project, there is also an expanding array vendors offering enterprise distributions as well as related tools and services.
This is increasingly helping to make Hadoop more robust, secure, and enterprise-hardened.
HERE ARE THE WINNERS OF THE 2015 DBTA READERS' CHOICE AWARDS FOR BEST HADOOP SOLUTION
Winners' Circle by Tom Reilly, CEO
As organizations evolve to become more data-driven, a new approach to working with data is required. Cloudera offers an innovative new solution—an enterprise data hub, powered by Apache Hadoop—that extends existing architectures to help customers turn unlimited data into value at lower cost. We help solve your most challenging business problems with data...read on.
Amazon Elastic MapReduce