Representing a continued expansion of its big data analytics portfolio, IBM has introduced a new addition to the Netezza product family of analytics appliances that is designed to help organizations uncover patterns and trends from extremely large data sets. The appliance is the first to be delivered by IBM since it acquired Netezza in November 2010.
The new IBM Netezza High Capacity Appliance aims to address a growing challenge for organizations such as banks, insurance companies, healthcare organizations and communications services providers that are collecting vast amounts of data which in some cases they are required by industry regulators to retain for up to a decade. As data retention laws continue to evolve, organizations are faced with the challenge to store and analyze ever-expanding volumes of data that may not be directly related to daily operations, yet still hold potential business value.
According to IBM, using the new appliance, businesses can now more easily sift through petabytes of data, including banking and mobile phone transactions, insurance claims, electronic medical records, and sales information, and they can also analyze this information to reveal new trends on consumer sentiment, product safety, and sales and marketing effectiveness.
The new high capacity appliance, joins the IBM Netezza family of data warehousing appliances which also includes the IBM Netezza TwinFin appliance for high performance BI and enterprise analytics that allows customers to process up to a petabyte of data at very high speed, and the Skimmer appliance for entry-level data warehousing.
"This new appliance takes the scalability to a completely new dimension," says Razi Raziuddin, senior director of product management at IBM Netezza. The appliance scales to more than 10 petabytes, giving organizations the ability to store longer histories of data with more details, and allows them to analyze up to 10 petabytes of data in a matter of minutes. The maximum storage capacity of the largest appliance is somewhere between 11 to 12 petabytes, depending on how compressible the data is, Raziuddin says.
"It is phenomenal in terms of how much customers can store in it and, in terms of price because at the end of the day it is all about price/performance, this is a system that makes data warehousing for high capacity use cases really affordable and very economical," he notes. The dollar per terabyte pricing on this is under $2,500, "which sets a new benchmark in the market."
The growth of data and the mushrooming of data sources, regardless of industry, is not news, but where the problem is really becoming acute is in regulated industries where companies are required to keep multiple years' worth of transactional history online, Raziuddin points out. Purely because of economic reasons, the majority of this historical data often ends of in Tier 3 storage, tape drives or disk farms, where it may be online but not easy to analyze, says Raziuddin. With the new appliance, companies can take this data and bring it into a system which he describes as a "queryable archive." According to Raziuddin, the low-cost-per-terabyte system allows organizations to store and query the data through SQL as well as run in-database analytics, so they can derive beneficial insights not possible before.
For example, in the area of financial services and banking, by building quantitative models, organizations could derive tremendous benefit out of analyzing 10 years' worth of trade information, but that was something that was not easily feasible for accessible before from a price/performance perspective. "That is one really interesting application of this appliance," says Raziuddin. Pointing to another example, Raziuddin says that although utilities typically were not data-rich, with smart grids which these organizations now need to manage, they all face a huge data challenge, as well.
IBM Netezza appliances are workload-optimized systems based on IBM BladeCenter technology. The new appliance can be up and running in 24 hours and, according to Raziuddin, it "shares the DNA" with the other appliances that Netezza has brought to market, with the same architecture, same software running, and offering all of the features.
IBM Netezza technology joins IBM's Hadoop-based BigInsights software, as well as Streams software - which both originated in IBM Research - creating a portfolio of workload-optimized big data technologies. The software incorporates Watson-like technologies, including unstructured text analytics and indexing that allows users to analyze rapidly changing data formats and types on the fly.
IBM recently announced a $100 million investment for continued research on technologies and services that will enable clients to manage and capitalize on data as it continues to grow in diversity, speed and volume.
The new appliance was in development prior to the acquisition but was accelerated based on the support Netezza received from IBM, says Raziuddin. "You will see a lot more coming in the not-so-distant future from Netezza based on all the R&D investment that has been made in the 6 months or so since the acquisition," Raziuddin adds.
For information about IBM Netezza, go to www.netezza.com.
For information on IBM business analytics, go to www.ibm.com/analytics.