At Strata + Hadoop World in New York, Microsoft announced an update to Microsoft Azure HDInsight, its cloud-based distribution of Hadoop. Customers can now process millions of Hadoop events in near real time, with Microsoft’s preview of support for Apache Storm clusters in Azure HDInsight.
In addition, as part of its integration with the Azure platform, Hortonworks announced that Hortonworks Data Platform (HDP) has achieved Azure Certification. HDP 2.2 will be generally available next month, and, as part of the HDP 2.2 release, Hortonworks will deliver hybrid data connectors so customers can extend their on premise Hadoop deployments to Azure and leverage the cloud for backup scale and testing.
Microsoft has also introduced new machine learning capabilities in the Azure Marketplace so customers can access the capabilities as web services. These include a set of R packages, a popular programming language used by data scientists, and a recommendation engine to help improve search results and anomaly detection for fraud prevention, as well as an anomaly detection service.
“Big data, including Hadoop and advanced analytics, is changing the way our customers do business,” said TK "Ranga" Rengarajan, corporate vice president, Data Platform, Microsoft, in a recent blog post. “As they collect and store more data than ever before, they expect more from their data and want more insights from it, including being able to do real-time analytics over streams of data to complement their existing Hadoop deployments. Microsoft’s approach is to make it easier for our customers to work with data of any type and size — using the tools, languages and frameworks they want — in a trusted cloud environment.”
The preview availability of Storm in HDInsight continues Microsoft’s investment in the Hadoop ecosystem and HDInsight, said Rengarajan, noting that Microsoft recently announced support for HBase clusters and the availability of HDInsight as the first global Hadoop big data service in China.