Hortonworks, Inc. is advancing its Hortonworks Data Platform (HDP) with the in-memory analytics capabilities of Spark.
Apache Spark 1.5.2 includes support for Spark SQL and Spark Streaming and will be available by the end of this month on the platform.
“The important things about the update are delivering on some of the key capabilities that our customers have been asking for, particularity around Spark SQL and Spark Streaming,” said Tim Hall, vice president of product management at Hortonworks.“These are two things that have been developed within the community over time and folks have been really eager to take advantage of. We believe they’re ready for the enterprise now to use.”
Hortonworks’ commitment to Spark is focused on assisting customers to accelerate data science, maintain seamless data access, drive innovation at the core, and ultimately scale for the enterprise.
By collaborating with Spark and integrating its abilities, users can deploy Spark-based applications alongside Hadoop workloads in a consistent, predictable and reliable way.
“The thing that customers are really excited about is really being able to access data very seamlessly across all the investments that they’ve made in the Hadoop ecosystem technologies over the last few years and Spark fits very nicely into that ecosystem,” Hall said.
In order to meet the consumer demands, Hortonworks with Spark will focus on improving data science productivity by enhancing Apache Zeppelin (currently available as a technical preview) and by contributing additional Spark algorithms and packages to ease the development of key solutions., refining Spark’s integration with YARN, HDFS, Hive, HBase and ORC because customers are running Spark on YARN in combination and in conjunction with many of the other popular data access engines, and enhancing Spark’s enterprise security, governance, operations and overall readiness for real-world production deployment.
Along with Spark integration, Hortonworks has launched Hortonworks Community Connection (HCC).
HCC is a new online collaboration destination for developers, DevOps, customers, and partners to get answers to questions, collaborate on technical articles and share code examples from GitHub.
This new online community is an extension of Hortonworks’ open source roots and underscores its commitment to engaging with the community, and fostering community innovation and knowledge. More information about Project Magellan can be found within HCC.
“What we’re trying to do is provide a super compelling experience for all the folks that work with us so that they can ask and answer questions that they can share and collaborate online including finding access to unique and compelling technology as they are being developed within the broader ecosystem,” Hall said.
For more information about Hortonworks and Spark along with the new Community Connection, visit www.hortonworks.com.