Cloudera is announcing the general availability of DataFlow Functions (DFF) for Cloudera DataFlow within the Public Cloud on AWS, Microsoft Azure, and Google Cloud Platform. With a no-code UI for running NiFi flows in a serverless mode, users can optimize cost expenditure and decrease resource waste at scale.
“Most people have the problem of bringing the data in. What Cloudera DataFlow does is provide a no-code, low-code solution—with about 450-plus connectors—that allows you to quickly create these data collection and data movement pipelines,” said Sudhir Menon, chief product officer at Cloudera. “It’s a completely open solution; it prevents vendor lock-in and allows you to create data pipelines from the edge to wherever your data sources might be—and do so without relying on destination specific target solutions.”
DFF allows developers to minimize manual, laborious function startups through a no-code interface. No longer will they need to write code and rely on code samples to initiate function processes; NiFi flows can be designed quicker and directly run as functions in AWS Lambda, Azure Functions, and Google Cloud Functions.
Among the benefits of DFF is its running in serverless environments within public clouds, which yields a reduction in infrastructure management resources. It is entirely fitted for event-driven use cases, according to the vendor, as flows are only executed when an event triggers the function—saving time and money on operational responsibilities associated with managing and maintaining NiFi flow runtime environments.
“Every industry has the need to collect data from a variety of data sources and get it to places where they can transform it, get value from it, and ultimately use it for unstructured and structured analytics, as well as conduct ML and model training on that data. Plus, the number of sources has exploded,” said Menon. “Any organization that needs a rapid development framework that allows them to connect from the edge to any major public clouds, as well as SaaS solutions like Snowflake, can benefit from Cloudera DataFlow.”
Cloudera now offers two NiFi runtime options in the public cloud to address specific use-cases: DataFlow deployments and DataFlow Functions. With deployments, runtime is optimized for high-throughput, low-latency streaming use cases. Functions runtime is designed for event-driven, short-lived use cases.
To learn more about Cloudera DataFlow, please visit https://www.cloudera.com/.