Hybrid-cloud and multi-cloud enterprise data architectures will become increasingly the norm as data sources and their data gravity shift from on-premise applications to SaaS applications and integrate more external data sources for data scientists. These cloud migrations can be lengthy and arduous due to the learning curve and skills development for cloud platforms and the data architecture redesign required to be cloud-native. Having a mindset whereby analytics and AI projects drive cloud adoption is a proven and effective way to make a business impact while refactoring the data architecture. Data scientists and supporting business analysts will need to have access to both on-premise data sources and newer, cloud-based data sources for building their datasets. This is where data prep becomes crucial to pull data from either environment. Data science and AI projects also influence how hybrid and multi-cloud architectures evolve due to the availability of specific AI services from one particular cloud vendor or another.
What’s Ahead
Based on Radiant Advisors’ research, the modern analytics lifecycle represents how data scientists and business analysts work together most efficiently to develop, govern, and deploy analytic models in the enterprise. The seven-step process was designed to align the requirements for a modern data analytics platform with functions needed to perform analytics. Additional foundational components include data catalogs, collaboration platforms, and API servers for deployed AI services. At its core, the modern analytics lifecycle is a tool to identify where to empower data scientists and business analysts to be more self-sufficient on the enterprise data platform itself.
Self-service data analytics and the modern analytics lifecycle are at the heart of architecting the modern enterprise data analytics platform. When fully optimized at scale, business analysts can work directly with data for a desired goal—data science workflows or enterprise data hubs analysis. For data science and AI projects, the modern analytics lifecycle is the journey to discover the best datasets to be used by data science and AI models. This innovative process with the freedom for the business analysts to focus on their desired outputs can later be “industrialized” into data engineering pipelines that automate and scale the functionality for enterprise deployments and impacts. This same innovative process with the freedom for business analysts to resolve issues and understand viability of data integrations can then be “institutionalized” by data management and governance functions for fully vetted enterprise data hub models and the data pipelines that populate them.
For more articles like this one, go to the 2020 Data Sourcebook
The goal should be to enable everyone in the enterprise to work proficiently with data and to create a data culture with the enterprise. With everyone’s business domain knowledge being effectively applied to data creation, this potentially offers exponential amounts of new data sources for data science and AI projects to take advantage of. However, keep in mind that the primary challenges in AI projects are data management, governance, and quality; therefore, self-service data analytics must incorporate aspects of data governance and collaboration.
AI may seem to have burst onto the enterprise data and analytics scene, but when you take a step back, you can understand how this is a natural progression that follows the era of big data. While data science searched for faint signals in the noise of data, machine learning and deep learning require meticulously crafted training datasets to build their analytic models. This is where data scientists, machine learning engineers, and business analysts benefit from having strong data management and governance features in their new modern enterprise data architectures. The ability to create quality, unbiased datasets will help companies realize the full potential of AI with data and analytics.