Image courtesy of Shutterstock
Modern data architecture doesn’t just happen by accident, springing up as enterprises progress into new realms of information delivery. Nor is the act of planning modern data architectures a technical exercise, subject to the purchase and installation of the latest and greatest shiny new technologies. Rather, the design and creation of modern data architectures is an uplifting process that brings in the whole enterprise, stimulating new ways of thinking, collaborating, and planning for data and information requirements. It’s an opportunity for business decision makers to sit down with IT colleagues and figure out what kind of business they want to be in, what kinds of information they seek to propel that business forward, and what needs to be done to capture and harness that information.
One thing is clear: The old models of data architecture aren’t enough for today’s data-driven business demands. An architecture designed a decade ago, that rapidly and seamlessly moves data from production systems into data warehouses, for example, may not be capable of meeting the needs of today’s real-time, data-driven enterprises.
For more articles on Moving to a Modern Data Architecture, access DBTA's Best Practices Special Section.
Architecture is more important than ever because it provides a road map for the enterprise to follow. Without a well-planned, careful, deliberate approach to data architecture, another type of architecture rises to take its place—a “spaghetti architecture” approach that occurs when every business unit or department sets out to buy its own solutions.
Here are the essential components that need to go into building a modern data architecture:
- WORK WITH BUSINESS USERS TO IDENTIFY THE TYPES OF DATA THAT ARE THE MOST VALUABLE - The purpose of good data architecture is to bring together the business and technology sides of enterprises to ensure they are working to a common purpose. To be of value, information needs to have a high business impact. This data may have been within enterprise data environments for some time, but the means and technologies to surface such data, and draw insights, have been prohibitively expensive. Today’s open source and cloud offerings enable enterprises to pull and work with such data in a cost-effective way.
- MAKE DATA GOVERNANCE A FIRST PRIORITY - Working closely with the business side requires guarantees that data not only be of value, but that it is also well-vetted. The process of identifying, ingesting, and building models for data needs to assure quality and relevance for the business. Responsibility for data must be established—whether it’s individual data owners, committees, or centers of excellence.
- BUILD SYSTEMS TO CHANGE, NOT TO LAST - A key rule for any data architecture these days it is not wedded in any way to a particular technology or solution. If a new solution comes on the market —the way NoSQL arose a few years back—the architecture should be able to accommodate it. The types of data coming into enterprises can change, as do the tools and platforms that are put into place to handle them. The key is to design a data environment that can accommodate such change.
- DEVELOP A REAL-TIME FOUNDATION - A modern data architecture needs to be built to support the movement and analysis of data to decision makers and at the right time it is needed. Also, it’s important to focus on real-time from two perspectives. There is the need to facilitate real-time access to data, which could be historical; and there is the requirement to support data from events as they are occurring. For the first category, existing infrastructure such as data warehouses have a critical role to play. For the second, new approaches such as streaming analytics are critical. Data may be coming from transactional applications, as well as devices and sensors across the Internet of Things and mobile devices. A modern data architecture needs to support data movement at all speed—whether it’s sub-second speeds, or with 24-hour latency.
- BUILD SECURITY INTO THE FOUNDATION - A modern data architecture recognizes that threats are constantly emerging to data security, both externally and internally. These threats are constantly evolving—they may be coming through email one month, and through flash drives the next. Data managers and architects are in the best and most knowledgeable position to understand what is required for data security in today’s environments.
- DEVELOP A MASTER DATA MANAGEMENT STRATEGY - With a master data management repository, enterprises have a single “gold copy” that synchronizes data to applications accessing that data. The need for an MDM-based architecture is critical—organizations are consistently going through changes, including growth, realignments, mergers, and acquisitions. Often, enterprises end up with data systems running in parallel, and often, critical records and information may be duplicated and overlap across these silos. MDM also assures that applications and systems across the enterprise have the same view of a customer, versus disparate or conflicting pieces of data.
- POSITION DATA AS A SERVICE- Many enterprises have a range of databases and legacy environments, making it challenging to pull information from various sources. Access is enabled through a virtualized data services layer that standardizes all data sources—regardless of device, applicator, or systems. Data as a service is by definition a form of internal cloud, in that data—along with accompanying data management platforms, tools, and applications—are made available to the enterprise as reusable, standardized services. The potential advantage of data as a service is that processes and assets can be prepackaged based on corporate or compliance standards and made readily available within the enterprise cloud.
- OFFER SELF-SERVICE ENVIRONMENTS - With self-service, business users can configure their own queries and get the information or analyses they want, or conduct their own data discovery, without having to wait for their IT or data management departments to deliver the information. In the process, data application can reach and serve a larger audience than previous generations of more limited data applications. The route to self-service is providing front-end interfaces that are simply laid out and easy to use for business owners. In the process, a logical service layer can be developed that can be re-used across various projects, departments, and business units. IT still has an important role to play in a self-service-enabled architecture—providing for security, monitoring, and data governance. There is a new generation of tools and templates now available from vendors that enable users to explore datasets with highly visual, even 3D views, that can be adjusted, re-adjusted, and manipulated to look for outliers and trends.
For more articles on Moving to a Modern Data Architecture, access DBTA's Best Practices Special Section.