It’s no secret that data management has changed dramatically in the nearly 10 years since the onslaught of “big data.” A decade ago, the relational database management system (RDBMS) ruled the enterprise. While research shows that the RDBMS is still the backbone of most enterprise infrastructure, newer technologies are steadily making their way into organizations to allow them to tap into the unprecedented growth and variety of data. As a result, today, polyglot persistence is the mandate, and innovative NoSQL and NewSQL databases, cloud services, and data processing technologies have arisen alongside stalwart RDBMSs, data warehouses, and traditional on-premise deployments.
Simple, data management is not. Requirements are increasing but budgets are staying flat. “Do more with what you have” is the demand of CIOs, says Robert Reeves, CTO of Datical. “Companies want to take advantage of new technologies, like big data. Call it 'FOMO' (fear of missing out) or fear of “being Ubered,” he said, “but companies are desperate to avoid missing out on market opportunities or being disrupted.”
'Call it “FOMO” (fear of missing out) or fear of “being Ubered" but companies are desperate to avoid missing out on market opportunities or being disrupted.'
However, despite the increasingly challenging data environment, experts agree, the opportunities are boundless today for DBAs—provided they are willing to continue learning and growing and to step in to fill gaps where they open up.
DBAs’ Changing Roles
As we go deeper into the big data era, some industry experts see the role of the DBA evolving to be an amalgam of several different jobs with a more general expertise, while others have identified the emergence of specialty DBAs with distinct expertise.
Historically, the DBA controlled and managed “all things data” within the enterprise, observed Joe Caserta, president of Caserta, a New York-based innovation consulting and technology services firm. “The role of the DBA has transitioned into several new roles.”
This article is part three of a six-part series by editors of IOUG SELECT and Big Data Quarterly on "The Changing Role of the Modern DBA," with three articles appearing on SELECT and three articles appearing on Big Data Quarterly.
Caserta has defined the concept of a Corporate Data Pyramid (CDP), which includes a data ingestion area, a data lake, data science laboratories, and a big data warehouse, and, he said, each of the tiers in the pyramid can potentially require different skills to design, develop, and deploy.
“Rather than traditional DBAs, we now need different roles as well as specialists in the emerging technologies. Specifically, administrative or system DBAs should be knee-deep in understanding cloud technologies. We are also seeing these roles transform into DevOps specialists. Developer and application DBAs are no longer limited to SQL and most new development is now being done in Python. These DBAs may also migrate to an ‘integration architect’ role whose core responsibilities may include building microservice components or APIs to interface between the CDP ingestion area, data lake, data science workspaces, big data warehouse, and other business applications.”
With the rise of big data, responsibilities for protecting data in the face of new risks become acute.
As the velocity and types of data continue to accelerate, a vortex that corporate data falls into is created, said Michael Corey, co-founder of LicenseFortress, and a Microsoft Data Platform MVP, Oracle ACE, and VMware vExpert. “As organizations race to exploit this information, many of the traditional safeguards on data go by the wayside. The more progressive DBAs understand this problem and are stepping in to protect their organization from the organization’s need for speed before it’s too late. Over time, traditional DBAs will transform to becoming DBAs of the ‘dataverse’ (Hadoop, NoSQL, etc.) and, whatever form or velocity the data takes, they will make sure it’s accessible, accurate, and secure.”
And, as data stretches across the cloud and different geographic barriers, the modern DBA will need to understand the rules of law that apply, said Corey. “What data needs to be retained and for how long?”
‘Over time, traditional DBAs will transform to becoming DBAs of the "dataverse" and, whatever form or velocity the data takes, they will make sure it’s accessible, accurate, and secure.’ – Michael Corey
Amidst the complexity of many more data types and management systems, there is also the need for a higher-level, seasoned data expert who has a comprehensive view of the infrastructure and how to access critical information. With the rise of big data, DBAs are evolving to higher-level roles in their organizations, observed Joe McKendrick, Unisphere Research lead analyst and independent author. “They are moving away from being administrators of specific database systems and maintaining rows and columns and security. They increasingly play more of a consulting role within their organizations, helping business users and executives identify and connect with the data sources they need to make decisions. In addition, DBAs today need a more generalist approach to data technologies, expanding their ability to manage data from any type of data environment, in any format.”
New Big Data Challenges
With the rise of big data, there are new challenges to make more information useful for business insight, which requires greater governance—and that responsibility also falls on the shoulders of DBAs.
For at least the last 10 years, DBAs have been challenged on multiple fronts, said Guy Harrison, CTO of Southbank Software, Oracle ACE, and a MongoDB certified DBA. “The commercial databases such as Oracle increasingly automate bread-and-butter DBA tasks; DBA work is being offshored to low-cost locations; databases are moving into the cloud and managed by or within those clouds; and the DevOps movement is undermining the traditional barriers between development and production environments.”