The rise of DevOps, which promises to bring development and operations teams into alignment, is bringing database managers and administrators ever closer to the key touchpoints of their businesses. DevOps seeks to ensure the continuous delivery and consistent cadence of software releases and means that data professionals will have a key role to play in their organizations’ information technology strategy. It’s not happening a moment too soon—nimble, tech-savvy competitors have learned to leverage data as strategic business assets and are biting ever deeper into just about every existing market.
Speed is of the essence, pushing existing companies to work harder “to move fast and better align development and IT ops teams,” said Patrick Chanezon, chief developer advocate at Docker. As a result, “companies are turning to container technology and DevOps to gain operational efficiencies and a platform for innovation.”
DevOps methodologies “have been the de facto model for startups and born-in-the-cloud organizations for many years,” agreed Bob Quillin, vice president of the Oracle Container Group. “Now those approaches and techniques are moving to larger teams and organizations based on clear business advantages, cultural benefits, and the emergence of repeatable patterns driven by open source technologies tested at cloud-scale by these early adopters.”
Containers are built on microservices, which in turn are supported by cloud computing. This culmination of new technology architectures forms the not-so-secret sauce that makes DevOps possible—enabling DevOps teams to “iterate on and patch applications at a greater rate to stay at the forefront of innovation,” Chanezon said.
Quillin described the connection between containers, microservices, and DevOps this way: “Technology has advanced over the last few years with the introduction of a broadly available set of open, container-native technologies such as Docker and Kubernetes that have become the killer apps for DevOps.” This, he said, effectively links development and operations together “through a common container structure that allows the same container artifact to move through development, test, staging, and production using open, automated technologies available to everyone.”
The final piece is automation, which reduces complexity and administration and ensures higher reliability, security, and operational efficiency at the lowest cost. “DevOps relies on these same automation principles to modernize key processes to build, deploy, and run software—automating testing, software deployment, and application orchestration.”
Containers and microservices also open the door to DevOps within legacy environments—of which there are many. According to Chanezon, DevOps was initially tied to greenfield applications, and continuous integration/continuous deployment processes would be set up around these applications and powered by containers. “But a majority of the applications sitting within enterprises are legacy apps,” said Chanezon. Businesses want to get the most out of these existing investments and they can by containerizing existing applications to make them more secure, cost-efficient, and portable. It gives companies an entry point into the container world and they can gain the immediate benefits of containers, while adding greater functionality through microservices at a pace that best suits their business needs.”
DevOps is arising out of a rapidly decentralizing IT function, with its ever-expanding mandate to deliver digital capabilities to the business. “At the turn of the millennium, enterprise IT organizations were being centralized under CIOs to impose top-down governance in reaction to both the opportunities and risks presented by the web and its associated technologies,” said Matt McLarty, vice president of the API Academy at CA Technologies. “This generally slowed down IT’s delivery, and with the new opportunities presented by digital business, mobile, and cloud technology, the pendulum has swung toward decentralization.”
DevOps Gains
A DevOps approach—which is bundled with microservices and containers—delivers a much higher degree of flexibility to IT systems than previously possible, proponents contend. It provides an “easier ability to scale software with the use of containers and orchestration software like Kubernetes,” said Jim Scott, director of enterprise strategy and architecture for MapR. “This move also pushes more software toward message-driven architectures. This ensures loose coupling and easy replacement of software components. These technologies bring a major benefit which is not requiring a full re-architecting of the software when there is a need to scale the components.”
A fully functioning DevOps process also enables greater collaboration and teamwork. “What’s needed is an approach that brings the agility, automation, and collaboration of DevOps to these data science and engineering teams,” said Jason Schroedl, vice president of marketing at BlueData. “They need to operationalize the data science lifecycle in a streamlined and repeatable way. They require an agile and lean process that enables them to iterate quickly and fail fast. They need the ability to easily share data, models, and code in a secure distributed environment. And, they need the flexibility to use their own preferred tools and try out new technologies.”
In the long run, DevOps, containers, and microservices will change the way organizations themselves are managed. “Microservices allow you to build apps which are loosely coupled and easier to update and scale independently,” said Michael Elder, IBM distinguished engineer for the IBM Cloud Private platform. “With a loosely coupled architecture, you can also build loosely coupled organizations—smaller, more independent teams which operate on a particular set of apps from end to end. You can intentionally design an organization which can move with greater agility because you’ve engineered an architecture which allows it.”
DevOps Challenges
Changes in data modeling and the use of containers and microservices require organizational and cultural changes, which experts agree may be more challenging than technical modifications. As DevOps is an expansive initiative, it’s essential that people across the organization be prepared and bought into the concept. However, as famously expressed by management guru Peter Drucker: “Culture eats strategy for breakfast.” Organizational culture “is the number-one factor which will determine how quickly an organization can embrace and implement DevOps,” said Elder. “Many organizations are entrenched in their silos, and it’s challenging to break out of that mentality and really begin operating as a single unified team with consistent end-to-end accountability for what is developed and then delivered. Often, development teams ultimately take on significantly more responsibility for the health and quality of their running services; and operations teams must adopt many of the practices that have become the fabric of agile development teams.”
Cultural evolution takes time and organizational commitment to see it through, Quillin agreed. “It requires visionary leadership, communication, and recognition that gives the process the time it needs for training and learning-curve issues.”
And, once organizations get past some of the cultural and operational hurdles and start putting all the pieces together, they face another challenge—scaling, said Tim Buntel, vice president of products at XebiaLabs. “They see their early successes with a greenfield project or a handful of applications that use a narrow set of technologies and want to duplicate that success to multiple applications across the organization,” explained Buntel. “Just as you can’t tool your way to success, you can’t script your way to scalability. As organizations mature in their DevOps implementation, the next struggle becomes, ‘How do I know if it’s working? Is it optimizing the value of software delivery to the business?’ The next step is figuring out how to measure the end-to-end value stream so they can focus their efforts on what’s most important for delivering business value.”
There has been a great deal of concern about the security of DevOps initiatives, since code and data are changing hands on a frequent basis. “DevSecOps” refers to building security in from the outset, while “cognitive DevOps” also aims to improve the DevOps process. However, not everyone sees the terms as descriptive. According to Scott, “Security of data is a fundamental consideration to all system designs. If security is ignored at the onset then it shows there was a clear process flaw by ignoring it. If it is not ignored at the onset then it is just DevOps.”
With the emphasis on “speeding the delivery of high quality software, security has been somewhat of a blind spot,” said McLarty. He added that he has seen more attention being paid to security considerations lately. “Though I don’t like the name, I like the DevSecOps approach of using the empathetic mindset of DevOps to factor in security thinking,” said McLarty. “Since the systems being built in our current digital age are more widely distributed with more integration points, they generally have a larger attack surface. So, security considerations are more important than ever, but the way security is being addressed must fit the decentralized/streamlined approach or it will become a bottleneck.”
Issues with DevOps still need to be sorted out, said Razi Sharir, vice president of products and marketing for Robin Systems. “DevOps is still very much early on, and many organizations still struggle when it comes to deciding what the best way is to embrace it—on the dev/test side, on the IT/ops side, or in between. It is still too early, and the jury is still out on how to best implement it.”
Building DevOps
Tellingly, DevOps “has been accepted widely in name by organizations of all sizes,” McLarty said. “In the large enterprise space, DevOps is often misconstrued as being a category of tools or an organizational approach, rather than as a shift in culture and mindset. This could be due to a lack of understanding or because the existing culture is so deeply institutionalized.”
DevOps needs to be more than the IT department—it needs to encompass the entire organization. That means cultural and organizational changes are essential to putting these new methodologies in place. Today, “core pieces of DevOps—particularly the application of automation to delivery of IT system management functions—have been widely deployed,” said IBM’s Elder. “In many cases, the adoption has been compartmentalized into DevOps teams, which to some extent misses the key cultural transformation; to really adopt DevOps as a philosophy often means a pretty dramatic refactoring of the human organization which delivers software. DevOps is about empowering segments of the human organization to act with greater flexibility and at their own pace; rather than bottleneck around the slowest components of the architecture—which is typical of monolithic updates.”
DevOps is not a department, “it is a conceptual way of doing things,” Sharir said. DevOps, usually associated with CI, or continuous integration, and CD, or continuous deployment, has gotten very popular in recent years as cloud adoption has grown and frequent updates and releases have been required to deal with never-ending changes, he noted.
“When companies seek to emulate the success of Amazon or Netflix, they shouldn’t assume that consuming the tech stack is enough on its own,” McLarty said. “For organizations to be successful with DevOps specifically and more generally in this digital age, they need to be honest with themselves about who they are, what they want to achieve, and then align their principles and practices accordingly, while tackling small changes incrementally in order to build positive momentum.”
Scott echoed this advice, urging IT managers to “start small, get comfortable with the new technologies, and adapt to the new approaches offered.” This requires a culture that is open to change, he added. “If there are too many people afraid or unwilling to adopt and learn new technologies, then there is certainly failure waiting ahead. The move to DevOps brings new benefits never before realized in systems management.”
Likewise, moving to container and microservices requires an ability to scale and grow with the organization. “Developers working with containers on a laptop is one thing, but moving them into production, at scale, is completely different,” said Jeff Chou, CEO of Diamanti. He added that “DIY container infrastructure isn’t free—it’s expensive and time-consuming to both build and support. Focus on building and operating your application—not the infrastructure,” he advised, “where seemingly simple issues can consume days or weeks of troubleshooting.”
There needs to be a greater emphasis on containers and microservices for the operations side of the equation. “Container technology does help promote DevOps, but today it is to some degree only one sided—the dev side,” Sharir cautioned. “DevOps often refers to development, testing, and staging phases, and rarely to the ops side beyond day-one deployment.”
Putting container technology and DevOps to work successfully “will require a broader view of DevOps, covering well beyond CI/CD and into ongoing app lifecycle management,” Sharir continued.
Ultimately, DevOps is less about the specific tools that are used and more about a commitment to learning the requisite approaches and patterns “and then providing the leadership that supports the team through the ensuing learning curves,” Quillin said.