Zions Bancorporation is one of the nation's premier financial services companies with total assets exceeding $65 billion. Zions operates under local management teams and distinct brands in Arizona, California, Colorado, Idaho, Nevada, New Mexico, Oregon, Texas, Utah, Washington, and Wyoming. The company is a national leader in Small Business Administration lending and public finance advisory services, and is a consistent top recipient of Greenwich Excellence awards in banking. In addition, Zions is included in the S&P 500 and NASDAQ Financial 100 indices.
Zions Bancorporation is a 2020 DBTA Database DevOps Innovation Award Winner
Goals
Zions wanted to ensure that there is continuous software delivery in a fully automated fashion that includes all aspects and layers of the product stack. A key goal was to enable delivery teams to be self-sufficient, without handoffs or silos in other organizations or disciplines. Championing the effort was Russell Webster, vice president and senior manager, delivery tools and services, Zions Bancorporation.
To achieve its goals, Zions selected Datical to help automate and align the delivery of database changes as a standard part of application delivery. Zions wanted to ensure that developers could use the same development practices, SDLC standards, and pipeline tools and controls with database changes as they do with application code. It also wanted to allow DBA oversight at the right points in time within the delivery chain, without having to be attached to and involved in the delivery team. In addition, Zions wanted to allow automated code control checks and rules on SQL code (like static analysis) to ensure best practices and safety, and to reduce cycle time and increase education. Finally, Zions wanted to do all this with as little overhead and DBA involvement as possible, but with their confidence and trust that the SQL delivery chain enforces proper controls and rules
The Outcome
Now, for each application using Datical for the database delivery chain, the delivery team is able to deliver all changes with no delays and handoffs. This has allowed its DBAs to remove themselves from the regular daily events of delivery and focus on DBA-centric activities rather than wasting their time and effort on handling deployments.
These capabilities have given the DBAs freedom from being on-call for every release and a greater work-life balance with reduced hours and responsibilities and the ability to properly focus on their discipline. This empowers the delivery team to go end-to-end without handoffs, input from other teams, or permissions.
These changes have added security, traceability, and proper controls to database delivery, including proper source control, versioning, and chain-of-custody. In addition, manual errors have been reduced or removed, and developers are able to find out more rapidly if-and-when they introduced code that is not aligned with the delivery format.
Metrics
As one delivery-stream example, the team no longer has wait times of hours or days for DBA availability for each change that has been made. Team members simply commit code and deliver to the environments as they iterate. This value plays out multiple times per day during the development cycle.
At the point of release or promotion, the value is seen again. Large coordinated efforts are reduced by hours of wait time. The applied time is reduced as well. Manual deployments of scripts and the accompanying errors and problems are gone, and scripts run in seconds, rather than over the course of an hour or more.
In addition, the team’s available time to test increases by the same amount because the product and environment is ready faster.
Moreover, less coordination and fewer people are required for each delivery to production because the entire delivery chain, including Datical, is coordinated behind one button integrated into Zions' deploy and release tools used to run it all. The software delivery is reliable, seamless, quick, and repeatable, and overall E2E (end-to-end) time has been reduced by hours along with reduced risk.
Lessons Learned
According to Zions, with this implementation, there are four key takeaways.
First, database development and deployment is not special, and should not be treated as such. Second, teams can deliver “everything” together and be enabled to do so. Third, enterprises do not necessarily have to pay for micro-management of the database delivery chain by others not involved in the delivery team, but can still have proper controls and oversite in place. And finally, DBAs may not know it yet, but they will love having their time back to innovate in their own space.
To automate database delivery, some design work and changes are likely required to how you consider process, as well as how you deal with DDL (Database Definition Language versus DML (Data Manipulation Language). Decisions will have to be made about what is important and when it is important, as well as other designs that involve database state, on a per product and per database basis, depending on needs.
What's Next
Zions plans to roll out the same capabilities across every applicable application in the enterprise. The company will target multiple database types (Oracle, SQL Server, DB2, and others), and have a pattern for each. Zions will be investigating all vendor-derived applications with regular database changes that should come through Datical as well. Based on compatibility, priority, change cadence, and efficacy, the company plans to determine if this should be automated. Zion's plan is to remove any manual overhead and inconsistencies wherever possible in the chain-of-custody and delivery events.