You are moving to the cloud. That means your data is moving to the cloud, too. But, are you prepared for the challenges that you will face when moving your databases to the cloud?
The time is now to prepare for those challenges. You cannot afford to delay your cloud data management strategy until after you have pushed your application to the cloud.
Today, companies are managing their databases the same way they have been for the past 25 years: manual SQL script execution. It’s time to change that. If you are moving your databases to the cloud and are managing them the same way you have with your on-premise databases, you are going to get your lunch eaten.
But there is a way to fix this: decentralizing data management. Instead of having a single, centralized group in your company that pushes database changes, you need to decentralize this task and allow your application teams to make those changes, just as they do with compiled bits of the application.
Let’s take a look at how decentralized data management fits into the two schools of thought when it comes to cloud migrations: Lift & Shift and Green Field-Only.
School of Thought No. 1: Lift & Shift
The first cloud migration school of thought is “Lift & Shift.” This essentially means taking your existing application and databases, as is, and moving them to the cloud.
There are a number of resources that can help you with this, namely Amazon Web Services’ Database Migration Service (DMS). DMS can inspect your database schema (structure and stored logic) and verify that it is ready to use AWS infrastructure. It can then replicate data from your old on-premise database to your new cloud infrastructure. Then, when you are ready, you can direct users to the new AWS site.
However, you cannot continue business as usual when your data is in the cloud. Remember, you are moving to the cloud for a reason, and cost is not it. Sure, the CFO might be backing your move to the cloud to save money, but that is low-hanging fruit. The real reason to move to the cloud is flexibility and speed.
When we manage our databases on physical hardware, we have a physical limit to the number of databases we can have. For example, with only a single database server for multiple development teams, the rate of change for the single database server was low due to the need to avoid team conflict. This physical limit prevents us from increasing the number of application releases and required database schema and logic changes. After moving to the cloud, this physical limitation goes away because your development teams are able to have their own cloud database instances; thus you will see a huge increase in the rate of application change. And that’s the point. You now have the flexibility needed to change quickly and deliver great application experiences more frequently.
But don’t get too excited—you’re not done. There is still another big bottleneck to remove.
When databases are in the cloud, development teams have the flexibility to create and destroy new database instances. This enables fast development. The good news is that is exactly what your customers are demanding. But, if you are still relying on a person to review each and every change on those database instances, you’re doing it wrong.
We’ve already seen this change with infrastructure as code using tools such as Puppet, Chef, and now Kubernetes. System administrators realized that they needed to offer self-service to their customers (development and test). DBAs need to do the same with the database. This is decentralized data management. Simply put, it means enabling your product teams to handle some of the tasks typically managed by your data teams. You absolutely must remove the data bottleneck for your “Lift & Shift” cloud migrations. Decentralized data management allows organizations to eliminate the wait-state created by the need to have a human review each and every database change. Thus, development teams can get the most out of the speed and flexibility provided by their cloud application platform.
School of Thought No. 2: Green Field-Only
The second school of thought for cloud migration is “Green Field-Only.” You might decide that only new applications are going to move to the cloud (remember that cloud native re-platformed applications count as green field). However, every single issue you have around data governance on physical servers will be an issue for your green field applications and their databases in the cloud.
We often hear the phrase, “You build it; you run it,” with cloud-native applications. Of course, that leaves a lot of judgment calls up to the product team that is building and running the application. Sometimes, those judgment calls directly affect data. For example, you should always limit the number of indexes on each table. At some point, too many indexes will slow performance. To shift left and correct this issue early in the application development cycle, there needs to be decentralized data management so product teams can update their database without waiting for the data team. This ability to shift left does not negate the need to enforce enterprise standards. In fact, the need to enforce standards becomes even more important in a decentralized world.
Furthermore, some cloud databases are different than their on-premise versions. For example, Azure SQL (a SQL Server variant) requires all indexes to be clustered. This is not necessary for on-premise SQL Server, so making sure product teams adhere to this best practice is a challenge—especially if they are developing locally against SQL Server and pushing their changes to Azure SQL. Thus, having a mechanism to allow for decentralized data management, while maintaining standards, allows for product team empowerment with enforcement. You need to have both.
Decentralized Data Management Is the Key
The key to success with cloud databases is decentralized data management for both “Lift & Shift” and “Green Field-Only” strategies. The ability to provide your product teams (previously called development teams) with the ability to manage their databases will be absolutely necessary. But, speed and product team empowerment must be coupled with standards enforcement to avoid unforced errors.
Your entire purpose in moving to the cloud is to provide cost savings, application development flexibility, and to remove impediments to application release. When moving to the cloud, you must remove your biggest impediment, data management, to reach these goals. Whether you are using “Lift & Shift” or “Green Field-Only” for your cloud migration strategy, both will fail if you do not adopt decentralized data management. You can no longer rely on the same process for managing your databases as you did 25 years ago. If you don’t change, the cloud will eat your lunch … or, your competitors will.