Written by Jagdish Mirani, Sr. Databases Product Marketing Manager, AWS
Gowri Balasubramanian, Sr. Manager Solutions Architecture, AWS
Serverless databases are here, and there is no going back. We first saw the emergence of serverless technology in the application layer, with fixed capacity for relational databases. Now, with serverless databases, both the application and database layers can scale automatically with no performance bottleneck. Data and modern databases are critical to an organization’s data strategy and digital transformation initiatives. With serverless databases, IT teams have the ability to drive more of these initiatives.
Most workloads have some degree of “spikiness,” and serverless databases automatically adjust to these intermittent workloads. Serverless databases are more cost effective than provisioning for peak workloads. They also eliminate the negative experience of under-provisioned and under-performing databases. In addition, manual scaling requires developing and maintaining related skills, and is more susceptible to human error.
Over the years, we have observed customers adopting serverless architectures for a range of use cases such as event processing, web applications, multi-tenant SaaS applications, new applications with unknown capacity requirements, and lots more. These applications leverage purpose-built serverless database options such as Amazon Aurora, Amazon DynamoDB, Amazon Keyspaces, Amazon Quantum Ledger Database (QLDB), and Amazon Timestream for persistence. Our focus here is on Amazon Aurora.
With the launch of Amazon Aurora Serverless v1, AWS provided a highly-available, highly-scalable and cost-effective relational database option for workloads that involved spiky and unpredictable traffic patterns. This helped customers achieve the true potential of serverless capabilities from the application to database layer, without worrying about managing the server capacity. AWS further innovated in this area and released Amazon Aurora Serverless v2, which scales instantly to hundreds of thousands of transactions in a fraction of a second.
With Aurora Serverless v2, customers can benefit from faster and granular scaling and lower costs. Aurora Serverless v2 supports the full breadth of features including high-availability, scale and scalability, Global Database, multiple Availability Zone (Multi-AZ) deployments, and read replicas. This has helped customers achieve the true potential of serverless capabilities from the application to the database layer without worrying about managing server capacity.
Separation of Storage from Compute—A Key Serverless Enabler
With Amazon Aurora, AWS re-imagined how relational databases are designed for the cloud. The Aurora architecture decouples the storage layer from compute layer, with data replicated across three availability zones in a single AWS region. The Aurora database cluster has one primary instance that supports read/write operations with the option of adding multiple read replicas sharing the same distributed storage subsystem. Aurora cluster deployments, with a particular instance type (e.g., db.r6g.xlarge), are also known as provisioned clusters, which is ideal for steady workloads. Aurora’s cluster storage volumes automatically grow with the data, up to 128 TiB per cluster.
Aurora Serverless v2 leverages the same underlying design, with the added benefit of automated provisioning and management. The application simply and seamlessly connects to the Aurora Serverless cluster endpoint for read/write traffic and a reader endpoint for read only workloads.
Defining Characteristics of Serverless
In the remainder of this article, we will examine the fundamental characteristics of serverless architectures and describe how Aurora Serverless v2 fulfills these needs:
• Automated Capacity Management
• Designed for Resilience
• Pay Per Use
• Simpler Experience
• Supports Event-Based Architectures
We will also highlight various use cases that can benefit from the above architecture.
Automated Capacity Management
Pre-serverless databases, scaling up a primary node in a provisioned cluster involved a brief outage and a multi-step process. Often, customers either scale the primary instance or scale one of the reader instances with higher capacity and then fail-over to the scaled instance. This becomes a major challenge for multi-tenant workloads such as SaaS because the capacity requirement can vary based on tenant. Each tenant may have specific busy and idle periods depending on the time of day, time of year, promotional events, and so on. With AWS serverless databases, you can specify the minimum and maximum capacity and have the database dynamically adjust within the specified range.
You can scale reads by adding reader nodes in serverless configuration. Each reader remains within the minimum and maximum values you specify for the cluster. Depending on failover priority, readers can scale vertically at the same time as the associated writer, or independently from the writer. For batch applications or business reporting that experience periodic spikes with a lot of idle periods, this configuration provides better cost-efficiency without impacting performance.
Aurora Serverless v2 continuously tracks the utilization of resources such as CPU, memory, and network. When capacity is constrained by any of these, the database scales incrementally and in a non-disruptive way even when there are thousands of active connections/transactions occurring.
Designed for Resilience
Aurora Serverless v2 supports Multi-AZ database cluster deployments with a reader deployed in a different AZ. Aurora automatically fails over to a reader in the event of a writer instance database failure, or an entire AZ failure. To ensure that your application traffic load continues without any performance issues even after failover, Aurora Serverless v2 readers in promotion tiers 0 and 1 scale at the same time as the writer. As a best practice, it is recommended to designate one of your readers as a tier 0 or tier 1. Readers in promotion tiers 2–15 scale independently from the writer.
Aurora Serverless v2 also supports the Aurora Global Database feature, which allows launching a reader instance in a different AWS region from the primary. You can use this setup to protect your business-critical applications from region-wide outages or to serve low-latency reads closer to users in that region. Global Database supports Recovery Point Objective (RPO) of 1 second and a Recovery Time Objective (RTO) of less than 1 minute. It is worth highlighting, Global Database uses a dedicated infrastructure in Aurora’s purpose-built storage layer to handle replication across regions without compromising the database performance.
Pay Per Use
Aurora Serverless v2 provides on-demand capacity where the billing unit is Aurora Capacity Unit (ACU) hours instead of database instance hours. Each ACU is a combination of processing and memory capacity. Aurora Serverless v2 capacity increases and decreases, within a range that you specify, depending on the load on your database. For information about how Aurora Serverless v2 ACUs work, see How Aurora Serverless v2 works. With granular billing based on usage, customers can achieve savings of up to 90% on database cost when compared to provisioning for peak.
Additionally, customers with large development and test fleets can achieve improved cost-efficiency, since they can provision a low minimum ACU and can still support the variance in their testing such as performance testing, by setting the maximum capacity high enough to keep their working dataset in memory. When Aurora Serverless v2 scales down, the resource usage is measured on a per-second basis, making it ideal for cost savings.
The cost advantages of automatic, dynamic resource scaling in Amazon Aurora Serverless v2 as well as other benefits are well articulated in IDC’s spotlight on serverless, cloud-native databases.
Simpler Experience
Managing capacity for a database fleet with thousands of databases on a budget is a daunting task for any organization. Aurora Serverless v2 frees you from the administrative burden of managing capacity, starting/stopping, and choosing the right instance class. Security is paramount to our customers, and at AWS, we treat security with the highest standards to ensure the trust of our customers. Aurora Serverless v2 allows you to encrypt your databases using encryption keys you manage through AWS Key Management Service (KMS). It also supports secure connections via Transport Layer Security/Secure Sockets Layer (TLS/SSL) protocol. Customers can leverage AWS Command Line Interface (AWS CLI) and/or Amazon Relational Database Service (Amazon RDS) for provisioning and managing Aurora Serverless v2 clusters at scale. Like other AWS services, Aurora Serverless v2 has integration with CloudWatch for monitoring and logging. Specifically, customers can monitor the capacity consumed by each serverless instance with the ServerlessDatabaseCapacity metric to better understand the cost of their fleet.
Supports Event-Based Architectures
You can build serverless event-based architectures using AWS Lambda, Aurora Serverless v2, and Amazon RDS Proxy to improve extensibility, scalability, and to reduce the complexity of your business applications. Consider a scenario in which an ecommerce application uses Aurora Serverless v2 for storing transactional data. The business logic can run on AWS Lambda which can be triggered by an event or notification from other AWS Services such as Amazon S3 or Amazon API Gateway. Your application can automatically scale in response to demand including the database layer. Your Lambda functions can process the event and store the results in Aurora Serverless v2 by connecting via fully managed RDS Proxy. The RDS Proxy allows Lambda based applications to pool and share connections, thus improving database efficiency and application scalability.
Migration
AWS provides several tools, best practices, and professional services in support of migrations. AWS Database Migration Service (DMS) is a simple to use self-service migration tool that helps you migrate your databases from commercial database engines such as Oracle, SQL Server to Aurora Serverless v2 with minimal disruption to your business.
The AWS Schema Conversion Tool automatically converts the source database schema and a majority of the database code objects, including views, stored procedures, and functions, to a format compatible with the target database (Amazon Aurora in this case).
If your workload is already running in Aurora provisioned clusters, you can simply modify the existing database instances from provisioned to Aurora Serverless v2 or you can run Aurora Serverless v2 alongside with provisioned (mixed-mode configuration). You can also leverage the PostgreSQL or MySQL native logical replication capabilities to migrate your data into Aurora Serverless v2. At the time of this writing, Aurora Serverless v2 is available for the MySQL 8.0- and PostgreSQL 13-compatible editions of Amazon Aurora.
You can also work with AWS Professional Services or a database migration partner to facilitate the migration. AWS IQ connects you with AWS Certified freelancers. You can browse AWS Database Migration Service experts and offers, or post a custom request and get qualified bids.
Say Goodbye to Capacity Planning and Allocation
This fungibility of serverless databases redraws the boundaries of data gravity, and empowers us with fast and secure access to data wherever and whenever it is needed. With Aurora Serverless v2, data is readily available at any capacity, fulfilling the promise of a “liquid” infrastructure in the cloud where resources can be quickly spun up and down.
Serverless computing has a multiplicative effect. The deeper your adoption up and down the technology stack, the less likely you are to experience performance bottlenecks or over-capacity. Serverless microservices, functions, and databases are all among the building blocks for reaping the benefits of serverless.
To learn more about Amazon Aurora Serverless, head over to the service overview or refer to the AWS documentation. Gartner’s solution scorecard for Amazon Relational Database Service (RDS), which includes Amazon Aurora, gave RDS an overall rating of 95, the highest ever for a cloud-native database. You can go deeper by taking AWS training related to developing serverless solutions.