Serverless Database — Data Persistence for Serverless Applications
In this tutorial, you will learn about Serverless Database. We cover key concepts, practical examples, and best practices to help you master this topic.
Serverless databases automatically scale capacity based on demand and charge only for resources consumed, with DynamoDB and Aurora Serverless being the most common options for serverless applications.
What You'll Learn
By the end of this lesson you will understand serverless database options, when to use DynamoDB vs Aurora Serverless, single-table design patterns, connection pooling with RDS Proxy, and caching strategies.
Why It Matters
Choosing the wrong database for a serverless application leads to high costs, poor performance, or scaling limitations. Serverless databases must handle extreme scaling patterns from zero to thousands of concurrent connections.
Real-World Use
DodaTech's user management system uses DynamoDB for session data and user profiles, and Aurora Serverless for relational reporting data. DynamoDB handles the high-throughput key-value access while Aurora handles complex queries.
flowchart TD
L[AWS Lambda] --> D[DynamoDB - Key-Value]
L --> A[Aurora Serverless - Relational]
L --> E[ElastiCache - Cache]
A --> RDS[RDS Proxy]
RDS --> DB[(Aurora DB)]
style D fill:#f90,color:#fff
DynamoDB vs Aurora Serverless
DynamoDB is a NoSQL key-value and document database with single-digit millisecond latency. Aurora Serverless is a relational database compatible with MySQL and PostgreSQL that scales automatically.
# database_comparison.py
# Comparing serverless database options
def compare_databases():
databases = {
"DynamoDB": {
"type": "NoSQL (Key-Value/Document)",
"scaling": "Instant, automatic",
"latency": "Single-digit ms",
"pricing": "Pay per request (RCU/WCU or on-demand)",
"max_item": "400KB",
"use_case": "Session store, user profiles, metadata"
},
"Aurora Serverless": {
"type": "Relational (MySQL/PostgreSQL)",
"scaling": "Auto-scales ACUs (seconds)",
"latency": "Low ms (warm), higher on cold start",
"pricing": "Pay per ACU (Aurora Capacity Unit)",
"max_item": "64TB total",
"use_case": "Complex queries, joins, transactions"
}
}
for name, config in databases.items():
print(f"\n{name}:")
for key, value in config.items():
print(f" {key}: {value}")
compare_databases()
RDS Proxy for Connection Pooling
Lambda functions can exhaust database connections during scaling. RDS Proxy manages connection pooling.
# rds_proxy.py
# RDS Proxy connection pooling
import json
class RDSProxyConnection:
def __init__(self, proxy_endpoint):
self.proxy = proxy_endpoint
self.connections = 0
print(f"RDS Proxy configured at {proxy_endpoint}")
def get_connection(self):
if self.connections < 10:
self.connections += 1
print(f" -> Borrowed connection from pool ({self.connections}/10 active)")
return {"connection_id": self.connections}
else:
print(" -> Waiting for connection to become available")
return None
def release_connection(self, conn):
self.connections -= 1
print(f" -> Returned connection to pool ({self.connections}/10 active)")
def lambda_handler(event, context):
proxy = RDSProxyConnection("my-proxy.proxy-abc.us-east-1.rds.amazonaws.com")
conn1 = proxy.get_connection()
print(f"Querying users...")
proxy.release_connection(conn1)
conn2 = proxy.get_connection()
print(f"Querying orders...")
proxy.release_connection(conn2)
return {"statusCode": 200, "body": json.dumps({"queries_completed": 2})}
print(lambda_handler({}, None)["body"])
Caching with ElastiCache
Use ElastiCache (Redis or Memcached) to reduce database load and improve response times.
# caching.py
# Caching patterns for serverless
class Cache:
def __init__(self):
self.store = {}
self.hits = 0
self.misses = 0
def get(self, key):
value = self.store.get(key)
if value:
self.hits += 1
print(f"Cache HIT: {key}")
return value
self.misses += 1
return None
def set(self, key, value, ttl=300):
self.store[key] = value
print(f"Cached: {key} (TTL: {ttl}s)")
def get_user_data(user_id, cache):
cached = cache.get(f"user:{user_id}")
if cached:
return cached
print(f"Querying database for user {user_id}")
data = {"id": user_id, "name": "Alice", "email": "alice@example.com"}
cache.set(f"user:{user_id}", data)
return data
cache = Cache()
for _ in range(3):
result = get_user_data("1", cache)
print(f" Result: {result['name']}\n")
print(f"Cache stats: {cache.hits} hits, {cache.misses} misses")
Common Mistakes
Using DynamoDB for complex relational queries: DynamoDB is not designed for joins or complex filtering. Use Aurora Serverless for relational needs.
Not using RDS Proxy with Lambda: Lambda scale creates hundreds of database connections. RDS Proxy prevents connection exhaustion.
Ignoring DynamoDB hot partitions: Uneven access patterns create hot partitions. Use write Sharding with random suffixes.
Setting too high or low provisioned capacity: Over-provisioning wastes money. Under-provisioning causes throttling. Use on-demand for variable workloads.
Not caching frequently accessed data: Repeatedly querying the database for the same data wastes capacity. Use ElastiCache or DynamoDB DAX.
Practice Questions
What is the difference between DynamoDB and Aurora Serverless? DynamoDB is NoSQL key-value with instant scaling. Aurora Serverless is relational with auto-scaling ACUs.
Why do Lambda functions need RDS Proxy? Lambda scales rapidly, creating many database connections. RDS Proxy pools connections to prevent database overload.
What is DynamoDB on-demand capacity? Pay-per-request pricing that scales instantly without capacity planning, suitable for variable workloads.
How do you handle caching in serverless applications? Use ElastiCache Redis or DynamoDB DAX for frequently accessed data with TTL-based invalidation.
Challenge: Design a data layer for a serverless application that uses DynamoDB for user profiles, Aurora Serverless for orders, and ElastiCache for session caching.
FAQ
Mini Project
Create a serverless application data layer with DynamoDB for user profiles, ElastiCache for session caching, and pattern for data access from Lambda functions.
import json
import time
class DataLayer:
def __init__(self):
self.cache = {}
self.db = {}
def get_user(self, user_id):
cached = self.cache.get(f"user:{user_id}")
if cached and cached["expires"] > time.time():
print(f"Cache HIT for user {user_id}")
return cached["data"]
print(f"Fetching user {user_id} from DynamoDB")
user = self.db.get(user_id, {"id": user_id, "name": "Unknown"})
self.cache[f"user:{user_id}"] = {"data": user, "expires": time.time() + 300}
return user
def create_user(self, user_id, name, email):
print(f"Creating user {user_id} in DynamoDB")
self.db[user_id] = {"id": user_id, "name": name, "email": email, "created": time.time()}
self.cache[f"user:{user_id}"] = {"data": self.db[user_id], "expires": time.time() + 300}
return self.db[user_id]
data = DataLayer()
data.create_user("1", "Alice", "alice@example.com")
print(json.dumps(data.get_user("1")))
print(json.dumps(data.get_user("1"))) # From cache
What's Next
Next: Serverless Project for a complete capstone project.
Built by the developers of DodaTech
Doda Browser, DodaZIP & Durga Antivirus Pro