RDS — managed Postgres / MySQL / MariaDB / Oracle / SQL Server. Multi-AZ for HA, read replicas for scale.
Aurora — RDS-compatible, storage-decoupled. 6 copies across 3 AZs. Failover is faster than classic RDS (typically under a minute) because storage is shared and survives the instance: a promoted replica attaches to the existing 6-way-replicated volume instead of restoring from a snapshot.
DynamoDB — managed k-v / document. Single-digit ms p99. PK + optional SK. GSI / LSI for secondary access. On-demand vs provisioned RCU/WCU.
ElastiCache — managed Redis / Memcached.
Redshift — columnar warehouse, OLAP.
Athena — serverless SQL on S3 (Presto).
Kinesis — managed streaming. Data Streams (Kafka-like), Firehose (load to S3/Redshift), Managed Service for Apache Flink (formerly Kinesis Data Analytics).
Networking
VPC — private virtual network. Subnets per AZ, public/private split.
SQS — managed queue. Standard (best-effort order, at-least-once) or FIFO (ordered per message group; exactly-once processing via a 5-min dedup window — not end-to-end exactly-once delivery, so consumers must stay idempotent). FIFO throughput is bounded because order is preserved per message group, so a single group is processed serially; scale by spreading messages across many message-group IDs. DLQ for poison messages.
IAM — users, roles, policies. Trust + permission policies.
STS — short-lived credentials via AssumeRole.
KMS — managed keys, envelope encryption. CMK + data keys.
Secrets Manager — secret storage with rotation; Parameter Store for non-secret config.
Cognito — user pools (identity) + identity pools (federated).
WAF / Shield — L7 firewall + DDoS.
Least-privilege policy granting read-only access to one S3 prefix. Note the two ARN granularities: s3:GetObject acts on objects (bucket/prefix/*), while s3:ListBucket acts on the bucket ARN and is scoped to the prefix via an s3:prefix condition — getting that level wrong is the classic least-privilege bug.
# multi-stage example
FROM golang:1.26 AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 go build -o /out/app
FROM gcr.io/distroless/static
COPY --from=build /out/app /app
USER 65532:65532
ENTRYPOINT ["/app"]
Pod disruption budgets (PDB) protect availability during voluntary disruption.
A complete Deployment + Service + HPA with requests/limits and the three probes. The CPU-utilization HPA computes load as a percentage of the pod's CPU request, so the Deployment must set resources.requests.cpu or the HPA has no denominator and cannot scale.
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
spec:
replicas: 3
selector:
matchLabels: { app: web }
template:
metadata:
labels: { app: web }
spec:
containers:
- name: web
image: registry.example.com/web@sha256:abc123 # pin digest, not tag
ports:
- containerPort: 8080
resources:
requests: { cpu: "250m", memory: "256Mi" } # scheduler reservation + HPA denominator
limits: { memory: "512Mi" } # cap memory; omit CPU limit to avoid throttling
startupProbe: # gates the others until the app has booted
httpGet: { path: /healthz, port: 8080 }
failureThreshold: 30
periodSeconds: 5
readinessProbe: # fail -> removed from Service endpoints (no restart)
httpGet: { path: /ready, port: 8080 }
periodSeconds: 5
livenessProbe: # fail -> kubelet restarts the container
httpGet: { path: /healthz, port: 8080 }
periodSeconds: 10
---
apiVersion: v1
kind: Service
metadata:
name: web
spec:
selector: { app: web }
ports:
- port: 80
targetPort: 8080
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: web
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: web
minReplicas: 3
maxReplicas: 10
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 70 # % of the 250m request
Networking & Probes
Probes — three jobs, different failure semantics
Startup — runs first and gates the other two; while it is failing, liveness and readiness are suspended. Use it for slow-booting apps so a long warm-up isn't mistaken for a crash.
Readiness — failure removes the pod from its Service's endpoints, so it stops receiving traffic but is not restarted. This is the lever for graceful drain and "not ready yet."
Liveness — failure makes the kubelet restart the container. Reserve it for unrecoverable deadlock; a too-aggressive liveness probe turns a slow dependency into a restart loop.
Pod networking — flat L3
The K8s network model requires every pod to get its own routable IP and to reach every other pod without NAT — a flat L3 fabric. This is why pods can talk cross-node as if on one network.
The CNI plugin (Calico, Cilium, AWS VPC CNI, …) implements that contract: it wires the pod's veth into the node and programs routes. Swapping CNI changes how IPs are allocated and how NetworkPolicy is enforced, not the flat-network guarantee.
Service data path
A Service is a stable virtual IP (ClusterIP). The endpoints controller watches pods matching the selector that are Ready and writes their IPs into EndpointSlices (the scalable replacement for the single Endpoints object).
kube-proxy on each node watches EndpointSlices and programs the dataplane — iptables mode installs DNAT rules that randomly pick a backend; IPVS mode uses an in-kernel hash table that scales better with thousands of services. Either way, ClusterIP traffic is rewritten to a real pod IP in-kernel; there is no proxy process in the hot path.
Because only Ready pods appear in EndpointSlices, a failing readiness probe is exactly what pulls a pod out of this data path.
Service discovery — CoreDNS
CoreDNS runs in-cluster and resolves <service>.<namespace>.svc.cluster.local to the Service's ClusterIP. Pods get it as their resolver via the kubelet, so apps address dependencies by name and let the Service/EndpointSlice layer handle the live pod set.
Blue/Green: two deployments, switch service selector.
Canary: weighted routing via Service Mesh (Istio/Linkerd) or Argo Rollouts.
MODULE 6
Infrastructure as Code
Terraform / Pulumi / CloudFormation / CDK.
Terraform Concepts
Providers (aws, gcp, k8s) → resources → state file.
State: source of truth for what TF created. Remote backend (S3 + DynamoDB lock, Terraform Cloud).
Plan / Apply: dry-run shows diff. Apply executes.
Modules: reusable groups of resources. Inputs / outputs / variables.
Workspaces or directories for env separation (dev / staging / prod).
Drift = real infra ≠ state. Detect via plan; reconcile or import.
Remote state on S3 with a DynamoDB lock table. The DynamoDB table provides a distributed lock so two concurrent applys can't interleave writes and corrupt state; its key attribute must be named LockID (String) or Terraform won't find the lock item. Note backend blocks accept only literal values — no variables or interpolation — so the bucket/table names are hardcoded.
terraform {
backend "s3" {
bucket = "my-org-tfstate"
key = "prod/network/terraform.tfstate"
region = "us-east-1"
dynamodb_table = "tf-locks"
encrypt = true
}
}
# The lock table (created once, e.g. in a bootstrap config):
resource "aws_dynamodb_table" "tf_locks" {
name = "tf-locks"
billing_mode = "PAY_PER_REQUEST"
hash_key = "LockID"
attribute {
name = "LockID"
type = "S"
}
}
Tooling Comparison
Tool
Lang
Pros
Cons
Terraform
HCL
Multi-cloud, large ecosystem, mature
HCL limited; state mgmt headaches
Pulumi
Python/TS/Go
Real lang, tests, conditionals
Smaller community
CloudFormation
YAML/JSON
Native AWS, no state to manage
AWS-only, slow rollback
AWS CDK
TS/Python/Java/Go
Synth to CFN; constructs
AWS-only; opinionated abstractions
Crossplane
K8s CRDs
GitOps-native; multi-cloud as K8s
K8s-shaped solutions only
MODULE 7
Cloud Networking
VPC anatomy + connectivity patterns.
VPC Topology
VPC = isolated address space (e.g., 10.0.0.0/16).
Subnets per AZ (typically 2–3 AZs). Public subnet = route to IGW; private = no IGW route.
NAT Gateway in public subnet → outbound internet from private subnet.
Route tables associate with subnets. Security groups (stateful, instance-level) + NACLs (stateless, subnet-level).
VPC endpoints: Interface (ENI for AWS service) or Gateway (S3 / DynamoDB free).
VPC reachability: public vs private subnet routing
Match access pattern to tier; lifecycle policies move automatically.
S3 Classes
Class
Cost / GB-mo
Retrieval
Use
Standard
$0.023
Instant
Hot data
Intelligent-Tiering
$0.023 → $0.0036
Instant
Unknown access pattern
Standard-IA
$0.0125
Instant + retrieval $
Backup, infrequent access
One-Zone-IA
$0.01
Instant
Re-creatable, single AZ
Glacier Instant Retrieval
$0.004
Instant
Archive accessed rarely
Glacier Flexible
$0.0036
Minutes–hours
Backup
Glacier Deep Archive
$0.00099
12 hours
Compliance, long retain
EBS Volume Types
gp3 — console-default GP SSD (gp2 still the API/launch default). 3000 IOPS / 125 MiB/s baseline free; IOPS & throughput provisioned independently of size (vs gp2's 3 IOPS/GiB); scale to 80,000 IOPS / 2,000 MiB/s.
io2 Block Express — up to 256k IOPS, <1 ms; expensive.
st1 / sc1 — HDD, sequential workloads, cheap.
Snapshots = incremental, stored in S3 internally.
MODULE 9
Cost Optimization
FinOps basics. Engineering decisions = cost decisions at scale.
Pricing Levers
On-Demand — full price, no commit.
Reserved Instances / Savings Plans — 1 or 3 yr commit; up to ~72% off. Compute SP = flexible across services.
Spot — up to 90% off; can be reclaimed in 2 min. Pair with checkpointing or Karpenter.
Right-sizing — match instance to workload. Compute Optimizer surfaces.
Auto-scaling — scale to zero off-hours where possible.
Watch For
Idle EBS volumes / snapshots / Elastic IPs.
NAT Gateway egress for internal traffic — use VPC endpoints.
Cross-AZ chatter — colocate replicas / cache.
S3 unfinished multipart uploads (lifecycle rule to expire).
Log retention default = forever; set retention.
RDS over-provisioned IOPS; switch to gp3.
MODULE 10
Docker Internals — How Containers Work
Namespaces, cgroups, and a layered filesystem — how one Linux kernel pretends to be many isolated machines.
A Container Is Just a Process
The single most important mental correction for the interview: a container is not a lightweight VM. There is no hypervisor, no guest kernel, no emulated hardware. A container is an ordinary Linux process running on the host kernel — the same ps, the same scheduler, the same page cache. What makes it feel isolated is three kernel features layered on top of that process:
Namespaces — change what the process can see (its own PID tree, network stack, mounts, hostname).
cgroups v2 — limit and account what the process can use (CPU, memory, IO, PID count).
Union filesystem — give it a root filesystem assembled from stacked read-only image layers plus one thin writable layer.
Strip those away and a "container" is a process like any other. That framing answers most Docker-internals questions before they're fully asked.
Container vs Virtual Machine
Both provide isolation, but at completely different layers. A VM virtualizes hardware and boots a full guest OS; a container virtualizes the OS interface and shares the host kernel. That single difference cascades into boot time, density, and the strength of the isolation boundary.
Dimension
Container
Virtual Machine
Kernel
Shared host kernel
Own guest kernel per VM
Isolation boundary
Kernel syscall surface (namespaces + cgroups)
Hypervisor / hardware virtualization (VT-x)
Start time
Tens of milliseconds
Seconds to tens of seconds
Density per host
Hundreds to thousands
Tens
Overhead
Near-zero (process-level)
Full OS RAM + CPU per guest
Isolation strength
Weaker — one kernel is shared
Stronger — separate kernels
Image size
MBs to low hundreds of MBs
GBs (full disk image)
You still reach for a real VM or a microVM (Firecracker, Kata) when you need a hard security boundary: multi-tenant workloads running untrusted code, compliance requirements demanding kernel-level separation, or running a different kernel/OS than the host. Containers give you speed and density; VMs give you a stronger wall.
The Linux Primitives
Docker is largely a friendly UX over kernel features that predate it by years. Know the namespaces cold — each one isolates exactly one kind of global resource:
pid — process ID tree. The container's entrypoint sees itself as PID 1; host PIDs are invisible.
net — network stack: its own interfaces, routing table, iptables rules, ports.
mnt — mount points, so the container sees its own root filesystem.
uts — hostname and domain name (UTS = Unix Time-sharing System).
ipc — System V IPC, POSIX message queues, shared memory segments.
user — UID/GID mapping; root (0) inside can map to an unprivileged UID outside. Foundation of rootless containers.
cgroup — hides the host's cgroup hierarchy, showing the container its own root.
cgroups v2 — the accountant
Where namespaces isolate visibility, cgroups v2 enforce consumption through a single unified hierarchy: CPU (cpu.max), memory (memory.max plus OOM kill), block IO (io.max), and process count (pids.max — your fork-bomb defense). They both limit and account, so docker stats reads straight out of cgroup files.
Hardening layers
Capabilities — root is split into ~40 discrete caps (NET_BIND_SERVICE, SYS_ADMIN…). Docker drops most by default; drop the rest with --cap-drop ALL.
AppArmor / SELinux — mandatory access control (MAC) restricting file/resource access beyond standard permissions.
You can build a "container" by hand to prove the point — no Docker daemon required:
# Create a new PID + mount + UTS + net namespace and run a shell as PID 1
sudo unshare --pid --mount --uts --net --fork --mount-proc bash
# Inside: you are PID 1, hostname is isolated, ps sees only this tree
hostname container-demo
ps aux # only the shell and ps — host processes are invisible
echo $$ # prints 1
# That is the whole trick. Docker just automates unshare + cgroups + overlayfs.
Image Anatomy — Content-Addressable Layers
A Docker image is a stack of read-only layers, each identified by the SHA-256 digest of its content (content-addressable). Identical layers are stored and pulled exactly once, shared across every image that references them. Each instruction in a Dockerfile that changes the filesystem (RUN, COPY, ADD) produces one new layer diffed on top of the previous.
At runtime an overlayfs mount unifies them: the image layers become read-only lowerdirs, and a fresh writable upperdir sits on top. A read walks top-down through the stack; a write triggers copy-on-write into the upper layer.
Internal scratch space overlayfs uses to stage atomic writes
merged
The unified view the container sees as /
Layer cache and ordering is the single biggest build-speed lever: order instructions least-changing → most-changing. Copy your dependency manifest and install deps before copying source code, so a code edit doesn't bust the dependency layer. Multi-stage builds let a heavy build stage compile the artifact, then a tiny final stage copies only the binary — the toolchain never ships.
# ---- Stage 1: build (heavy toolchain, discarded) ----
FROM golang:1.26 AS build
WORKDIR /src
COPY go.mod go.sum ./
RUN go mod download # cached unless deps change
COPY . .
RUN CGO_ENABLED=0 go build -o /app ./cmd/server
# ---- Stage 2: runtime (tiny, distroless) ----
FROM gcr.io/distroless/static:nonroot
COPY --from=build /app /app
USER nonroot:nonroot
EXPOSE 8080
ENTRYPOINT ["/app"]
Base image
Size (approx)
Contains
Use when
scratch
0 MB
Nothing — empty
Fully static binary, no libc
distroless
~2–20 MB
Runtime libs, no shell/pkg-mgr
Prod services; minimal attack surface
alpine
~5–8 MB
musl libc + busybox shell
Need a shell; watch musl vs glibc bugs
debian-slim
~30–80 MB
glibc + apt
Broad compatibility, glibc required
Image vs Container
The relationship is the same as a class vs an instance. The image is the immutable, read-only stack of layers — a template. A container is that image plus one thin writable layer on top, created the moment you docker run.
Image — N read-only layers, content-addressed, shared across containers. Never mutated.
Container — image layers + a per-container writable upper layer (copy-on-write). All runtime file changes land here.
Ephemerality — docker rm deletes only the writable layer. The image is untouched; any data not on a volume is gone.
This is why two containers from the same image are cheap: they share every read-only layer on disk and in page cache, and diverge only in their tiny upper layers.
Container Lifecycle
What actually happens between pressing Enter on docker run and your process running:
Resolve & pull — resolve the tag to an image digest; pull any missing layers by digest (existing layers are skipped).
Create writable layer — allocate the thin copy-on-write upperdir on top of the image's read-only layers.
Set up namespaces — create fresh pid, net, mnt, uts, ipc (and optionally user) namespaces for the process.
Attach cgroups — place the process in a cgroup and apply CPU/memory/IO/pids limits.
Mount rootfs — overlay-mount the merged filesystem and pivot_root into it; wire up networking (veth into the bridge).
Exec entrypoint — the entrypoint process starts as PID 1 inside the pid namespace.
PID 1 is special. In a normal init system PID 1 reaps zombie children and forwards signals. Most app processes don't do this, so an app running as PID 1 may leak zombie processes and, worse, ignore SIGTERM — making docker stop hang until the 10-second timeout escalates to SIGKILL. Fix it with a minimal init: docker run --init (injects tini) or bake tini/dumb-init as your entrypoint.
Networking
Each container gets its own net namespace — a private network stack. Docker's default bridge mode connects it to the host with a veth pair: a virtual cable with one end inside the container (eth0) and the other plugged into the host bridge docker0. Outbound traffic is NAT'd through the host's IP via iptables masquerade.
Because the container has its own port space, you publish ports to reach it: docker run -p 8080:80 maps host :8080 to container :80 via an iptables DNAT rule.
Storage — Persistence Beyond the Writable Layer
Anything written to the container's copy-on-write upper layer dies with the container. For data that must survive restarts, rebuilds, or rescheduling, mount storage from outside the layer stack.
Mount type
Managed by
Lifecycle
Use for
Volume
Docker (/var/lib/docker/volumes)
Independent of container
Databases, persistent app data
Bind mount
You (any host path)
Tied to host path
Dev: mount source code live
tmpfs
Kernel (RAM)
Vanishes on stop
Secrets, scratch, never-to-disk
# Named volume — survives docker rm, portable, Docker-managed
docker run -d -v pgdata:/var/lib/postgresql/data postgres:17
# Bind mount — live-edit source from the host (dev workflow)
docker run -v $(pwd):/app -w /app node:22 npm run dev
# tmpfs — sensitive data kept in RAM, never touches disk
docker run --tmpfs /run/secrets:size=64m myapp
Rule of thumb: volumes for state, bind mounts for local development, tmpfs for secrets. Never rely on the writable layer for anything you'd be sad to lose.
The Runtime Stack & OCI
"Docker" is a chain of daemons, not one binary. Understanding the split explains why Kubernetes dropped the Docker daemon yet still runs your images unchanged.
docker CLI — the client; talks to dockerd over a socket.
dockerd — the daemon: builds images, exposes the REST API, manages networks/volumes.
containerd — the core runtime: pulls images, manages container lifecycle, exposes CRI (Container Runtime Interface) so Kubernetes can drive it directly.
runc — the low-level OCI runtime that actually does the unshare + cgroup + pivot_root dance and execs your process.
Two OCI specs make this interoperable: the image spec (how layers + manifest are packaged) and the runtime spec (how a bundle is executed). Because Docker builds OCI images and containerd/runc run them, Kubernetes talks to containerd via CRI and never needs the Docker daemon — the same image you built locally runs unchanged in the cluster (cross-reference: the Kubernetes module builds directly on this CRI boundary).
Hardening in production
Rootless — run the whole daemon/runtime as an unprivileged user via user namespaces; a container root escape lands on a nobody UID.
Drop caps — --cap-drop ALL --cap-add NET_BIND_SERVICE.
Read-only rootfs — --read-only plus a tmpfs for scratch; nothing can tamper with the filesystem.
Non-root USER — set USER 10001 in the Dockerfile; never run app code as UID 0.
Sandboxed runtimes — swap runc for gVisor (userspace kernel), Kata, or Firecracker (microVMs) for multi-tenant / untrusted workloads.
# Hardened run: no new caps, read-only fs, non-root, sandboxed runtime
docker run --rm \
--read-only --tmpfs /tmp \
--cap-drop ALL --cap-add NET_BIND_SERVICE \
--security-opt no-new-privileges \
--user 10001:10001 \
--runtime runsc \
myorg/api:1.4.2
MODULE 11
AWS Deep-Dive — How the Cloud Works
Regions, IAM, and the core compute/storage/network primitives — and how one HTTP request flows through them.
What AWS Actually Is
Strip away the marketing and AWS is a fleet of regional data centers exposed as API-driven primitives. Every action — launch a VM, put an object, open a firewall port — is an authenticated HTTPS call to a service endpoint, gated by IAM and billed per unit of use. You do not need to memorize the 300-service catalog. You need the small set of primitives (compute, storage, network, identity) and a mental model of how a request flows through them. Everything else is a specialized wrapper on top.
API-first — the console, CLI, and SDKs all call the same signed REST/JSON APIs. There is no hidden control plane.
Auth-gated — no request executes without an IAM decision. Identity is the security perimeter, not the network.
Pay-per-use — you rent capacity by the second/GB/request. Idle still costs (EBS volumes, EIPs, provisioned capacity).
Global Infrastructure
AWS capacity is organized in three tiers. Get the failure-domain boundaries right and HA design falls out naturally.
Region — a geographic area (e.g. us-east-1, eu-west-1) with its own API endpoint and isolated from other regions. Data does not leave a region unless you explicitly replicate it. ~30+ regions worldwide.
Availability Zone (AZ) — one or more physically separate data centers within a region, on independent power/cooling/network but linked by low-latency (single-digit-ms), high-bandwidth fiber. A region has 3–6 AZs. Spreading across ≥2 AZs is the fundamental HA move.
Edge / PoP — 400+ points of presence that terminate TLS and serve cached content close to users. CloudFront (CDN) and Route53 (DNS) run here.
Some services are global (a single control plane, not tied to a region): IAM, Route53, CloudFront, WAF. Most are regional: EC2, S3, RDS, Lambda, DynamoDB. This matters for disaster recovery — a regional outage takes out regional services unless you have multi-region replication.
Region: us-east-1
<-------------- own API endpoint, isolated blast radius -------------->
+-----------+ +-----------+ +-----------+
| AZ-a | | AZ-b | | AZ-c | <-- physically separate DCs
| subnets, | | subnets, | | subnets, | low-latency fiber between
| EC2, EBS | | EC2, EBS | | EC2, EBS |
+-----------+ +-----------+ +-----------+
Design HA across >=2 AZs. Global services (IAM/Route53/CloudFront) sit outside.
IAM — How Auth Works
Every API call carries a principal (an IAM user, an assumed role, or an AWS service acting on your behalf). AWS evaluates the request against all attached identity policies and any resource policies, then applies a fixed precedence: an explicit Deny always wins, otherwise an explicit Allow grants access, otherwise the implicit Deny default blocks it.
Principal signs the request with credentials (access key or temporary STS token).
AWS authenticates the signature, then gathers all applicable policies.
User — a persistent identity with long-lived access keys. Fine for humans (behind MFA); an anti-pattern for workloads because keys leak and rarely rotate.
Role — an identity you assume to get short-lived STS credentials (minutes to hours). EC2/Lambda/ECS use instance/task roles so code never holds static keys. This is the correct workload pattern.
Policy — a JSON document listing Effect, Action, Resource, and optional Condition. Attach to identities or resources.
Design for least privilege: grant the narrowest actions on the narrowest resources, prefer assume-role over shared keys, and scope with conditions.
# Assume a role -> get short-lived STS creds (no long-lived keys on the box)
aws sts assume-role \
--role-arn arn:aws:iam::111122223333:role/app-read \
--role-session-name web-tier
# What identity am I right now?
aws sts get-caller-identity
Compute Spectrum
Compute is a ladder from raw VMs to pure functions — each rung trades control for less operational burden. Pick the highest rung that still fits your workload.
Service
Unit
You manage
Scaling
Best for
EC2
Virtual machine
OS, patching, capacity, autoscaling
Auto Scaling Groups (minutes)
Full control, legacy/stateful, custom kernels, GPU
Container orchestration, AWS-native, simpler than K8s
EKS
Pod
K8s workloads (AWS runs the control plane)
HPA / Cluster Autoscaler / Karpenter
Kubernetes portability, existing K8s tooling
Fargate
Container/task
Just the container (no nodes)
Per-task, serverless
Containers without managing EC2 hosts
Note Fargate is a launch mode, not a separate orchestrator — both ECS and EKS can run tasks/pods on Fargate so there are no EC2 nodes to patch or scale.
Storage
Three questions decide the storage primitive: object or block or file? single-AZ or multi-AZ? shared or attached-to-one-host? Match the access pattern, not the buzzword.
Service
Type
Durability / scope
Access
Best for
S3
Object store
11 nines, multi-AZ replicated
HTTPS API, flat namespace + key prefixes
Static assets, backups, data lake, logs; event notifications
EBS
Block volume
Single-AZ (replicated within the AZ)
Attached to one EC2 at a time
Boot disks, databases, low-latency block IO
EFS
File (NFS)
Multi-AZ, shared
POSIX mount from many instances
Shared filesystem across a fleet, lift-and-shift
Instance store
Ephemeral block
Local disk, lost on stop/terminate
Directly attached NVMe
Scratch, caches, shuffle data you can rebuild
S3 has no real folders — the “flat” namespace uses / in keys purely as a prefix convention. Object PUT/DELETE can fire event notifications to Lambda/SQS/SNS, which is the backbone of serverless data pipelines.
# Object put (S3) vs a volume that lives and dies with one EC2 host
aws s3 cp ./build/ s3://my-app-assets/site/ --recursive
aws ec2 create-volume --availability-zone us-east-1a --size 100 --volume-type gp3
Networking (VPC)
A VPC is your private, software-defined network in a region (a CIDR block like 10.0.0.0/16). You carve it into subnets, each pinned to one AZ. A subnet is public if its route table sends 0.0.0.0/0 to an Internet Gateway (IGW); it is private if outbound internet goes through a NAT Gateway (private hosts reach out, but nothing reaches in).
Security Group (SG) — stateful, allow-only, attached per-ENI (per instance/task). Return traffic is implicitly permitted. Your primary firewall.
NACL — stateless, allow and deny rules, evaluated per-subnet. You must open both directions; used for coarse subnet-level guardrails.
Partition-key sharded, auto-scales; on-demand or provisioned
Single-digit-ms at any scale, predictable key-based access
Aurora’s trick is a distributed storage layer shared by compute nodes, so replicas and failover are cheap. DynamoDB throughput hinges on partition-key design — a hot key throttles a single partition no matter how much capacity the table has. Choose on-demand for spiky/unknown traffic, provisioned (with auto-scaling) for steady, cost-optimized load.
How a Request Flows
Trace one HTTP request end-to-end. This is the single most-asked AWS interview question — name each hop’s job and the SG rule that permits it.
Route53 (global DNS) resolves app.example.com to the CloudFront/ALB endpoint. Job: name → address, health-checked routing.
CloudFront (edge CDN) terminates TLS near the user, serves cached static assets, forwards dynamic requests to the origin. Job: latency + TLS + caching + WAF.
ALB in a public subnet receives the request, does L7 routing, load-balances across healthy targets. Job: distribute + health-check. Its SG allows :443 from the internet.
ECS/Fargate task in a private subnet runs your app code. Job: business logic. Its SG allows :8080 only from the ALB’s SG — not the internet.
RDS in a private subnet serves the query. Job: persistence. Its SG allows :5432 only from the app task’s SG.
User
| DNS
v
Route53 --> CloudFront (TLS, cache, WAF) [edge]
|
v
ALB [public subnet] SG: :443 from 0.0.0.0/0
|
v
Fargate task [private subnet] SG: :8080 from ALB-SG only
|
v
RDS [private subnet] SG: :5432 from app-SG only
Only the ALB is internet-reachable. App + DB never have inbound from the world;
each tier's SG references the previous tier's SG (chained least privilege).
Well-Architected + Cost
The Well-Architected Framework is AWS’s review lens — six pillars to pressure-test any design. Interviewers love hearing you frame trade-offs against them.
Operational Excellence — run and monitor systems; automate change, respond to events, learn from failure.
Security — least privilege, encryption, IAM as the perimeter, auditability.
Reliability — recover from failure; multi-AZ, health checks, backups, graceful degradation.
Performance Efficiency — right-size, use the fitting primitive, scale elastically.
Cost Optimization — pay only for what you need; match pricing model to load shape.
Sustainability — minimize energy/resource footprint of the workload.
Cost hinges on picking the right pricing model for the load shape:
Model
Commitment
Discount vs on-demand
Best for
On-demand
None, pay per second/hour
Baseline (0%)
Spiky, short-lived, unpredictable
Reserved / Savings Plans
1 or 3 years
Up to ~72%
Steady, predictable baseline capacity
Spot
None; can be reclaimed with ~2-min notice
Up to ~90%
Fault-tolerant, interruptible batch/CI/stateless
MODULE 12
AWS Compute Deep-Dive
EC2 families & pricing, Auto Scaling, and the serverless-to-VM spectrum: Lambda, ECS, EKS, Fargate.
EC2 Instance Families
Every EC2 instance name encodes a family letter, a generation number, an optional capability suffix, and a size — e.g. c7g.2xlarge is Compute-optimized, gen 7, Graviton (g), 8 vCPU. Picking the wrong family is the most common right-sizing mistake: you pay for a resource dimension your workload never touches. The letter tells you the vCPU-to-memory ratio and any attached accelerator.
Family
Class
vCPU:Mem ratio
Reach for it when
T (t3, t4g)
Burstable general
~1:2–1:4, CPU credits
Bursty low-baseline: dev boxes, small web apps, microservices idle most of the time.
M (m7g)
General purpose
1:4
Balanced steady load: app servers, mid-size databases, backends with no skew.
C (c7g)
Compute optimized
1:2
CPU-bound: batch, media transcode, game servers, HPC, ad-serving, high-RPS APIs.
Huge in-memory sets: SAP HANA, big in-memory DBs, memory-resident caches at TB scale.
P (p5)
Accelerated (GPU)
NVIDIA H100/A100
ML training, distributed deep learning, CUDA/HPC simulation.
G (g5, g6)
Accelerated (GPU)
NVIDIA L4/A10G
ML inference, graphics/rendering, video encoding — cheaper GPU than P.
I (i4i, i3en)
Storage optimized
NVMe instance store
High local IOPS: NoSQL (Cassandra, Aerospike), OLTP, data warehousing, search.
Suffix decoder — g = Graviton (ARM64, ~20% cheaper/faster), a = AMD EPYC, i = Intel, n = extra network bandwidth, d = local NVMe.
T-family gotcha — burstable instances earn CPU credits at idle and spend them under load; sustained 100% CPU exhausts credits and either throttles (standard) or bills overage (unlimited mode).
Pricing Models
The list price is On-Demand; every other model trades flexibility for a discount. The interview skill is matching the commitment model to the workload's predictability and fault tolerance, not memorizing percentages — but know the ballpark numbers.
Model
Discount vs On-Demand
Commitment
Best for
On-Demand
0% (baseline)
None, per-second billing
Spiky/unknown load, dev, short-lived work.
Savings Plans (Compute)
up to ~66%
$/hour for 1 or 3 yr
Steady spend, flexible across EC2 + Fargate + Lambda, any region/family.
Reserved Instances (RI)
up to ~72%
1 or 3 yr, family/AZ scoped
Locked, steady-state fleets; less flexible than Savings Plans.
Licensing tied to physical sockets (BYOL Windows/Oracle), compliance isolation.
Savings Plans vs RIs — both need a term commitment; Savings Plans commit to a dollars-per-hour spend and float across family/region/service, while RIs pin to a specific config for a slightly deeper max discount.
Spot mechanics — you bid the current Spot price; AWS reclaims capacity with a 2-minute warning via instance metadata / a CloudWatch event. Handle SIGTERM and checkpoint work.
AMIs, Auto Scaling & Load Balancing
An AMI (Amazon Machine Image) is the immutable boot template — root-volume snapshot (OS + baked app), block-device mapping, and launch permissions. The modern pattern is immutable infrastructure: bake a golden AMI in CI (e.g. Packer), then roll the fleet by replacing instances rather than patching live ones. Never bake secrets into an AMI.
EC2 Auto Scaling stitches these together. A launch template (the successor to launch configurations) declares AMI, instance type, security groups, and user-data; the Auto Scaling Group (ASG) keeps a desired count across multiple AZs and registers new instances into an ELB target group so traffic follows capacity.
Launch template — versioned config the ASG uses to stamp out identical instances; supports mixed instances policy (blend On-Demand + Spot, multiple types) in one group.
Scaling policies — target tracking (hold avg CPU at 60%) is the default; step scaling reacts to alarm thresholds; scheduled scaling pre-warms for known peaks; predictive uses ML on history.
Health & draining — the ALB target group health check gates traffic; connection draining (deregistration delay) lets in-flight requests finish before an instance is terminated on scale-in.
ALB vs NLB — ALB is L7 (path/host routing, HTTP), NLB is L4 (raw TCP/UDP, static IP, ultra-low latency, millions of connections).
Lambda inverts the model: no servers, just a function plus a trigger. AWS creates one execution environment per concurrent request and freezes it warm for reuse (~5–15 min) after the handler returns. Understanding that lifecycle explains cold starts, concurrency math, and every scaling limit.
Trigger class
Sources
Retry / error model
Synchronous
API Gateway, ALB, Cognito
Caller waits; no auto-retry; client handles failure.
Asynchronous
S3, SNS, EventBridge, SES
Lambda retries twice, then DLQ / on-failure destination.
Poll-based (stream/queue)
SQS, Kinesis, DynamoDB Streams, MSK
Batch polling; use ReportBatchItemFailures for partial retry.
Concurrency = invocation rate × average duration. At 1,000 req/s and 200 ms each you need 1,000 × 0.2 = 200 concurrent environments. The account starts with a soft limit of 1,000 concurrent executions, a burst of 500–3,000 (region-dependent), then +500/min.
Cold start — first request in a new environment pays for code download + runtime init + your out-of-handler init. Go/Rust ~50–100 ms; Python/Node ~200–500 ms; Java 3–10 s (or ~200–500 ms with SnapStart, free).
Reserved concurrency — free; both caps a function's max and guarantees it that slice (protects a downstream DB, isolates a noisy function).
Provisioned concurrency — pre-warms N environments to kill cold starts on latency-critical paths; costs even while idle (~$0.0000045/GB-s).
Init outside the handler — put SDK clients and DB connections at module scope so they persist across warm invocations instead of rebuilding every call.
import boto3
# Runs once per cold start, reused on every warm invocation:
table = boto3.resource("dynamodb").Table("orders")
def handler(event, context):
failures = []
for record in event["Records"]: # SQS batch
try:
process(record)
except Exception:
failures.append({"itemIdentifier": record["messageId"]})
return {"batchItemFailures": failures} # only failed msgs retry
ECS vs EKS
Both orchestrate containers; the split is AWS-native simplicity versus Kubernetes portability. ECS is AWS's homegrown scheduler with a free, fully managed control plane. EKS runs upstream Kubernetes with an AWS-managed control plane billed at $0.10/hour (~$73/month) per cluster. Same containers, very different operational surface.
Concept map — ECS Task ≈ k8s Pod; ECS Service ≈ Deployment + HPA; Task Definition ≈ Pod spec + ConfigMap/Secret; Capacity Provider ≈ Node Group + autoscaler.
Both run on either Fargate (serverless) or EC2/managed-node data planes, and both use the awsvpc ENI-per-task/pod networking model.
Fargate vs EC2 Launch Type
Fargate is the serverless data plane for ECS and EKS — you hand AWS a task/pod with a vCPU+memory request and it provisions, patches, and isolates the host. There are no nodes to manage, no cluster capacity to plan, and each task gets a dedicated micro-VM kernel (stronger isolation than containers sharing an EC2 host OS). The EC2 launch type is the opposite: you own the instances, the AMI, patching, and bin-packing — in exchange for full control and GPU access.
Aspect
Fargate
EC2 launch type
You manage
Tasks/pods only
Instances + OS + patching + capacity
Billing
Per-task vCPU+memory, per-second
Per-instance (running or idle)
Isolation
Micro-VM per task (kernel-level)
Containers share host kernel
Startup
~30–90 s cold provision
Fast if instance already running
GPU / custom AMI
No
Yes
Spot
Fargate Spot (~70% off)
EC2 Spot (~90% off)
Wins on cost when
Variable / bursty / small
Steady, high sustained utilization
Right-size hard — Fargate memory is a hard ceiling; hit it and the OOM killer terminates the task instantly (no swap). Set CPU/memory from real Container Insights data, not guesses.
Cut startup — small images, ECR in-region, VPC endpoints, and Seekable OCI (SOCI) lazy-loading can drop task start from ~90 s to ~10 s.
Cost reality — for a flat 24/7 workload Fargate can run several times the price of a right-sized EC2 instance; the premium buys away all node management.
The whole module collapses into one axis: how much of the stack do you want to manage versus how much control do you need? Lambda is maximum abstraction (function), EC2 is maximum control (VM), and ECS/EKS/Fargate sit between on the container spectrum.
Service
Unit of deploy
You manage
Scaling
Best for
Lambda
Function
Code only
Automatic, per-request (0 → thousands)
Event-driven, bursty, short (<15 min) work.
Fargate
Task / Pod
Container + task spec
App Auto Scaling on task count
Serverless long-running containers, no node ops.
ECS (EC2)
Task on your EC2
Containers + EC2 fleet
Service scaling + ASG/capacity provider
Steady containers, GPU, high utilization, AWS-native.
EKS
Pod
k8s workloads (+ nodes if not Fargate)
HPA / Cluster Autoscaler / Karpenter
Kubernetes shops, portability, complex platforms.
EC2
Virtual machine
OS, runtime, everything on it
EC2 Auto Scaling Groups
Full control, legacy/stateful, custom kernels, GPU.
MANAGE MORE <-----------------------------------------------> CONTROL MORE
Lambda --> Fargate --> ECS/EKS on Fargate --> ECS/EKS on EC2 --> EC2
function task serverless pods you own nodes raw VM
event-driven no nodes k8s w/o node ops GPU / bin-pack everything
Lambda vs Fargate — Lambda for <15 min, bursty, event-driven; Fargate for long-lived services, WebSockets, big memory (up to 120 GB), or full Docker needs.
Fargate vs EC2 launch type — Fargate for variable load and zero ops; EC2 for GPU, custom AMIs, or steady high utilization where reserved capacity is cheaper.
MODULE 13
AWS Storage Deep-Dive
Object, block, file, and archive — S3 classes and durability, EBS volume types, EFS, Glacier.
The Object Model: S3 Is Not a Filesystem
S3 exposes a flat, per-bucket key namespace. There are no real directories. A key like logs/2026/07/app.log is a single opaque string; the "folders" you see in the console are a UI fiction rendered by splitting on /. This matters: you cannot mv a "folder" atomically, and listing is a prefix scan, not a directory read.
Bucket — globally unique namespace, region-pinned. Bucket names collide across all AWS accounts on Earth.
Object — data blob (0 bytes to 5 TB) + metadata + a version ID. The unit of durability and access control.
Key — the full path-like string that uniquely identifies an object within a bucket. Prefix = any leading substring of the key.
ETag — content hash (MD5 for single PUT; composite for multipart). Used for integrity and conditional requests.
Eleven Nines of Durability & Strong Consistency
S3 Standard is designed for 99.999999999% (11 nines) annual durability. The mechanism: every PUT is synchronously replicated across a minimum of three Availability Zones before the request returns 200 OK. Continuous integrity checks (checksums, cross-AZ repair) heal bit-rot and device loss. Practically, 11 nines means if you store 10 million objects you can expect to lose one object roughly every 10,000 years.
Since December 2020, S3 provides strong read-after-write consistency for all objects in all regions, at no cost and with no performance penalty:
A successful PUT of a new object is immediately readable by a subsequent GET — no eventual-consistency window.
Overwrite PUTs and DELETEs are also strongly consistent for read-after-write and list-after-write.
Durability ≠ availability. Standard targets 99.99% availability; One Zone classes drop to 99.5% because a single-AZ loss takes the data offline.
S3 Storage Classes: Cost vs Retrieval
Every class stores the same object identically; you trade first-byte latency and per-GB price against retrieval fees and minimum storage durations. Prices below are us-east-1, per GB-month, as a stable reference point for relative comparison.
Class
$/GB-mo
Retrieval latency
Min duration
AZs
Best for
S3 Standard
0.0230
ms
none
≥3
Hot data, active workloads
Intelligent-Tiering
tier price + monitoring
ms
none
≥3
Unknown/shifting access patterns
Standard-IA
0.0125
ms
30 days
≥3
Infrequent but latency-sensitive
One Zone-IA
0.0100
ms
30 days
1
Recreatable, non-critical copies
Glacier Instant Retrieval
0.0040
ms
90 days
≥3
Archive needing instant access
Glacier Flexible Retrieval
0.0036
1 min – 12 hr
90 days
≥3
Archive, minutes-to-hours OK
Glacier Deep Archive
0.00099
12 – 48 hr
180 days
≥3
Compliance, tape replacement
Intelligent-Tiering auto-moves objects across frequent/infrequent/archive tiers based on observed access, charging a small per-object monitoring fee instead of a retrieval fee — the right default when you genuinely cannot predict access.
Lifecycle, Versioning & Events
Versioning keeps every mutation as a distinct version ID; a DELETE just writes a delete marker, so nothing is truly lost. Combined with MFA Delete and Object Lock (WORM), it is the primary defense against ransomware and fat-finger overwrites. Lifecycle policies then transition or expire objects by age and prefix, automating the cost ladder from the table above.
Event notifications fire on object create/delete/restore to SQS, SNS, Lambda, or EventBridge — the backbone of serverless pipelines (e.g. thumbnail generation on upload).
Security: Block Public Access, Policies & Encryption
Access is granted only when there is an explicit Allow, no explicit Deny, and Block Public Access settings permit it. The layers, outermost first:
Block Public Access (BPA) — account- and bucket-level master switch (four flags). Overrides any policy or ACL that would expose data publicly. Enable all four unless you are deliberately hosting a public site.
IAM policy — identity-based; what a given principal may do.
ACLs — legacy per-object/bucket grants. Disabled by default on new buckets ("Bucket owner enforced"); avoid.
Pre-signed URLs — time-boxed access to one object (seconds to 7 days) without granting standing permissions.
Encryption at rest is on by default (SSE-S3). Options: SSE-S3 (AES-256, AWS-managed keys), SSE-KMS (KMS key, per-request audit trail + key-policy control; use S3 Bucket Keys to cut KMS cost), and SSE-C (customer-supplied key, you manage rotation).
S3 scales request rate per prefix, not per bucket. Each prefix sustains 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second, and S3 auto-partitions as load grows. To go higher, spread keys across many prefixes — a workload hitting one prefix hard can be sharded into N prefixes for roughly N× the throughput.
Multipart upload — recommended above 100 MB, mandatory above 5 GB. Up to 10,000 parts, 5 MB–5 GB each, uploaded in parallel with per-part retry.
Byte-range GETs — fetch parts of a large object in parallel; resume interrupted downloads.
Transfer Acceleration — routes uploads through CloudFront edge locations for long-haul speedups.
S3 Select — push down SQL to return only matching rows/columns from CSV/JSON/Parquet, cutting bytes scanned and egress.
Modern guidance: random high-entropy key prefixes are no longer required for hot buckets since S3 auto-scales partitions — but deliberate prefix design still helps you exceed the per-prefix ceiling predictably.
EBS: Block Storage Volume Types
EBS is network-attached block storage for a single EC2 instance, presented as a raw device. A volume lives in exactly one AZ and is replicated within that AZ only. Snapshots are incremental point-in-time backups stored in S3 (not reachable via the S3 API); they are how you clone a volume into another AZ or region.
Throughput-heavy sequential: logs, big data, streaming
sc1
HDD
250
250 MiB/s
Cold, infrequently accessed, lowest-cost HDD
gp3 baseline is 3,000 IOPS / 125 MiB/s free of size, then you pay to provision more — usually ~20% cheaper than gp2 at equal performance. Elastic Volumes let you grow size, change type, or tune IOPS live without detaching. Multi-Attach (io1/io2 only) attaches one volume to multiple instances in the same AZ for clustered apps.
EFS vs FSx vs Instance Store
When multiple instances need a shared filesystem, block storage (single-attach EBS) does not fit. EFS is a fully managed elastic NFSv4 filesystem, mounted concurrently by thousands of Linux instances / containers across multiple AZs. It grows and shrinks automatically — no capacity to provision — and offers Standard, One Zone, and IA tiers with lifecycle management, plus General Purpose vs Max I/O performance modes and Bursting/Provisioned/Elastic throughput modes.
FSx is the family for when you need a specific protocol or engine that EFS does not speak:
Service
Protocol / engine
Reach
Use case
EFS
NFS v4 (Linux)
Multi-AZ, elastic
Shared Linux/container storage, home dirs, CMS
FSx for Windows File Server
SMB / NTFS
Single/Multi-AZ
Windows apps, Active Directory integration
FSx for Lustre
Lustre (HPC)
Scratch/persistent
HPC, ML training, S3-linked high-throughput
FSx for NetApp ONTAP
NFS+SMB+iSCSI
Multi-AZ
Multiprotocol, snapshots, on-prem ONTAP lift
FSx for OpenZFS
NFS (ZFS)
Single/Multi-AZ
ZFS snapshots/clones, low-latency NFS
Instance Store is physically-attached NVMe/SSD on the host — the fastest storage available (millions of IOPS) but ephemeral: data is lost on stop, hibernate, or termination, and it cannot be snapshotted. Use it only for scratch, caches, buffers, and replicated/reconstructable data.
Property
EBS
Instance Store
Persistence
Survives stop/terminate
Lost on stop/terminate
Snapshots
Yes (to S3)
No
Detach / re-attach
Yes
No (bound to host)
Performance
Good (gp3 16k IOPS)
Extreme (local NVMe)
Glacier: Archives, Vaults & Retrieval Tiers
Glacier is the archival tier, now surfaced mainly as S3 storage classes but retaining the vault/archive model. An archive (up to 40 TB, immutable/WORM once written) is the storage unit; a vault groups archives. Vault Lock enforces compliance controls that even the account owner cannot revoke — e.g. "no deletion for 7 years" for SEC 17a-4 or HIPAA retention.
Retrieval from Glacier Flexible/Deep Archive is asynchronous — you request a restore, then read once it lands. Tiers trade speed for cost:
Tier
Flexible Retrieval
Deep Archive
Cost
Expedited
1–5 min
n/a
Highest
Standard
3–5 hr
~12 hr
Default
Bulk
5–12 hr
~48 hr
Lowest
Glacier Instant Retrieval is the exception: it is an S3 class with millisecond synchronous access (no restore step) for archives you occasionally need fast, priced between Standard-IA and Flexible. Early-delete fees apply below the class minimum (90 days for Glacier tiers, 180 for Deep Archive).
MODULE 14
AWS Database Deep-Dive
Pick the right store: RDS/Aurora, DynamoDB, ElastiCache/Redis, Redshift — and why.
RDS vs Aurora: Managed Relational, Two Storage Models
Both speak SQL and run MySQL/PostgreSQL, but they replicate differently. RDS bolts HA on at the instance layer — a synchronous standby you never read. Aurora decouples compute from a shared, distributed storage fleet, which is why its failover and read-scaling numbers are structurally better.
RDS engines — PostgreSQL, MySQL, MariaDB, Oracle, SQL Server (+ Aurora). Oracle/SQL Server are RDS-only; Aurora is MySQL/PostgreSQL-compatible only.
Multi-AZ (HA, not scale) — synchronous standby in a second AZ. Not readable; it just waits. Automatic failover on infra failure via DNS CNAME swap.
Read replicas (scale, not HA) — asynchronous copies that serve reads; eventually consistent; promotable to standalone. RDS: up to 15 for MySQL/MariaDB/PostgreSQL (5 for Oracle/SQL Server). Aurora: up to 15, all sharing one storage volume.
Aurora storage — log-structured, auto-grows to 128 TB, 6 copies across 3 AZs. Writes ack on 4/6 quorum; reads need 3/6. Replicas attach to the same volume, so no replication lag from copying data blocks.
Dimension
RDS (Multi-AZ)
Aurora
Replication
Instance-level, 1 sync standby
Storage-level, 6-way across 3 AZs
Failover time
~60–120 s (DNS + standby promote)
< 30 s (replica already attached)
Read replicas
Up to 15 (5 for Oracle/SQL Server), each its own copy
DynamoDB: Serverless NoSQL, Single-Digit ms at Any Scale
DynamoDB trades JOINs and ad-hoc queries for predictable single-digit-millisecond latency that does not degrade as the table grows to terabytes. The price of that: you design the schema around your access patterns, not around normalized entities.
Partition key (PK) — hashed to place the item on a physical partition. Determines distribution.
Sort key (SK) — items sharing a PK are stored together, sorted by SK. Enables range queries (begins_with, between) within a partition.
GetItem = PK(+SK), one item, fastest. Query = PK + SK condition, a range. Scan = full table read — expensive, avoid.
GSI — different PK/SK, own capacity, eventually consistent, up to 20/table, addable anytime. LSI — same PK, alternate SK, shares table capacity, strongly-consistent option, up to 5, must be defined at create time.
DAX — in-memory, API-compatible cache; drops reads to microseconds (~10x) with no app changes. Streams — ordered change log feeding Lambda (CDC, invalidation, fan-out).
Capacity mode
How you pay
Use when
On-demand
$0.25 / M reads, $1.25 / M writes; scales instantly
Spiky / unknown traffic, new apps, scale-to-zero
Provisioned
RCU $0.00013/hr, WCU $0.00065/hr; + Auto Scaling
Steady, predictable load; reserved capacity = up to 77% off
Single-table design collapses related entities into one table so a single Query hydrates a whole aggregate:
PK SK GSI1PK GSI1SK
-------------- ---------------- --------------- --------------
USER#123 PROFILE STATUS#pending ORDER#2024-001
USER#123 ORDER#2024-001 STATUS#pending ORDER#2024-002
ORDER#2024-001 ITEM#PROD-789
PRODUCT#789 METADATA
Get profile -> PK=USER#123, SK=PROFILE
Get user orders -> PK=USER#123, SK begins_with "ORDER#"
Orders by status-> GSI1, PK=STATUS#pending
import boto3
tbl = boto3.resource("dynamodb").Table("app")
# All orders for a user in one round trip
resp = tbl.query(
KeyConditionExpression="PK = :u AND begins_with(SK, :p)",
ExpressionAttributeValues={":u": "USER#123", ":p": "ORDER#"},
) # single-digit ms, no Scan
ElastiCache: Redis vs Memcached
ElastiCache is managed in-memory (sub-ms) for caching, sessions, and real-time features. Two engines with very different capabilities — the answer is Redis roughly 95% of the time.
Cluster mode disabled — one shard, primary + up to 5 read replicas. Total data = one node (up to ~635 GB on r6g.16xlarge). Vertical scaling only.
Cluster mode enabled — data partitioned across shards, capacity = sum of shards, online resharding, higher write throughput. Watch CROSSSLOT on multi-key ops.
Serverless — auto-scales 0→TBs, Multi-AZ by default, pay per ECPU + storage; use for unknown/variable load.
Durability tier — need a Redis-compatible primary store with zero data loss? That is MemoryDB, not ElastiCache.
The high-value patterns lean on Redis data structures, not plain key-value:
# Leaderboard: Sorted Set, O(log n) rank + range
r.zadd("lb:game", {"player1": 1600, "player2": 1200})
r.zrevrange("lb:game", 0, 9, withscores=True) # top 10
r.zrevrank("lb:game", "player1") # exact rank
# Cache-aside with TTL (the default caching pattern)
def get_user(uid):
key = f"user:{uid}"
if (hit := r.get(key)):
return json.loads(hit)
row = db.query("SELECT * FROM users WHERE id=%s", uid)
r.setex(key, 3600, json.dumps(row)) # 1h safety TTL
return row
# Distributed lock: SET NX EX, release via Lua compare-and-del
r.set("lock:job", token, nx=True, ex=10)
Redshift: Columnar MPP Data Warehouse
Redshift is not an OLTP database — it is a petabyte-scale analytics engine. The design choices (columnar storage, MPP, compression) all optimize scanning billions of rows over a few columns, which is exactly what OLTP indexes are bad at.
Leader node parses SQL, builds the plan, aggregates results. Compute nodes execute slices of the plan in parallel (MPP).
Columnar storage — data laid out by column, so a query touching 3 of 200 columns reads only those blocks; huge I/O reduction vs row stores.
Compression (encoding) — per-column, auto-selected; shrinks storage and further cuts I/O.
Spectrum queries S3 directly (no load step); Concurrency Scaling adds transient clusters for bursts; Materialized views pre-compute; WLM prioritizes short queries over long ETL.
-- Columnar + MPP: scans only the columns referenced, across all slices
SELECT date_trunc('day', order_ts) AS day,
region,
sum(amount) AS revenue
FROM fact_orders -- billions of rows
WHERE order_ts > current_date - 30
GROUP BY 1, 2
ORDER BY revenue DESC; -- seconds, not minutes
Selection Matrix: Workload → Service
The interview move is to start from the access pattern, not the brand name. Map the workload, then justify.
Workload
Service
Why
JOINs, ACID, ad-hoc queries
RDS / Aurora
Relational model, full transactions
Same, variable/bursty load
Aurora Serverless v2
Scales ACUs in seconds, no idle cost
Same, need Oracle / SQL Server
RDS
Aurora is MySQL/PostgreSQL only
Key-value / key-range at massive scale
DynamoDB
Flat < 10 ms latency, serverless, no JOINs
Read-heavy hotspot on any DB
ElastiCache Redis
Sub-ms cache-aside, offload the primary
Sessions / leaderboards / rate limit
ElastiCache Redis
Hashes, Sorted Sets, TTL, atomic ops
DynamoDB reads at microseconds
DAX
API-compatible in-memory front cache
BI / dashboards / big scans
Redshift
Columnar MPP over petabytes
Redis as durable primary store
MemoryDB
Multi-AZ log, zero data loss on failover
Design Drills
Every store decision reduces to: what are the read/write patterns, the scale, the consistency need, and the latency budget? Rehearse defending a pick out loud.
Global session store, 10M sessions. ElastiCache Redis (or Serverless), Hash per session, 30-min sliding TTL, Multi-AZ so a mass re-login does not stampede auth. Sessions are ephemeral → optimize speed/availability over durability.
E-commerce orders, unpredictable spikes. DynamoDB on-demand, single-table (USER#id PK, ORDER#date SK), GSI on status, Streams → Lambda for fulfillment. Add DAX if a product page is a read hotspot.
Financial ledger with reporting. Aurora PostgreSQL (ACID + JOINs), read replicas for reports, nightly ETL into Redshift for BI dashboards.
MODULE 15
AWS Networking Deep-Dive
VPC design, routing, and edge: subnets, NAT, peering, Transit Gateway, PrivateLink, Route53, CloudFront, API Gateway.
VPC Anatomy & CIDR
A VPC is a logically isolated slice of AWS you carve out with a private CIDR block. Everything else — subnets, route tables, gateways — hangs off that address space. CIDR is chosen once and effectively forever: overlap with a future peer, VPN, or Transit Gateway and you cannot route, so you rebuild.
CIDR block — the VPC's IPv4 range, e.g. 10.0.0.0/16 = 65,536 addresses. Prefer RFC1918 space; avoid the default 172.31.0.0/16. Secondary CIDRs can be bolted on later.
Subnet — a CIDR slice pinned to exactly one AZ. AWS reserves 5 IPs per subnet (network, VPC router, DNS, future, broadcast), so a /24 gives 251 usable, a /28 (minimum) gives 11.
Public subnet — route table sends 0.0.0.0/0 to an Internet Gateway; instances may hold public IPs.
Private subnet — 0.0.0.0/0 points at a NAT Gateway (outbound-only) or nowhere (isolated).
IGW — horizontally scaled, redundant, free; one per VPC; enables bidirectional internet.
NAT Gateway — AWS-managed, AZ-scoped, needs an Elastic IP; lets private instances reach out (patches, APIs) while blocking inbound initiation.
Elastic IP — a static public IPv4 you own; attached to the NAT GW or a bastion. Idle EIPs now bill, and all public IPv4 carries an hourly charge.
Public vs private is purely a routing property. The same subnet flips public or private by swapping its route table — nothing about the subnet's CIDR changes.
CIDR
Total IPs
Usable (−5)
Typical use
/16
65,536
65,531
Whole VPC
/20
4,096
4,091
Large tier subnet
/23
512
507
App tier per AZ
/24
256
251
Public / data tier
/28
16
11
Min subnet size
VPC 10.0.0.0/16
AZ-a public 10.0.0.0/24 private 10.0.10.0/23 data 10.0.20.0/24
AZ-b public 10.0.1.0/24 private 10.0.12.0/23 data 10.0.21.0/24
AZ-c public 10.0.2.0/24 private 10.0.14.0/23 data 10.0.22.0/24
Route Tables & NAT Economics
Every subnet associates with exactly one route table; routes are evaluated most-specific-prefix-first. The implicit local route (the VPC CIDR) always wins for in-VPC traffic and cannot be removed or overridden.
Destination
Target
Meaning
10.0.0.0/16
local
Stays inside the VPC
172.16.0.0/16
pcx-…
Peered VPC
192.168.0.0/16
vgw-…
On-prem via VPN gateway
0.0.0.0/0
igw-… / nat-…
Default route (public / private)
Internet Gateway
NAT Gateway
Direction
Bidirectional
Outbound only
Cost
Free
~$0.045/hr + $0.045/GB
Availability
Region-wide, redundant
AZ-scoped (1 per AZ for HA)
Throughput
Unmetered
Bursts to ~45 Gbps
Subnet
Public
Lives in public, serves private
Security Groups vs NACLs
Two firewall layers, deliberately different. Security Groups guard the instance ENI and are stateful; NACLs guard the subnet boundary and are stateless. In interviews the discriminator is return traffic.
Feature
Security Group
NACL
Level
Instance / ENI
Subnet
State
Stateful
Stateless
Rules
Allow only
Allow and Deny
Evaluation
All rules (logical OR)
Numbered, first match wins
Default
Deny in, allow out
Default NACL allows all
Can reference
Other SG IDs
CIDR only
SG chaining is the least-privilege pattern: reference SGs, not hardcoded IPs, so scaling instances stay covered. ALB-SG allows 80/443 from the world; App-SG allows 8080 from ALB-SG; DB-SG allows 5432 from App-SG. No IP ever appears past the edge.
Peering vs Transit Gateway vs PrivateLink
Three ways to connect networks, each solving a different scaling problem. Peering is a 1:1 wire; Transit Gateway is a hub-and-spoke router; PrivateLink exposes a single service, not a whole network.
VPC Peering — direct point-to-point, private IP routing, cross-region/cross-account. Free (data transfer only). No transitive routing (A↔B and B↔C does not give A↔C) and no overlapping CIDRs. Great for 2–3 VPCs.
Transit Gateway — a regional cloud router; each VPC/VPN/DX is an attachment. Transitive by design, centralized route tables, scales to thousands of VPCs. ~$0.05/hr per attachment + $0.02/GB.
VPC Endpoints — Gateway type — route-table target for S3 and DynamoDB only. Free. Traffic never leaves the AWS backbone; controlled by a prefix list.
VPC Endpoints — Interface type (PrivateLink) — an ENI with a private IP for 100+ services (SSM, ECR, SQS, SNS, Kinesis) or your own NLB-fronted service. ~$0.01/hr per AZ + $0.01/GB; DNS resolves to the private IP; SG-controlled.
Peering
Transit Gateway
PrivateLink
Topology
Full mesh (n²)
Hub-and-spoke
Service exposure
Transitive
No
Yes
N/A (single endpoint)
Scope
Whole VPC ↔ VPC
Many VPCs + hybrid
One service only
CIDR overlap
Not allowed
Not allowed
Allowed (no routing)
Cost
Data only
Attach + GB
Hourly + GB
Route 53 — DNS & Routing Policies
Route 53 is authoritative DNS plus a traffic router. A hosted zone holds records for a domain: public zones answer the internet, private zones answer only inside associated VPCs (internal service discovery). The AWS-only Alias record is the workhorse — it points at ELB, CloudFront, or S3, works at the zone apex (where CNAME is illegal), resolves with no extra lookup, and is free.
Record
Maps to
Note
A / AAAA
IPv4 / IPv6
Direct address
CNAME
Another hostname
Not valid at apex
Alias
AWS resource
Apex-safe, free, health-aware
MX / TXT / NS
Mail / metadata / delegation
SPF, DKIM, subdomain NS
Policy
Behavior
Use case
Simple
One record, no health check
Single resource
Weighted
Split by weight (e.g. 80/20)
A/B, canary rollout
Latency
Lowest-latency region wins
Multi-region perf
Failover
Primary→secondary on health fail
Active-passive DR
Geolocation
By user's country/continent
Localization, compliance
Multivalue
Up to 8 healthy answers
Cheap health-checked LB
Health checks probe an endpoint (HTTP/HTTPS/TCP) on an interval; failover, multivalue, and weighted policies consult them to withdraw dead targets. Alias records with evaluate_target_health = true inherit the ELB's own health.
resource "aws_route53_record" "primary" {
zone_id = aws_route53_zone.main.zone_id
name = "app.example.com"
type = "A"
alias {
name = aws_lb.primary.dns_name
zone_id = aws_lb.primary.zone_id
evaluate_target_health = true
}
failover_routing_policy { type = "PRIMARY" }
set_identifier = "primary"
health_check_id = aws_route53_health_check.primary.id
}
CloudFront — the Edge CDN
CloudFront is a global CDN of edge PoPs that caches content close to users, terminates TLS at the edge, and shields the origin. A distribution has one or more origins (S3, ALB/EC2 custom origin, MediaPackage) and a set of cache behaviors that map path patterns to an origin plus a cache/origin-request policy.
Cache behavior — path pattern (e.g. /static/*) → origin, with TTLs and which headers/cookies/query strings form the cache key.
TTL & invalidation — objects live for their TTL; forcing early eviction via invalidation costs money, so version asset filenames instead.
OAC (Origin Access Control) — the modern replacement for OAI; locks an S3 origin so objects are reachable only through CloudFront (SigV4-signed), never via the public bucket URL.
Security — free ACM TLS certs, AWS WAF and Shield for L7/DDoS, signed URLs/cookies for paid content.
Edge compute — CloudFront Functions (lightweight JS, header/URL rewrites, sub-ms) vs Lambda@Edge (Node/Python, heavier logic like auth or image resize).
CloudFront Functions
Lambda@Edge
Runtime
JS (edge)
Node / Python
Latency
Sub-millisecond
Milliseconds
Triggers
Viewer req/resp
Viewer + origin req/resp
Use
Header/URL rewrite, redirects
Auth, image resize, A/B
API Gateway — REST, HTTP & WebSocket
API Gateway is a managed front door for APIs: it authenticates, throttles, transforms, caches, and routes to Lambda, HTTP backends, or AWS services directly. Pick the API flavor by feature-vs-cost, not habit.
REST API
HTTP API
WebSocket API
Cost/latency
Higher
~70% cheaper, lower latency
Per-message + connection-minutes
Features
Full: validation, transforms, WAF, API keys, cache
Lean: JWT, proxy, CORS
Persistent bidirectional
Authorizers
IAM, Cognito, Lambda
JWT, Lambda, IAM
Lambda ($connect)
Best for
Feature-rich public APIs
Serverless microservices
Chat, live dashboards
Authorizers — IAM (SigV4), Cognito User Pools, or a Lambda authorizer for custom/3rd-party JWT logic. HTTP APIs add native JWT validation.
Throttling — account and per-stage/per-method rate + burst limits; Usage Plans + API Keys enforce per-client quotas (monetization, tiering).
Stages — named deployments (dev/test/prod), each with its own variables and throttles; canary releases shift a % of traffic to a new deployment.
Caching — REST APIs can provision a response cache with a TTL to cut backend calls and latency.
Tie it together by tracing one request end to end. Notice where NAT is and is not involved, and how stateful SGs let the response flow back with no extra rules.
User browser
| 1. DNS: Route 53 resolves app.example.com -> CloudFront / ALB
v
CloudFront edge (optional) -- TLS, cache check, WAF
| 2. miss -> origin fetch over HTTPS
v
Internet Gateway (region edge of VPC)
| 3. 0.0.0.0/0 route brought traffic to the public subnet
v
Public subnet: ALB (ALB-SG allows 443 from 0.0.0.0/0)
| 4. listener rule -> target group, internal routing
v
Private subnet: EC2 app (App-SG allows 8080 FROM ALB-SG only)
| 5. queries DB via App-SG -> DB-SG (5432)
v
Data subnet: RDS
^
| 6. response retraces path; SGs are STATEFUL so replies
| need no return rule. NAT is NOT used -- all in-VPC.
+-- outbound to internet (patch pull) WOULD use NAT GW.
MODULE 16
AWS Security & Compliance
IAM policy evaluation, KMS envelope encryption, WAF, and the shared-responsibility model.
Shared Responsibility Model
AWS splits security into two accountable halves. The dividing line moves depending on the abstraction level of the service you pick — the more managed the service, the more AWS owns.
AWS — security of the cloud — physical data centers, hypervisor, host OS patching, network fabric, managed-service internals (e.g. the RDS engine host, Lambda runtime sandbox).
Customer — security in the cloud — IAM policies, guest OS patching (EC2), security groups, encryption choices, data classification, application code, and who can call what.
Model
Customer patches OS?
Customer owns
IaaS (EC2)
Yes
Guest OS, runtime, app, data, firewall, IAM
Container (ECS/EKS on EC2)
Node OS yes
Image, task role, app, data
Serverless (Lambda, Fargate)
No
Code, IAM, data, dependencies
Managed (S3, DynamoDB)
No
Access policy, encryption config, data
IAM Principals & Policy Types
A principal is any entity that makes a signed request. IAM answers one question per request: is this principal authorized for this action on this resource under these conditions?
Users — long-lived identity for a human or legacy app. Has a console password and/or access keys. Access keys are the thing that leaks — prefer roles.
Roles — an identity with no long-term credentials. Assumed to mint short-lived STS credentials. Used by services (EC2, Lambda), cross-account access, and federated SSO.
Service principals — AWS services (e.g. lambda.amazonaws.com) named in a role's trust policy so the service can assume it.
Root user — the account owner; unrestricted, cannot be limited by IAM or SCP. Lock it: hardware MFA, no access keys, billing only.
Permissions come from up to five policy types evaluated together. Know which are guardrails (cap the max) versus grants (add permission).
Policy type
Attached to
Role in evaluation
Identity-based
User / group / role
Grants — what the principal can do
Resource-based
Resource (S3 bucket, KMS key, SQS)
Grants — names a Principal; enables cross-account
SCP
Org OU / account
Guardrail — caps max perms, never grants
Permission boundary
User / role
Guardrail — caps that identity's max perms
Session policy
Passed at AssumeRole
Guardrail — caps the temporary session
Policy Evaluation Logic
The single most tested IAM fact: evaluation is deny-biased. Everything starts denied; an allow can lift that; any explicit deny — from any policy type — is final.
Request → collect ALL applicable policies
1. Explicit DENY anywhere? ──► DENY (wins, always)
2. SCP / boundary / session allow the action? no ──► DENY
3. Explicit ALLOW in identity or resource policy? no ──► DENY (implicit)
4. Otherwise ──► ALLOW
Precedence: explicit deny > explicit allow > implicit (default) deny
For guardrails, the effective permission set is an intersection: a principal can only do what identity policy AND SCP AND permission boundary AND session policy all permit. Widening one does nothing if another is narrower.
Same account — identity or resource policy granting is enough (a union of grants), still subject to guardrails.
Cross account — you need a grant on both sides: the resource policy in Account B must name Account A's principal, AND Account A's identity policy must allow the action.
Permission boundary use case — let developers create roles without privilege escalation: any role they mint is capped by the boundary you attached, so they cannot grant more than they hold.
STS, AssumeRole & Instance Profiles
Roles are useless until assumed. STS (Security Token Service) exchanges a trust relationship for temporary credentials — an access key, secret key, and a session token — that expire in 15 minutes to 12 hours (1 hour default; role chaining is hard-capped at 1 hour).
The role's trust policy (a resource-based policy on the role) says who may assume it via sts:AssumeRole.
A principal calls AssumeRole; STS checks the trust policy, then returns short-lived creds scoped by the role's permission policy (and any session policy).
Creds auto-expire — nothing to rotate, nothing to leak long-term.
An instance profile is the container that binds an IAM role to an EC2 instance. The EC2 metadata service (IMDS) surfaces auto-rotating credentials at a link-local endpoint; the SDK reads them transparently. Always enforce IMDSv2 (token-based) to block SSRF credential theft.
KMS creates and guards cryptographic keys inside FIPS-validated HSMs. The root key material — the KMS key (formerly CMK) — never leaves KMS in plaintext. To encrypt bulk data efficiently, KMS uses envelope encryption.
ENCRYPT
1. App calls GenerateDataKey(KeyId)
2. KMS returns: plaintext data key + data key encrypted under the KMS key
3. App encrypts data with plaintext data key (fast, local, AES-256)
4. Store: ciphertext + encrypted data key ; wipe plaintext key from RAM
DECRYPT
1. App sends encrypted data key to KMS Decrypt
2. KMS unwraps it with the KMS key, returns plaintext data key
3. App decrypts data locally, then discards the key
Why envelope: only the tiny data key crosses the wire to KMS, so large payloads never transit KMS (throughput + cost), and the durable KMS key stays HSM-bound.
Key type
Rotation
Policy control
Cost
AWS owned
AWS-managed, opaque
None
Free
AWS managed (aws/s3)
Automatic, annual
Cannot edit policy
No monthly fee; pay per API call
Customer managed
Optional auto (default 365 days)
Full key policy + grants
~$1/key/month + $0.03/10K requests
Imported (BYOK)
Manual only
Full control
~$1/key/month + requests
Key policy — the primary access control on a customer-managed key. Even a full IAM admin is denied unless the key policy (or a grant) allows it; the policy must enable IAM delegation via the account root.
Grants — programmatic, temporary permission delegation to a principal for specific operations, ideal for AWS services acting on your behalf without editing the key policy.
Rotation keeps old backing keys — old ciphertext still decrypts; only new encryptions use the new material. The key ARN is stable, so consumers need no change.
SSE integration — S3 (SSE-KMS), EBS, RDS, and Secrets Manager all call GenerateDataKey/Decrypt under the hood; you just point them at a key ARN.
WAF & Shield
WAF is a layer-7 firewall that inspects HTTP(S) requests before they reach your origin. You attach a Web ACL (an ordered rule list) to CloudFront, ALB, API Gateway, AppSync, or Cognito user pools. Shield handles the layer 3/4 DDoS problem WAF cannot see.
Rule type
Catches
Action options
AWS Managed Rules
Core rule set, SQLi, XSS, known bad inputs, IP reputation, bot control
Allow / Block / Count / CAPTCHA / Challenge
Rate-based
> N requests per 5-min window from one IP (brute force, L7 flood)
Block / CAPTCHA over threshold
IP set / geo match
Allow/block lists, geo-blocking by country
Allow / Block
String / regex match
Headers, URI, body, query args
Allow / Block / Count
Count mode — deploy a new rule in Count first to measure false positives before flipping it to Block. This is the safe rollout pattern interviewers want to hear.
Shield Standard — free, always on, absorbs common SYN/UDP-flood and reflection attacks at the edge.
Shield Advanced — ~$3,000/month, adds the DDoS Response Team, cost-protection credits for scale-out during an attack, and richer L7 detection alongside WAF.
Secrets Manager vs Parameter Store
Both store sensitive values encrypted with KMS and gate access with IAM. The split is rotation, cross-account, and cost — secrets that must rotate versus config that just needs to be safe.
Compliance is proving your controls work. The Well-Architected Security Pillar frames it: strong identity foundation, traceability, defense in depth, encryption everywhere, automate security, prepare for incidents. Three services provide the traceability auditors demand.
Service
Question it answers
Mechanism
CloudTrail
Who called which API, when, from where?
Immutable API audit log across all regions
Config
Is this resource configured compliantly over time?
Config snapshots + rules, drift + history
GuardDuty
Is something malicious happening right now?
ML threat detection on CloudTrail, VPC Flow, DNS logs
Synchronous request/response couples producer and consumer in latency, availability, and scale: if the downstream is slow or down, the caller blocks or fails. Messaging inserts a durable buffer so the two sides scale and fail independently. The three primitives you must distinguish cold in an interview:
Queue (SQS) — point-to-point, one message consumed by exactly one worker in a group. Pull model. Load-leveling.
Topic (SNS) — pub/sub, one message pushed to N subscribers. Push model. Fan-out.
Event bus (EventBridge) — pub/sub with content-based routing rules and a schema catalog. Push, many targets, rich filtering.
SQS: Standard vs FIFO
SQS is a fully managed pull-based queue. A producer calls SendMessage; a consumer polls ReceiveMessage, does work, then DeleteMessage. Deletion — not receipt — is what removes a message, which is the root of SQS's at-least-once semantics.
Property
Standard
FIFO (.fifo suffix)
Throughput
Nearly unlimited
300 msg/s (3,000 with batches of 10); high-throughput mode raises this per message-group
Ordering
Best-effort
Strict order within a MessageGroupId
Delivery
At-least-once (dupes possible)
Exactly-once processing
Dedup
None
5-minute dedup window by MessageDeduplicationId or content hash
Parallelism
Unbounded consumers
One in-flight message per group; use many groups to parallelize
FIFO ordering and parallelism are the same knob: the MessageGroupId. All messages sharing a group are strictly ordered and processed one at a time; different groups run concurrently. Pick the group to match your ordering domain — e.g. userId so a single user's events stay ordered while users scale out.
import boto3
sqs = boto3.client("sqs")
# FIFO send: group preserves order, dedup id suppresses retries within 5 min
sqs.send_message(
QueueUrl=url,
MessageBody='{"event":"account.closed","userId":"u-42"}',
MessageGroupId="u-42", # ordering domain
MessageDeduplicationId="close-u42-2026-07-01", # idempotency key
)
Message Lifecycle: Visibility, Polling, DLQ
When a consumer receives a message it becomes invisible for the visibility-timeout window rather than being deleted. If the consumer finishes and calls DeleteMessage in time, it's gone. If it crashes or is too slow, the timeout expires and the message reappears for redelivery. This is exactly how SQS guarantees at-least-once without losing work — and exactly why duplicate processing happens.
Producer SendMessage → message durably stored across AZs.
Consumer ReceiveMessage → message hidden for the visibility timeout; ReceiveCount increments.
Work succeeds → DeleteMessage removes it. Work fails/timeout → message becomes visible again.
When ReceiveCount > maxReceiveCount, the redrive policy moves it to a Dead-Letter Queue for inspection instead of looping forever.
Short polling — returns immediately, may return empty even when messages exist (samples a subset of servers). Default; more empty receives = more cost.
Long polling — set WaitTimeSeconds up to 20; waits for a message, cuts empty responses and API cost. Effectively always enable it.
DLQ — a separate queue set via redrive policy; alarm on its ApproximateNumberOfMessagesVisible and use redrive-to-source to replay after a fix.
SNS: Pub/Sub and Fan-Out
SNS is push-based pub/sub. Publishers send to a topic; SNS immediately replicates each message to every subscribed endpoint. There is no storage and no polling — if a subscriber is down, SNS retries per protocol and then drops (unless the subscriber is a durable target like SQS). Supported subscriber protocols:
Category
Protocols
Typical use
App-to-App (A2A)
SQS, Lambda, HTTP/S, Kinesis Data Firehose
Fan-out to workers, webhooks, buffering to durable queues
App-to-Person (A2P)
Email, Email-JSON, SMS, mobile push (APNS/FCM)
Alerts, notifications, OTP/marketing SMS
Message filtering lets each subscription attach a filter policy so a subscriber receives only the subset it cares about — matched against message attributes or (opt-in) the message body. This turns one topic into many logical streams and avoids per-consumer filtering logic.
SNS also offers FIFO topics (ordered, deduplicated) which may only fan out to SQS FIFO queues — the ordering guarantee must be preserved end to end.
The SNS + SQS Fan-Out Pattern
The canonical decoupling pattern: publish once to an SNS topic; subscribe several SQS queues, one per consumer team. SNS gives parallel delivery and filtering; SQS gives each consumer its own durable buffer, retries, and DLQ. Consumers can be added/removed without the publisher knowing. A raw HTTP or Lambda subscriber that's temporarily down loses messages; an SQS subscriber does not — the message waits in the queue.
Buffering — a traffic spike (flash sale) floods the topic; each queue absorbs it so slow consumers never drop work.
Isolation — the analytics consumer failing does not back-pressure thumbnail generation; separate queues, separate DLQs.
Access policy — the SQS queue policy must allow sns.amazonaws.com to SendMessage, scoped by the topic ARN — a classic setup gotcha.
EventBridge vs SNS
EventBridge is a serverless event bus built for event-driven routing. Like SNS it's push pub/sub, but it adds first-class routing rules that match on the full event payload, a schema registry, event archive/replay, and native ingestion from ~130 AWS services and SaaS partners (Datadog, Stripe, Shopify, Zendesk). Choose SNS when you need cheap, low-latency, high-throughput fan-out; choose EventBridge when routing logic and integrations dominate.
Dimension
SNS
EventBridge
Model
Topic fan-out
Bus + rules (content routing)
Filtering
Attributes / body match
Rich pattern match on any field of the event JSON
Targets
SQS, Lambda, HTTP, SMS, email, Firehose
20+ AWS targets (Lambda, SFN, SQS, Kinesis, API destinations...)
Sources
Your publishers
Your apps + AWS services + SaaS partners; scheduled events
SQS/SNS/EventBridge give you choreography — services react to events with no central brain. Step Functions gives you orchestration: a state machine (defined in Amazon States Language, ASL) that explicitly sequences steps, branches, retries, and handles errors, with full execution history for debugging. Use it when a workflow has multi-step business logic, human-visible state, or compensation on failure (saga).
Property
Standard workflow
Express workflow
Max duration
Up to 1 year
Up to 5 minutes
Execution semantics
Exactly-once
At-least-once (async) / at-most-once (sync)
Rate
Lower start rate; long-running
Very high volume (100k+/s)
Pricing
Per state transition
Per request + duration/memory (cheaper at high volume)
History
Full, in-console
Via CloudWatch Logs only
Fit
Order fulfillment, ETL, human approval
High-rate event processing, IoT ingest, short APIs
SQS vs Kinesis — both buffer, but SQS deletes on consume (competing consumers) while Kinesis keeps an ordered, replayable log that many consumer groups read independently by offset.
SNS vs EventBridge — both broadcast; SNS is faster/higher-throughput fan-out, EventBridge trades latency for content routing, SaaS sources, and replay.
Combine them — EventBridge/SNS at the front for routing/fan-out, SQS behind each consumer for durability and retries.
MODULE 18
AWS Observability & Governance
See and control your account: CloudWatch metrics/logs/alarms, CloudTrail, X-Ray, Config, Organizations.
The Three Pillars, Mapped to AWS
Observability is the ability to explain a system's internal state from its external outputs. Monitoring tells you when something broke; observability tells you why. Three signal types carry that weight, and each maps to a first-party AWS service.
Metrics — aggregated time-series numbers (CPU, p99 latency, error rate). Cheap to store, fast to alert on, low cardinality. Answers "what is happening?" → CloudWatch Metrics.
Logs — discrete, high-detail event records. Answers "what exactly happened at 14:32:01?" → CloudWatch Logs.
Traces — a single request's path across many services, with per-hop timing. Answers "where did the time go?" → X-Ray (or OpenTelemetry via ADOT).
A CloudWatch metric is uniquely identified by a namespace plus its dimension set. AWS services publish standard metrics for free; you publish custom ones via PutMetricData or the CloudWatch agent.
Namespace — the container, e.g. AWS/EC2, AWS/ApiGateway, or your own Custom/OrderService.
Dimensions — up to 30 name-value pairs that scope a metric, e.g. InstanceId=i-0abc. Every distinct dimension combination is a separate billable metric.
Resolution — standard (60s) is free for AWS metrics; high-resolution (1s) custom metrics cost more and enable sub-minute alarms.
Statistic — Average / Sum / Min / Max / SampleCount / percentiles (p90, p99) are computed at query and alarm time.
Retention is automatic and tiered — data is rolled up to coarser granularity as it ages, so a 1-second point does not survive as a 1-second point for a year:
Logs are organized as log groups (one per app/function, e.g. /aws/lambda/checkout) containing log streams (one per source instance/execution). Retention is set per group — from 1 day to Never — and defaults to Never, which quietly accrues cost.
Metric filters — pattern-match log events and emit a metric (e.g. count of "ERROR 500"), turning text into an alarmable number without a query.
Subscription filters — stream events in near-real-time to Lambda, Firehose (→ S3), or OpenSearch for archival or search.
Logs Insights — an interactive query language billed per GB scanned; great for ad-hoc forensics on a bounded time window.
EMF (Embedded Metric Format) — write a specially-shaped JSON log and CloudWatch auto-extracts metrics from it, so a single structured log line yields both a searchable event and a metric with no separate PutMetricData call.
fields @timestamp, @message
| filter @type = "REPORT"
| stats avg(@duration) as avg,
pct(@duration, 50) as p50,
pct(@duration, 95) as p95,
pct(@duration, 99) as p99
by bin(5m)
| sort p99 desc
EMF is the idiomatic way to emit custom app metrics from Lambda — the extraction happens asynchronously off the log, so it adds no latency to the request path:
An alarm watches one metric (or a metric-math expression) against a threshold over N evaluation periods and transitions between OK, ALARM, and INSUFFICIENT_DATA, firing actions on transition.
Actions — notify an SNS topic, trigger an Auto Scaling policy, or perform an EC2 action (stop/terminate/reboot/recover). Set both alarm_actions and ok_actions so recovery is announced too.
Composite alarms — combine child alarms with boolean logic (AND/OR/NOT) to page only on genuine incidents and suppress noise.
Alert on symptoms, not causes — "error rate > 1% for 5 min" is actionable; "CPU at 80%" often is not.
Dashboards render metric widgets, metric-math (e.g. errors/requests*100), and even live Logs Insights query results in one pane. Define them as JSON in IaC so they version alongside the app.
CloudTrail: The Audit Trail
CloudTrail records the AWS API calls made in your account — who did what, when, and from which IP. It is your control-plane audit log, not a performance monitor.
Event type
Plane
Examples
Logged by default?
Management events
Control
RunInstances, CreateBucket, CreateUser
Yes (free, one copy)
Data events
Data
s3:GetObject, lambda:Invoke
No — high volume, extra cost
Event History — free, searchable, immutable 90-day view of management events in the console; no trail required.
Trails — persist events to S3 (and optionally CloudWatch Logs). Make them multi-region so a bad actor can't hide activity in an unwatched region.
Organization trail — a single trail defined in the management account that captures every member account, so members can't disable their own auditing.
Log file integrity validation — SHA-256 hashes plus digitally-signed digest files let you prove logs weren't tampered with — key for compliance.
X-Ray: Distributed Tracing
X-Ray stitches per-service timing into an end-to-end picture of one request, and aggregates traces into a service map — a live topology graph annotated with latency and error/fault rates per node and edge.
Trace — the full request journey, keyed by a trace ID propagated in headers.
Segment — the work done by one service; subsegments capture its downstream calls (DynamoDB, an external API, S3).
Annotations — indexed key-value pairs you can filter traces by (e.g. order_id). Metadata — non-indexed detail for debugging only.
Sampling — default reservoir of 1 req/sec guaranteed plus 5% of the rest; custom rules can trace 100% of 5xx and 1% of /health.
Enable it declaratively — Lambda via tracing_config { mode = "Active" }, API Gateway via xray_tracing_enabled = true — so the trace already spans the edge before your code runs.
AWS Config: Compliance & Drift
Where CloudTrail records API calls, AWS Config records resource state over time and continuously evaluates it against rules. It answers "is this resource compliant, and how did its configuration change?"
Configuration items — point-in-time snapshots of a resource's settings and relationships; the timeline shows every change.
Config rules — managed or custom (Lambda-backed) checks, e.g. "no S3 bucket is public", "EBS volumes are encrypted", flagging resources COMPLIANT / NON_COMPLIANT.
Remediation — attach an SSM Automation document to auto-fix violations (e.g. re-enable bucket block-public-access).
Conformance packs — bundles of rules mapped to frameworks (PCI, CIS, HIPAA) deployable org-wide.
Question
Service
Who made this API call, when, from where?
CloudTrail
What is this resource's config, and is it compliant?
AWS Config
Is the system healthy / fast right now?
CloudWatch
Where did this request's latency go?
X-Ray
Organizations, SCPs & Cost Guardrails
At scale you don't run one account — you run many, grouped under AWS Organizations. This gives consolidated billing, org-wide CloudTrail/Config, and, critically, Service Control Policies (SCPs): guardrails that cap the maximum permissions any principal in a member account can have — even the root user.
SCPs are a permission ceiling, not a grant — the effective permission is the intersection of the SCP and the account's IAM policies. An SCP can deny, never allow.
Organizational Units (OUs) — group accounts (prod, dev, sandbox); attach SCPs at the OU for inherited guardrails.
Common guardrails — deny disabling CloudTrail/Config, deny leaving the org, restrict to approved regions, block public S3.
Cost/billing alarms — the EstimatedCharges metric (namespace AWS/Billing, us-east-1 only) plus AWS Budgets alert before a runaway spend becomes a surprise invoice.
FinOps on AWS: pricing models, Savings Plans vs RIs vs Spot, rightsizing, and tagging.
The Pricing Levers
The cloud is only cheaper than on-prem if you manage it. Every EC2 hour can be purchased under one of five models, trading commitment for discount. The mental model: reserve your steady-state baseline, burst the peak on On-Demand, and run anything interruptible on Spot.
Model
Commitment
Flexibility
Max discount
Best for
On-Demand
None
Maximum
0%
Spiky / unpredictable, short-term
Compute Savings Plan
$/hr, 1 or 3 yr
High — any family, region, OS, tenancy
~66%
Mixed, evolving fleets (EC2 + Fargate + Lambda)
EC2 Instance Savings Plan
$/hr, family + region
Medium — size flexible within family
72%
Stable EC2 fleets
Reserved Instance
Instance type + AZ, 1 or 3 yr
Low (Convertible RI can swap)
72%
RDS, ElastiCache, Redshift (no SP equivalent)
Spot
None — 2-min reclaim notice
Lowest
90%
Batch, CI, stateless, ML training w/ checkpoints
Dedicated Host
Per-host
Varies
BYOL value
Licensing compliance (socket-bound)
Savings Plans vs Reserved Instances
Savings Plans replaced RIs as the default commitment vehicle. Instead of reserving a specific instance, you commit to a dollars-per-hour of compute spend; AWS auto-applies the discount cheapest-first across matching usage. RIs still matter for services Savings Plans do not cover — notably RDS and ElastiCache.
Compute SP — one commitment covers EC2, Fargate, and Lambda; swap families/regions freely. Start here.
EC2 Instance SP — locks family + region for ~6% more discount; use only for large, stable fleets.
Reserved Instances — the only commit path for RDS/ElastiCache/Redshift; Convertible RIs can be exchanged.
Baseline rule — commit to ~70% of steady-state usage, never 100% of peak. Reserving over-provisioned resources locks in waste.
Usage over time
┌───────────────────────────────────
│ ╱╲ ╱╲ ╱╲
│ ╱╲ ╱ ╲ ╱ ╲ ╱ ╲ <- On-Demand / Spot (the peaks)
│ ╱ ╲╱ ╲ ╱ V ╲
│ ───────────────────────── <- Savings Plan baseline (commit here)
└────────────────────────────────────
Rightsizing with Compute Optimizer
Most teams over-provision by 2-3x. Rightsizing means matching instance type to observed utilization — and it comes before reserving, so you commit to the smaller footprint. Eliminate, then rightsize, then reserve.
Collect 2+ weeks of metrics — CPU, memory (needs CloudWatch agent), network, disk I/O.
Feed AWS Compute Optimizer (free) — it flags over/under-provisioned EC2, EBS, Lambda, ECS-on-Fargate and Auto Scaling groups with ranked recommendations.
Change one tier at a time, verify p99 latency holds, then repeat.
Prefer Graviton (m7g/c7g/r7g) and burstable T-family for low-average, spiky CPU.
Storage waste is quiet: orphaned volumes, forgotten snapshots, and hot data sitting in Standard forever. Two moves dominate — migrate EBS gp2 to gp3, and let S3 lifecycle rules age data down the cost ladder automatically.
S3 class
$/GB-mo (us-east-1)
Access pattern
Retrieval
Standard
$0.023
Frequent
Instant
Intelligent-Tiering
$0.023 + monitor fee
Unknown / mixed
Instant, auto-tiers, no retrieval fee
Standard-IA
$0.0125
Monthly (30-day min)
Instant
Glacier Instant
$0.004
Quarterly
Milliseconds
Glacier Flexible
$0.0036
Yearly
Minutes-hours
Glacier Deep Archive
$0.00099
Compliance / cold
12-48 hours
gp3 over gp2 — gp3 is ~20% cheaper per GB and decouples IOPS/throughput from size (3,000 IOPS baseline free). Migrate in place, no downtime.
Delete unattached — available-state EBS volumes, unassociated Elastic IPs ($3.65/mo each), and snapshots older than 90 days bleed money silently.
Abort multipart uploads — add a lifecycle rule to purge incomplete uploads after 7 days; they accumulate invisibly.
Data transfer is the #1 surprise line item. Compute and storage are visible; the bytes moving between them are not. Three culprits dominate: internet egress, NAT Gateway processing, and cross-AZ chatter.
Path
Cost
Note
Internet -> AWS (ingress)
Free
Uploads are free
AWS -> Internet (egress)
$0.09/GB (first 10TB)
Tiers down at scale
Same AZ
Free
Co-locate hot paths
Cross-AZ (same VPC)
$0.01/GB each way ($0.02 round-trip)
App-in-AZ-a to DB-in-AZ-b
Cross-Region
$0.02/GB
Replication, DR
NAT Gateway processing
$0.045/GB + hourly
On top of egress
S3/DynamoDB Gateway Endpoint
Free
Replaces NAT for these
CloudFront -> user
$0.085/GB (first 10TB)
AWS origin -> edge is free; < direct egress
The fix pattern: route AWS-service traffic through VPC endpoints instead of NAT, keep chatty compute-and-storage pairs in the same AZ, front public content with CloudFront, and compress everything (gzip/brotli/zstd cut 60-80% of bytes).
The Serverless Cost Model
Serverless flips the model from paying-for-provisioned to paying-per-request. That is great at low/spiky volume and can bite at massive scale, where a right-sized always-on instance is cheaper. Know where the crossover is.
Lambda — billed on GB-seconds (memory x duration) + $0.20 per 1M requests. Memory also scales CPU, so more memory can be faster and cheaper — tune with Lambda Power Tuning. Use arm64 (-20%). At 1B+ invocations/mo, requests alone are ~$200 before compute.
Fargate — pay per vCPU-second + GB-second; no idle node cost. Most tasks over-allocate 2-3x — watch Container Insights. Fargate Spot saves up to 70%; Compute SP applies here too.
Aurora Serverless v2 — scales in 0.5-ACU steps (~$0.12/ACU-hr); great for variable DB load, no paying for idle peak capacity.
Watch the exhaust — CloudWatch Logs ingestion is $0.50/GB; verbose logging with no retention policy grows unbounded.
You cannot optimize what you cannot attribute. Visibility is a stack: raw data (CUR), interactive analysis (Cost Explorer), guardrails (Budgets), and ML tripwires (Anomaly Detection). All of it is powered by cost allocation tags — untagged spend lands in an unattributable bucket.
Tool
Purpose
Cadence
Cost Explorer
Trends, forecast, group-by dimension/tag
Weekly
AWS Budgets
Threshold alerts (80/100/120%), actual + forecast
Always-on
Cost Anomaly Detection
ML alerts on unusual spend, per service/tag
Daily
CUR + Athena
Line-item chargeback, split shared costs
Monthly
Compute Optimizer / Trusted Advisor
Rightsizing + idle-resource findings
Quarterly
Enforce tags at creation with an Organizations SCP so untagged resources cannot launch, then activate the keys as cost allocation tags in the billing console.
# Minimum tag taxonomy for allocation + chargeback
Team: platform-eng # who owns it
Environment: production # prod | staging | dev
Project: recommendations # product / service
CostCenter: CC-1234 # finance billing code
Owner: jane@corp.com # accountable human
The FinOps Loop
FinOps is the operating model that keeps costs low after the one-time cleanup. It is a continuous three-phase loop owned jointly by engineering and finance — not a spreadsheet audited once a year.
Inform — visibility, allocation, showback/chargeback. Every team sees its own bill; shared costs (NAT, Transit Gateway) split proportionally.
Optimize — eliminate waste, rightsize, commit (Savings Plans/RIs), adopt Spot and new instance families (Graviton, Inferentia).
Operate — automate: budget alerts to Slack, anomaly detection, scheduled stop/start for non-prod, Config rules for untagged resources, quarterly commitment reviews.
The ordering discipline that makes it work: eliminate, then rightsize, then reserve. Reserving before rightsizing bakes in over-provisioning for the whole commitment term.
MODULE 20
AWS Reliability: DR & HA
Design for failure: Multi-AZ vs multi-region, RTO/RPO, and the four DR strategies.
HA vs DR: Two Different Problems
Interviewers punish candidates who conflate these. High Availability keeps you running through routine failures inside one region; Disaster Recovery brings you back after a region-scale catastrophe. You need both — they solve different failure classes at different cost points.
High Availability (HA) — minimize downtime during normal operations. Scope is intra-region (multi-AZ). Failover is automatic and continuous. Examples: RDS Multi-AZ, ALB across AZs, Auto Scaling. Handles instance, component, and AZ failures.
Disaster Recovery (DR) — resume operations after a major disaster. Scope is cross-region (or cross-cloud). Failover often requires a human decision or an explicit runbook trigger. Examples: S3 Cross-Region Replication, Route 53 failover, Aurora Global Database. Handles full-region outages and widespread events.
Dimension
High Availability
Disaster Recovery
Failure handled
Instance / AZ / component
Entire region, natural disaster
Fault domain
Availability Zone
Region
Replication
Synchronous
Asynchronous
Failover
Automatic (seconds)
Manual / triggered (minutes)
Cost
~2x the tier
2%–120% of prod, by strategy
Fault Domains: AZ vs Region
Every AWS reliability decision is really a bet on a fault domain. An Availability Zone is one or more discrete datacenters with independent power, cooling, and networking, linked to sibling AZs by low-latency (<1–2ms) private fiber. A Region is a fully isolated geographic boundary — separate control planes, separate blast radius.
Multi-AZ (intra-region)
Multi-Region
Replication mode
Synchronous (zero-lag)
Asynchronous (lag > 0)
Inter-node latency
Sub-millisecond to ~2ms
Tens to hundreds of ms
Achievable RPO
Zero (no data loss)
Seconds to minutes
Data transfer cost
Cheap / often free intra-AZ
Cross-region egress charged
Operational complexity
Low — one control plane
High — DNS, IAM, quotas, drift per region
Protects against
Datacenter loss
Region loss
The uptime math sets the stakes: 99.9% is 8.76 hours of downtime/year, 99.99% is 52.6 minutes, and 99.999% ("five nines") is just 5.26 minutes. Each additional nine costs disproportionately more — which is why you match the fault domain to the requirement rather than buying max redundancy everywhere.
RTO vs RPO
These two numbers drive every architecture choice. Define them first; the design follows. They sit on opposite sides of the disaster on the timeline.
RPO (Recovery Point Objective) — how much data you can afford to lose, measured backward from the disaster. "RPO = 1 minute" means at most one minute of writes vanish. It maps directly onto your replication lag.
RTO (Recovery Time Objective) — how long you can be down, measured forward from the disaster. "RTO = 15 min" means service must be restored within 15 minutes.
<-------- RPO -------->|<--------- RTO --------->
|
Last durable DISASTER Service
replicated write strikes restored
------o------------------X------------------o--------> time
| |
data you may lose downtime window
Workload
RPO target
RTO target
Banking / payments
0 (no loss)
< 15 min
E-commerce
~1 hour
~1 hour
Dev / test
24 hours
24 hours
Cold archive
1 week
1 week
The Four DR Strategies
AWS frames DR as a spectrum from cheap-and-slow to expensive-and-instant. The choice is an economic one: pay continuously for standby capacity, or pay in recovery time when disaster hits.
Strategy
What runs in DR region
RTO
RPO
Cost (% of prod)
Complexity
Backup & Restore
Nothing — only backups in S3
Hours to days
Hours
2–7%
Low
Pilot Light
DB replica + core infra, app off
10s of minutes
Minutes
7–20%
Medium
Warm Standby
Scaled-down full stack, running
Minutes
Seconds to minutes
30–60%
Medium-High
Multi-Site Active/Active
Full stack serving live traffic
Near-zero
Near-zero
80–120%
Very High
Backup & Restore — snapshots and objects replicated to another region; you rebuild everything on disaster. Cheapest, slowest. Good for dev/test and non-critical archives.
Pilot Light — a minimal always-on core (chiefly a replicating database) sits idle; on failover you scale up app servers around it. The "spark" is always lit.
Warm Standby — a fully functional but scaled-down copy runs continuously and can take real traffic immediately, then scales out. Failover is just a scale-up plus DNS flip.
Multi-Site Active/Active — both regions serve production concurrently behind a global router. There is no "failover" — you just stop sending traffic to the dead region.
Data Replication: The RPO Engine
Your DR strategy is only as good as its data layer. These are the managed replication primitives, ordered roughly by achievable RPO.
Mechanism
Mode
Typical lag / RPO
Failover model
RDS Multi-AZ
Synchronous, same region
0 (HA only)
Automatic, 1–2 min, same endpoint
RDS Multi-AZ Cluster
Semi-sync, 1 writer + 2 readable standbys
0
Automatic, ~35s
RDS cross-region read replica
Async, cross region
Seconds to minutes
Manual promote to standalone
Aurora (in-region)
6 copies across 3 AZs
0
Automatic, < 30s
Aurora Global Database
Storage-level, cross region
< 1s (typ. 100–200ms)
Managed failover < 1 min
DynamoDB Global Tables
Async, active-active multi-region
< 1s
None needed — write anywhere
S3 Cross-Region Replication
Async object replication
Minutes
Repoint reads to DR bucket
Aurora Global Database hits sub-second RPO because it does not ship the binlog — dedicated storage-layer infrastructure streams changes in parallel to the secondary region's own 6-copy volume, decoupled from the database engine's CPU. DynamoDB Global Tables goes further with active-active writes and last-writer-wins conflict resolution, so any region accepts reads and writes with no promotion step.
import boto3
def promote_rds_replica(db_instance_id):
"""Pilot-light / warm-standby failover: promote a cross-region
read replica to a standalone writable primary."""
rds = boto3.client('rds')
rds.promote_read_replica(
DBInstanceIdentifier=db_instance_id,
BackupRetentionPeriod=7,
)
rds.get_waiter('db_instance_available').wait(
DBInstanceIdentifier=db_instance_id
)
def failover_aurora_global(global_id, target_cluster_id):
"""Managed planned failover of an Aurora Global Database."""
rds = boto3.client('rds')
rds.failover_global_cluster(
GlobalClusterIdentifier=global_id,
TargetDbClusterIdentifier=target_cluster_id,
AllowDataLoss=False,
)
Health Checks & Route 53 Failover Routing
DNS is the classic cross-region failover lever. Route 53 health checks probe your primary; when they fail, the failover routing policy answers queries with the secondary's record instead. The pattern: one PRIMARY record bound to a health check, one SECONDARY record as the backstop.
resource "aws_route53_health_check" "primary" {
fqdn = "primary.example.com"
port = 443
type = "HTTPS"
resource_path = "/health" # app-level, not just TCP
failure_threshold = 3
request_interval = 10
}
resource "aws_route53_record" "failover_primary" {
zone_id = aws_route53_zone.main.zone_id
name = "api.example.com"
type = "A"
set_identifier = "primary"
failover_routing_policy { type = "PRIMARY" }
alias {
name = module.primary.alb_dns
zone_id = module.primary.alb_zone_id
evaluate_target_health = true
}
health_check_id = aws_route53_health_check.primary.id
}
resource "aws_route53_record" "failover_secondary" {
zone_id = aws_route53_zone.main.zone_id
name = "api.example.com"
type = "A"
set_identifier = "secondary"
failover_routing_policy { type = "SECONDARY" }
alias {
name = module.dr.alb_dns
zone_id = module.dr.alb_zone_id
evaluate_target_health = true
}
}
Health checks must be application-level — probe a /health path that touches the DB, not a bare TCP port. A green TCP check on a broken app is a false negative you'll only discover mid-incident.
AWS Global Accelerator is the lower-latency alternative for active-active — anycast IPs shift traffic away from an unhealthy endpoint group in seconds, with no DNS caching in the path.
Backup: AWS Backup & Snapshots
Backups are the floor of every DR strategy and the whole of Backup & Restore. AWS Backup centralizes policy across RDS, EBS, DynamoDB, EFS, and more — a single plan with lifecycle transitions and cross-region copy.
3-2-1 rule — 3 copies, on 2 media types, with 1 offsite (here, the cross-region vault copy).
Point-in-time recovery (PITR) — enable it for RDS/DynamoDB to recover to any second, not just to the last snapshot.
Immutable / vault-locked backups — WORM protection so ransomware or a rogue credential cannot delete your recovery point.
Chaos Engineering & Game Days
DR that is never exercised decays silently as infrastructure drifts from runbooks. AWS Fault Injection Simulator (FIS) turns "we think it fails over" into measured evidence, with stop conditions to cap the blast radius.
FIS experiments — terminate EC2 in a target AZ, inject network latency, fail a percentage of API calls, drain ECS tasks, throttle DynamoDB, or sever AZ connectivity.
Stop conditions — wire the experiment to a CloudWatch alarm (e.g. error-rate) so it auto-aborts before it hurts real users.
Game days — scheduled, announced drills that run an actual region failover end-to-end, with stakeholders present, measuring the RTO/RPO you truly achieve versus the target.
GPU instances, distributed training/inference, vector databases, and AI platform services.
GPU & Accelerator Instance Families
Neural nets are matrix multiplies, so throughput scales with parallel FLOPs and memory bandwidth (HBM), not clock speed. On AWS the accelerator family is chosen by job type (train vs infer), model size (drives HBM needed), and ecosystem lock-in (CUDA vs AWS Neuron). P/G run NVIDIA GPUs with full CUDA; Trn/Inf run AWS silicon compiled through the Neuron SDK.
Inf2 (Inferentia2) — inference-only ASIC, up to 4x better $/throughput on supported transformers/CNNs.
Trn1 (Trainium, 512 GB HBM) — training ASIC, ~40-50% cheaper than equivalent NVIDIA when the model compiles.
Instance
Accelerator
Accel Mem
Network
On-Demand $/hr
Sweet spot
g4dn.xlarge
1x T4
16 GB
25 Gbps
~$0.53
Cheap inference/proto
g5.xlarge
1x A10G
24 GB
10 Gbps
~$1.01
Real-time inference, fine-tune
inf2.xlarge
1x Inferentia2
32 GB
15 Gbps
~$0.76
Cost-optimized inference
trn1.32xlarge
16x Trainium
512 GB
800 Gbps
~$21.50
Cost-optimized training
p4d.24xlarge
8x A100
320 GB
400 Gbps
~$32.77
Production LLM training
p5.48xlarge
8x H100
640 GB
3,200 Gbps
~$98.32
Frontier multi-node training
When is a GPU even warranted? Classical ML (XGBoost, random forests, scikit-learn) and single-sample low-QPS inference often run cheaper on CPU/Graviton. Reach for a GPU only when the workload is a large, batched matrix multiply.
import torch
if torch.cuda.is_available():
d = torch.cuda.current_device()
print(torch.cuda.get_device_name(d)) # NVIDIA A100-SXM4-40GB
props = torch.cuda.get_device_properties(d)
print(f"{props.total_memory/1e9:.0f} GB HBM") # 40 GB
Distributed Training
Once one GPU is saturated, you scale out. The parallelism strategy depends on whether the model fits in a single GPU's HBM.
Strategy
What is split
Use when
Comm pattern
Data parallel (DDP)
The batch; each GPU holds a full model copy
Model fits in 1 GPU
AllReduce gradients (NCCL)
Tensor parallel
A single layer's matmul across GPUs
Layer too big; needs NVLink
Per-layer, very chatty
Pipeline parallel
Contiguous layer blocks per GPU
Model > 1 GPU; has bubble idle
Activations between stages
FSDP / ZeRO-3
Params + grads + optimizer sharded 1/N
Large models, avoid DP memory waste
All-gather layer on demand
Multi-node scaling lives or dies on the network. EFA (Elastic Fabric Adapter) bypasses the OS kernel (RDMA-like): AllReduce across 64 GPUs drops from ~200 ms over TCP to ~5 ms. EFA requires P4d/P5/Trn1 instances placed in a cluster placement group (same rack/switch) so NCCL rides the low-latency fabric.
# Single instance, 8 GPUs: torchrun --nproc_per_node=8 train.py
import torch.distributed as dist
from torch.nn.parallel import DistributedDataParallel as DDP
dist.init_process_group("nccl") # NCCL rides EFA between nodes
model = DDP(model.cuda(), device_ids=[local_rank])
Inference is latency- and cost-sensitive, and often always-on, so hosting choice matters more than for training. Match the serving mode to the traffic shape.
Option
Latency
Cost model
Scale-to-zero
Best for
SageMaker Real-Time
50-200 ms
per instance-hr
No (min 1)
Steady online traffic
SageMaker Serverless
+ cold start
per invoke
Yes
Bursty / low volume
SageMaker Batch Transform
minutes
lowest (job)
Yes
Offline scoring
Bedrock
200 ms+
per token
Yes (managed)
Foundation models, no ops
Inf2 (self/SM)
10-50 ms
per instance-hr
Auto
High-throughput $/req at scale
ECS/EKS + GPU
10-100 ms
per instance-hr
Manual
Full control (vLLM/TGI)
Bedrock removes GPU management entirely: pay per token for Claude, Llama, Titan, Mistral. Auto-scale a self-hosted endpoint on SageMakerVariantInvocationsPerInstance, and use production variants for canary/blue-green rollout.
RAG grounds an LLM in your data: embed documents into vectors, store them, and at query time retrieve the nearest neighbors by cosine/dot-product similarity. Brute-force search is O(n), so vector stores use ANN indexes for sub-linear latency.
HNSW — layered proximity graph, O(log n) queries, fast inserts, high RAM. Default in OpenSearch and pgvector. Tune m (edges/node) and ef_search (candidates) for recall vs speed.
IVF/IVFFlat — k-means clusters; search only the nearest nprobe lists. Lower memory, slower inserts (re-clustering). Better when memory-bound.
Dimensions — Titan Embed v2 = 1024 (flexible 256/512/1024), Cohere = 1024, OpenAI text-embedding-3-small = 1536. Cosine is the text default.
CREATE EXTENSION vector;
CREATE TABLE docs (id serial PRIMARY KEY, content text,
category text, embedding vector(1024));
CREATE INDEX ON docs USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 200);
-- metadata filter THEN nearest-neighbour; <=> is cosine distance
SELECT content, 1 - (embedding <=> :q) AS similarity
FROM docs WHERE category = 'engineering'
ORDER BY embedding <=> :q LIMIT 5;
Parallel Processing for ML Data
Preprocessing and offline scoring are embarrassingly parallel — fan out across many workers. Pick the service by task duration and whether you need GPUs.
Service
Model
GPU
Best for
AWS Batch (array jobs)
per instance-sec, one API launches 10k tasks
Yes (auto-selects)
Genomics, rendering, bulk transforms
EMR (Spark)
per instance-hr, transient clusters
Limited
TB/PB ETL, feature engineering
Step Functions Distributed Map
per transition, up to 10k concurrency
No (Lambda)
S3-inventory fan-out, <15 min tasks
Lambda + SQS
per invoke
No
Short, event-driven bursts
Batch array jobs shard by index — each container reads AWS_BATCH_JOB_ARRAY_INDEX to claim its slice. Step Functions Distributed Map reads directly from an S3 listing and batches items per Lambda.
Watch for stragglers (one slow shard stalls the job — use Spark speculative execution or SQS work-stealing), data skew (salt hot keys), and the fan-in tail (aggregation can dwarf the fan-out if unplanned).
Storage & Cost for GPU Workloads
A starved GPU is wasted money, so training storage must feed data faster than the GPUs consume it. S3 is the durable source of truth; a high-throughput cache sits between S3 and the accelerators.
On cost: Spot saves 60-90% and training is inherently retryable, so it is the default for training — with checkpoints to S3 every 15-30 min so a 2-minute interruption notice costs at most one interval. p4d.24xlarge falls from ~$32.77/hr On-Demand to ~$9.83/hr Spot; a 100-hr job goes $3,277 to ~$983. SageMaker Managed Spot handles checkpoint/resume automatically. Reserve or use Savings Plans for always-on inference instead.
Architecting RAG on AWS
The canonical GenAI system design question ties the whole module together: an ingestion path that vectorizes documents and a query path that retrieves-then-generates.
Battle-tested reference architectures and a service-selection decision matrix.
There Is No Best Architecture
Senior interviews probe judgment, not recall. AWS ships 200+ services; a real system uses fewer than 15. The signal an interviewer wants is that you derive an architecture from constraints rather than reaching for whatever is trendy. State this explicitly: there is no best architecture, only the one that is right for a specific set of constraints.
Team size — 3 engineers cannot operate EKS + service mesh + a data platform. Operational capacity is a first-class constraint, not an afterthought.
Scale profile — steady, spiky, or zero-to-hero? Spiky and scale-to-zero favor serverless; sustained high utilization favors reserved containers or EC2.
Budget — a startup optimizes for a low floor and pay-per-use; an enterprise optimizes for predictability and committed-use discounts.
Timeline — an MVP due in 6 weeks and a 5-year platform demand opposite altitudes of investment.
Domain — fintech needs ACID, audit trails, and strong isolation; social needs eventual consistency and cheap fan-out.
The interview move: restate requirements as numbers, pick the highest level of abstraction that meets them (Lambda > Fargate > EC2 > EKS in ascending operational cost), and name what would make you revisit the choice.
Reference Architectures at a Glance
Memorize seven canonical patterns. Each is a named answer you can pattern-match onto a prompt, then defend with its tradeoff. Below is the compressed map; the following sections drill into the ones interviewers push hardest on.
Pattern
Key services
When to use
Primary tradeoff
3-tier web
ALB + Auto Scaling Group (EC2/Fargate) + RDS/Aurora
Familiar CRUD app, SQL data, steady traffic, team knows servers
You own capacity & patching; scales vertically at the DB tier
Serverless
API Gateway + Lambda + DynamoDB
Spiky/unknown traffic, MVP, small team, scale-to-zero economics
The 3-tier pattern is the default request-response shape: a load balancer fronts a stateless compute tier that Auto Scaling grows and shrinks, backed by a managed relational database. It is the right answer when the data is relational, queries need JOINs and ad-hoc reporting, and the team is SQL-proficient.
Route 53 resolves to CloudFront, which caches static assets and terminates TLS at the edge.
An ALB (Layer 7) does path-based routing to an Auto Scaling Group of Fargate tasks or EC2 instances across multiple AZs.
Compute stays stateless; session and cache state live in ElastiCache Redis.
Aurora/RDS holds durable state, with Multi-AZ for HA and read replicas for read scaling.
The serverless variant collapses the compute and scaling tiers into API Gateway + Lambda and swaps the relational store for DynamoDB. It is the right answer when access patterns are known and simple, traffic is bursty or can drop to zero, and you want no servers to patch. The catch: DynamoDB rewards you only if you can draw every access pattern on a whiteboard before building. If you can't, you want RDS.
Dimension
3-tier (ALB + ASG + Aurora)
Serverless (API GW + Lambda + DynamoDB)
Data model
Relational, JOINs, ad-hoc queries
Key-value / known access patterns
Scaling
Minutes (ASG warm-up), DB scales vertically
~100ms, per-request concurrency
Idle cost
You pay for running instances
Scales to zero — pay nothing
Latency floor
~1ms warm behind ALB
~10ms warm, 100ms–several s cold
Ops burden
Capacity planning, patching (EC2)
Minimal; AWS runs the fleet
resource "aws_dynamodb_table" "orders" {
name = "orders"
billing_mode = "PAY_PER_REQUEST" # on-demand: true pay-per-use with Lambda
hash_key = "PK"
range_key = "SK"
attribute { name = "PK" type = "S" } # e.g. TENANT#123
attribute { name = "SK" type = "S" } # e.g. ORDER#2026-07-01
stream_enabled = true # fan out changes to read models / search
stream_view_type = "NEW_AND_OLD_IMAGES"
}
Serverless vs Containers vs VMs
Almost every architecture reduces to one compute decision, and interviewers love to make you defend all three legs. The heuristic: pick the highest abstraction that fits, and only descend when a hard constraint forces you to.
Criterion
Lambda (serverless)
Fargate/EKS (containers)
EC2 (VMs)
Max duration
15 min
Unlimited
Unlimited
GPU / custom kernel
No
Fargate no; EKS on EC2 yes
Yes
Scaling speed
~100ms
~30–60s
~3–5 min
Ops overhead
Minimal
Low (Fargate) / high (EKS)
High
Cost at low volume
Cheapest (scales to zero)
Moderate flat floor
Wasteful when idle
Cost at high steady volume
Expensive
Good
Cheapest (Savings Plans/Spot)
Event-Driven and Streaming
Synchronous chains are only as fast as their slowest hop and couple services tightly. Event-driven architecture inverts this: a producer emits an event and moves on; consumers react independently. The three async primitives are not interchangeable — pick by the shape of the delivery, not by habit.
SQS — a queue, one message processed once by one consumer. Standard is at-least-once with best-effort ordering and near-unlimited throughput; FIFO gives exactly-once and strict order at ~3000 msg/s. Use for decoupling and work buffering.
SNS — pub/sub fan-out, one message to many subscribers. Use for notifications; pair with an SQS queue per consumer for durability.
EventBridge — an event bus with content-based routing rules and SaaS integrations. Use when many producers route to many consumers with filters, and you want a configurable-retention replay archive.
Kinesis / MSK — an ordered, replayable stream. Use for real-time clickstream, IoT telemetry, and log aggregation where ordering (per shard/partition) and multi-consumer replay matter. Kinesis is 1 MB/s per shard; MSK is managed Kafka for teams already in that ecosystem.
The senior distinction: choose Kinesis over SQS when you need ordering, replay, AND multiple independent consumers reading the same events — for example, one real-time path plus one analytics path plus an anomaly detector, all off a single stream partitioned by user_id.
+----------------------+
Order Service ->| EventBridge bus |
+----------------------+
| | |
rule: pay | rule: stock | rule: notify
v v v
Lambda SQS SNS
(charge) (fulfil) (email/SMS)
One event, three reactions, zero coupling between them.
Data Lake and Analytics
When the requirement is "store everything cheaply and query it later," the answer is a data lake, not a warehouse. Raw data lands in S3 in its native format (schema-on-read); Glue crawls it into a catalog; Athena runs serverless SQL directly over S3. Contrast with a data warehouse (Redshift), which stores processed, structured data with schema-on-write, optimized for BI dashboards.
Ingest — Kinesis Firehose, Glue, or DMS land data in S3.
Store in medallion tiers — raw (bronze) → processed (silver) → curated (gold) prefixes, with lifecycle rules tiering old data to Glacier.
Catalog — a Glue Crawler discovers schema into the Glue Data Catalog on a schedule.
Query — Athena for ad-hoc SQL; EMR/Spark for heavy ETL; Redshift Spectrum or SageMaker for BI and ML.
Cost levers that interviewers reward: partition by date, compress, and store columnar (Parquet). Athena bills $5 per TB scanned, so a Parquet + partition-pruning layout can cut a query's cost by 10–100x versus raw JSON. Without governance (Lake Formation permissions, a catalog, naming), the lake degrades into an unqueryable swamp.
-- Athena scans only the July 1 partition, not the whole bucket
SELECT customer_id, SUM(total) AS revenue
FROM curated.orders
WHERE dt = '2026-07-01'
GROUP BY customer_id
ORDER BY revenue DESC
LIMIT 20;
The Service-Selection Decision Matrix
Interviewers give you a requirement and expect a service in one breath, then the "why." Internalize the mapping so you spend your thinking budget on tradeoffs, not recall. Read each row as requirement → pattern/service → the constraint that makes it fit.
Requirement
Pattern / service
Because
Simple API, bursty traffic
API Gateway + Lambda
Scales to zero; cheapest under ~5M req/mo
Long-running service, no GPU
Fargate (ECS)
No 15-min cap; no server management
Kubernetes ecosystem / multi-cloud
EKS + Karpenter
Portability & CNCF tooling justify the overhead
Relational data, JOINs, < 64TB
RDS/Aurora PostgreSQL
ACID, ad-hoc SQL, evolving access patterns
Key-value at massive scale
DynamoDB
Single-digit ms at any scale; known patterns
Sub-ms reads, sessions, leaderboards
ElastiCache Redis
In-memory; rich data structures
Fan-out to many consumers
SNS or EventBridge
Pub/sub vs rule-based routing
Ordered real-time stream, replay
Kinesis Data Streams
Per-shard order + multi-consumer replay
Files, images, backups
S3
11 nines durability; cheapest object store
Global static/dynamic delivery
CloudFront + S3
Edge caching cuts latency & origin cost
Multi-region active-active writes
DynamoDB Global Tables
RDS/Aurora have one write region
Foundation models / RAG
Bedrock (+ Knowledge Bases)
Managed LLMs, pay-per-token, no hosting
Well-Architected: The Six Pillars as a Lens
The AWS Well-Architected Framework is the rubric interviewers implicitly grade against. Use its six pillars as a checklist to pressure-test any design you propose; naming them signals maturity.
"Migrate this on-prem system to AWS" is a staple prompt. AWS's 6 Rs give you a vocabulary to triage each workload instead of blindly lifting-and-shifting everything.
Strategy
What it means
When
Rehost
"Lift & shift" VMs as-is to EC2
Speed matters; refactor later
Replatform
Minor tweaks (self-run DB → RDS)
Quick wins without a rewrite
Repurchase
Move to a SaaS product
Commodity function (email, CRM)
Refactor / Re-architect
Redesign cloud-native
Scaling or agility limits justify cost
Retire
Decommission it
Nobody uses it anymore
Retain
Leave it where it is (for now)
Compliance or not-yet-worth-moving
For the refactor case, the Strangler Fig pattern is the safe incremental path: put an API Gateway facade in front of the monolith, extract one bounded context at a time (start with the least-coupled domain), route its traffic to a new service, and repeat until the monolith is retired. Extract data via events (EventBridge), not a shared database, and accept temporary duplication. Timeline is months to years, not weeks — saying so is itself a maturity signal.
How EC2 actually isolates you: the Nitro system, hypervisor evolution, and hardware offload.
Hypervisor Taxonomy
Every EC2 instance, Fargate task, and Lambda invoke sits on a hypervisor. Interviewers want you to place the technology precisely: which layer runs the virtual machine monitor (VMM), and how the guest's privileged instructions get trapped.
Type-1 (bare-metal) — VMM runs directly on hardware, no host OS underneath. Xen, VMware ESXi, Hyper-V, and the Nitro hypervisor. This is what clouds run.
Type-2 (hosted) — VMM is a process on a general-purpose OS. VirtualBox, VMware Workstation, QEMU-on-your-laptop. Extra scheduling hop, so slower.
Full virtualization — guest runs unmodified; the VMM emulates real hardware (an Intel e1000 NIC, an IDE controller). Correct but slow: every device register write traps.
Paravirtualization (PV) — guest is aware it is virtualized and calls the hypervisor cooperatively (hypercalls, virtio drivers). Fewer traps, higher throughput.
Hardware-assisted (HVM) — Intel VT-x / AMD-V add a CPU mode so the guest kernel runs unmodified at near-native speed while sensitive ops trap in silicon. Modern EC2 is HVM + paravirtual I/O (virtio/ENA).
Approach
Guest modified?
Privileged-op handling
Overhead
Binary translation (old VMware)
No
Rewrite guest instructions at runtime
High
Classic paravirt (old Xen PV)
Yes (hypercalls)
Guest calls hypervisor explicitly
Medium
Hardware-assisted (VT-x/AMD-V)
No
CPU traps sensitive ops to VMM
Low (<5%)
Nitro (HVM + HW offload)
No
CPU traps + I/O offloaded to cards
~0
How a Guest Is Isolated: Rings, EPT, IOMMU
Isolation is enforced by three independent hardware mechanisms. Know all three cold — they map to the three things a guest could otherwise abuse: the CPU, memory, and DMA-capable devices.
Rings + VMX modes — the guest kernel still runs in ring 0, but inside VMX non-root mode. Sensitive instructions (CPUID, MSR/CR access, I/O) cause a VM exit into VMX root mode where the VMM handles them. State lives in the VMCS (Intel) / VMCB (AMD).
EPT / NPT (nested page tables) — two-layer translation: guest page tables map GVA→GPA (guest-controlled), then EPT maps GPA→HPA (hypervisor-controlled). The guest can never forge a host physical address.
IOMMU (VT-d / AMD-Vi) — the "EPT for devices." A NIC doing DMA is confined to the VM's memory, which is what makes safe PCI passthrough / SR-IOV possible.
Because address translation is now two-dimensional, a TLB miss is expensive — a full walk touches both page-table trees:
x86-64 nested walk (worst case, cold TLB):
4 guest levels (PML4 -> PDPT -> PD -> PT) x
4 EPT levels per guest-physical access
= up to 24 memory accesses for ONE translation
=> TLB + large pages (2MB/1GB) are critical for VM perf
# Confirm hardware virtualization + IOMMU are live on a host
grep -Eom1 'vmx|svm' /proc/cpuinfo # vmx = Intel VT-x, svm = AMD-V
dmesg | grep -e DMAR -e IOMMU # VT-d / AMD-Vi enabled
lscpu | grep -i 'Hypervisor vendor' # e.g. "KVM" inside a Nitro guest
The Xen Era vs the Nitro System
From 2006 to 2017, EC2 ran on a customized Xen hypervisor. The host CPU ran the hypervisor, a privileged Dom0 Linux, device emulation, network/storage data paths, and encryption — all stealing cycles from your instance. Nitro (launched with the C5 family, late 2017) inverted this: push everything except thin VM lifecycle management off the main CPU and onto dedicated hardware.
Dimension
Xen era (pre-2018)
Nitro system
Hypervisor
Xen + fat Dom0 Linux
Thin KVM-based Nitro hypervisor
Network/storage data path
Emulated/PV in software on host CPU
Offloaded to Nitro cards (ASIC/FPGA)
Host CPU spent on virt overhead
Up to ~30%
~1% (near bare-metal)
Bare-metal instances
Not possible
*.metal — no hypervisor at all
Root of trust / operator access
Software-mediated
Nitro Security Chip; no operator shell to guests
Management plane
Runs beside guest on host
Runs on cards, isolated from guest
The Nitro hypervisor is deliberately anemic: it does VM create/destroy, vCPU and memory assignment, and little else. There is no general-purpose Dom0 to compromise and no software device model in the hot path — the guest talks to real PCIe devices exposed by the cards.
Nitro Cards: Offloading I/O and Security
The Nitro cards are a set of purpose-built PCIe devices. Each presents itself to the guest as a standard hardware endpoint, so the guest OS uses ordinary drivers while the card does the heavy lifting in silicon.
Nitro component
Responsibility
Guest sees
Nitro Card for VPC
VPC networking, overlay encap, security groups, encryption in transit
ENA NIC
Nitro Card for EBS
Remote block storage over NVMe, at-rest encryption
NVMe disk
Nitro Card for Instance Storage
Local NVMe SSD, transparent encryption
NVMe disk
Nitro Card Controller
Coordinates cards, exposes the system to the hypervisor
(hidden)
Nitro Security Chip
Hardware root of trust, secure boot, firmware validation; locks out operator access
(hidden)
Because encryption and the storage/network data path live on the card, they cost the customer zero CPU — EBS encryption is "free," and VPC traffic is authenticated/encrypted between hosts without touching your vCPUs.
SR-IOV, ENA, and EFA
The final mile of near-bare-metal I/O is letting the guest talk to hardware queues directly, bypassing the hypervisor per-packet. That is SR-IOV.
SR-IOV — one physical NIC exposes many lightweight Virtual Functions (VFs). The IOMMU confines each VF's DMA to one VM, so a guest gets its own hardware queue with no VMM in the data path.
ENA (Elastic Network Adapter) — AWS's SR-IOV NIC driver, standard on Nitro. Scales to 100 Gbps (200 Gbps on some newer instances) with multiqueue, checksum, and segmentation offload.
EFA (Elastic Fabric Adapter) — ENA plus an OS-bypass path. Its SRD protocol lets MPI/NCCL libraries (via libfabric) write to the fabric from userspace, skipping the kernel TCP stack — the backbone of HPC and large-model GPU training clusters.
# Inspect the ENA device inside a Nitro guest
ethtool -i eth0 # driver: ena
ethtool -S eth0 | grep -e bw_ -e allowance # bandwidth/allowance exceeded counters
# EFA presence (OS-bypass fabric)
fi_info -p efa # libfabric provider "efa" => SRD available
lspci | grep -i 'Elastic Fabric'
Watch the allowance_exceeded counters: EC2 network limits are enforced by the Nitro card as token buckets, so a "slow network" is often micro-bursting past the per-instance allowance, not congestion.
Containers vs VMs vs Firecracker MicroVMs
Nitro handles the EC2 layer; serverless needs a lighter unit. AWS built Firecracker, a minimal KVM-based VMM in Rust (~50k lines vs QEMU's ~1.4M), stripping device emulation down to virtio-net, virtio-block, and a serial console. Lambda and Fargate run every tenant workload inside a Firecracker microVM.
Property
Container
Firecracker microVM
Traditional VM (QEMU)
Kernel
Shared host kernel
Own guest kernel
Own guest kernel
Boundary
namespaces + cgroups + seccomp
VT-x/EPT + tiny VMM
VT-x/EPT + large VMM
Boot / start
~100 ms
<125 ms (often <50 ms)
500 ms – several s
Memory overhead
~1–5 MB
~5 MB
~100 MB+
Emulated devices
none (n/a)
~4
100+
Density / host
1000s
1000s (thousands per node)
hundreds
Isolation strength
Weakest (shared kernel)
Strong (HW-enforced)
Strong
Firecracker's jailer wraps the VMM in a chroot, cgroups, namespaces, and a seccomp allowlist of only ~25 syscalls — so even a VMM escape lands in a stripped host sandbox. That is how AWS safely packs untrusted, multi-tenant code onto one bare-metal worker.
What Actually Runs Your Lambda
Trace one invoke top to bottom and you have the whole module in one answer. The request hits the Lambda frontend, a placement service finds or spins a microVM on a bare-metal EC2 worker, and your handler runs inside it.
Invoke arrives at the Lambda frontend (auth, throttling, routing).
Placement service finds a warm Firecracker microVM or cold-starts a new one on a bare-metal Nitro worker.
Firecracker boots a minimal Linux guest (<125 ms) with your runtime + code + deps mounted.
Handler executes; the control plane talks to it over virtio-vsock (no network stack).
MicroVM is frozen/reused for the next invoke, or reaped after idle.
Cold vs warm is just microVM lifecycle: a warm path routes to a live VM in ~1–5 ms; a cold path pays Firecracker boot plus runtime init (~200 ms and up depending on language and package size).
{
"unit_of_isolation": "Firecracker microVM (one per execution env)",
"host": "bare-metal EC2 worker on the Nitro system",
"vmm": "Firecracker (KVM-based, Rust)",
"guest_to_control_plane": "virtio-vsock",
"cold_start": "Firecracker boot (<125ms) + runtime init",
"warm_start": "route to existing microVM (~1-5ms)"
}
MODULE 24
Cheat Sheet
Pick services fast. Defaults for new projects.
Default Stack (AWS)
Compute: ECS Fargate or EKS
Data: Aurora Postgres + DynamoDB
Cache: ElastiCache Redis
Queue: SQS + EventBridge
Object: S3 + CloudFront
Auth: Cognito or external IdP
Pick LB
HTTP(S) + path/host routing → ALB
TCP/UDP, static IP, ultra-low latency → NLB
Global anycast → Global Accelerator / CloudFront
Cross-region failover → Route 53 health checks
Pick DB
Relational + transactions → Aurora / RDS Postgres
K-V, ms latency, scale-out → DynamoDB
Search → OpenSearch
Time-series → Timestream / Influx on EC2
Graph → Neptune
Wide-column → Keyspaces (Cassandra)
K8s Sanity
Always set requests; limits with care
Liveness ≠ readiness; startup probe for slow boot
PDB on every Deployment in prod
NetworkPolicy default-deny ingress
Pin image digest, not tag
Use HPA on custom metrics, not just CPU
Cost Quick Wins
S3 lifecycle: Standard → IA → Glacier
EBS gp2 → gp3 (10–20% cheaper)
VPC endpoint for S3/Dynamo to skip NAT
Compute Savings Plan @ ~60% baseline
Spot for batch/CI/dev
Set log retention < 30 days unless required
IaC Hygiene
Remote state + state lock
One module per logical unit
Plan in PR; apply on merge
No hardcoded secrets — use SM / Vault
Drift detection scheduled
Tag everything: env, owner, cost-center
MODULE 25
CI/CD Delivery & Service Mesh
The delivery layer and the mesh: GitOps, progressive delivery, and Istio/Envoy sidecars with mTLS.
The CI/CD Pipeline: Source to Deploy
A pipeline is an automated assembly line that turns a Git commit into a running workload. Each stage is a gate: a failure stops promotion, so a bad commit never reaches production. The canonical stages are source, build, test, package, and deploy — each producing an immutable artifact the next stage consumes.
Source — a push or merge to a branch fires a webhook. The runner checks out the exact commit SHA (never "latest") so the build is reproducible.
Build — compile code, resolve dependencies, produce a binary or container image. The image is tagged with the commit SHA, not latest.
Test — unit tests (fast, run first), then integration and contract tests, then security/dependency scans. Fail fast: cheap checks gate expensive ones.
Package & publish — push the image to a registry (ECR, GCR, Harbor) and other artifacts to an artifact store. This is the promotable unit.
Deploy — roll the versioned artifact into an environment (dev → staging → prod), usually with an approval gate before prod.
The core discipline is build once, promote everywhere: you build the artifact a single time and move that exact bytes-identical image through every environment. Rebuilding per environment reintroduces "works in staging, breaks in prod" drift.
An artifact is any build output you keep: a container image, a JAR, an npm tarball, a Terraform plan. The registry is the versioned store for these — a container registry for images, an artifact repository (Artifactory, Nexus) for language packages. The registry is the boundary between CI (build/test) and CD (deploy).
Immutability — once api@sha256:9f2c… is pushed, it never changes. Deployments reference the digest, so what you tested is exactly what runs.
Provenance / SBOM — a Software Bill of Materials lists every dependency in the image. Signing (cosign/Notary) lets the cluster verify the image came from your pipeline, blocking supply-chain tampering.
Retention — registries grow fast; policies garbage-collect untagged and old images. Keep the last N releases plus anything a live deployment references.
# Sign at publish time, verify at admission
cosign sign ghcr.io/acme/api@sha256:9f2c...
cosign verify --key cosign.pub ghcr.io/acme/api@sha256:9f2c...
# An admission controller rejects unsigned images cluster-wide
GitOps: The Pull Model (ArgoCD & Flux)
Traditional CD pushes: the pipeline holds cluster credentials and runs kubectl apply from outside. GitOps inverts this to a pull model: a controller running inside the cluster continuously watches a Git repo of manifests and reconciles the live state to match. Git is the single source of truth; the cluster converges to it.
Aspect
Push CD (kubectl/Helm from CI)
Pull CD (GitOps: ArgoCD/Flux)
Credentials
CI holds cluster admin keys (blast radius)
Controller lives in-cluster; no external creds
Source of truth
Whatever CI last ran
Git repo, always
Drift
Manual kubectl edit goes unnoticed
Detected and auto-reverted to Git state
Rollback
Re-run pipeline
git revert — one commit
Audit
CI logs
Git history = full change log
The flow: CI builds and pushes an image, then commits an updated image tag into the config repo (often via an automated bump). ArgoCD/Flux sees the commit, syncs, and reports Healthy/Synced. Separating the app repo (code) from the config repo (manifests) keeps a merge to main from silently mutating prod.
Deploying is not the same as releasing. Progressive delivery limits the blast radius of a bad version by exposing it to a shrinking-risk slice of traffic or users before full rollout.
Strategy
How it works
Cost / rollback
Best for
Rolling
Replace pods N at a time, old + new coexist briefly
1x capacity; rollback = roll back pods (slow)
Default, stateless, backward-compatible changes
Blue-green
Full new stack (green) beside old (blue); flip router
2x capacity; rollback = instant flip back
Fast rollback, hard schema cutovers
Canary
Route 1%→5%→25%→100% to new, watch metrics
~1.1x capacity; rollback = shift traffic back
Risky changes, gradual confidence
Feature flags
Ship code dark, toggle per-user/segment at runtime
No redeploy; rollback = flip flag
Decoupling deploy from release, A/B tests
The key insight: rolling and blue-green operate at the deploy layer (which pods run), while canary operates at the traffic layer (who gets routed), and feature flags operate at the code layer (which branch executes). Flags let you deploy at 3pm and release at midnight to 1% of users, fully decoupling the two.
Automated Rollback on SLO Breach
Manual canary watching does not scale — a human staring at Grafana at 2am is a liability. Progressive-delivery controllers (Argo Rollouts, Flagger) automate the promote/abort loop: they shift a traffic weight, pause, query the observability backend for the new version's SLIs, and compare against a threshold. Breach → automatic rollback; pass → promote to the next step.
SLI — the measured signal (error rate, p99 latency, saturation). SLO — the target the SLI must meet (99.9% success). The gate aborts when the SLI crosses the SLO-derived threshold.
Baseline vs canary — compare the new version to the current stable running right now, not a static number, to cancel out time-of-day traffic effects.
failureLimit — require several bad samples before aborting so one noisy scrape doesn't trigger a false rollback.
Service Mesh: Sidecar, Data Plane vs Control Plane
A service mesh moves cross-cutting network concerns — mTLS, retries, timeouts, routing, telemetry — out of application code and into a dedicated infrastructure layer. It does this by injecting a sidecar proxy (Envoy in Istio; a lightweight Rust micro-proxy in Linkerd) into every pod. All of the pod's traffic is transparently redirected through its proxy, so the app talks plain HTTP to localhost and the proxy handles the network.
Data plane — the fleet of sidecar proxies that actually carry request traffic. This is where mTLS, load balancing, retries, and metrics collection happen, on the hot path.
Control plane — the brain (Istio's istiod). It has no request traffic through it; it distributes config and certificates to the proxies and watches the Kubernetes API. If the control plane dies, existing proxies keep routing with their last-known config — the data plane fails static, not closed.
Sidecar injection — a mutating admission webhook adds the proxy container at pod-creation time, so developers change nothing. Newer Istio "ambient" mode replaces per-pod sidecars with a per-node proxy to cut overhead.
CONTROL PLANE (istiod): config + CA certs, no traffic
| pushes xDS config & certs
+-------------+-------------+
v v
+--------- pod A ---------+ +--------- pod B ---------+
| app -> [Envoy sidecar] =====mTLS====> [Envoy] -> app |
+------------------------+ +-------------------------+
DATA PLANE: proxies carry every request
Because every hop goes through a proxy the mesh controls, it can enforce identity and shape traffic declaratively. mTLS (mutual TLS) means both sides present certificates: the caller proves who it is and the callee proves who it is. The mesh issues short-lived per-workload certs (SPIFFE identities) and rotates them automatically, so service-to-service traffic is encrypted and authenticated with zero app code — enabling zero-trust networking inside the cluster.
Traffic shifting — send a weighted split (e.g. 90/10) across two versions; the substrate for canaries, driven entirely by config.
Mirroring (shadowing) — copy live requests to a new version and discard its responses. You test v2 with real production traffic while users only ever see v1's answers — zero user risk.
Retries & timeouts — the proxy retries idempotent failures and caps how long a call may hang, keeping one slow dependency from stalling callers.
Circuit breaking / outlier detection — eject an endpoint that returns too many 5xx or 503s so it stops receiving traffic until it recovers, preventing cascading failure.
A mesh's best "free" feature is uniform observability: because every request crosses a proxy, you get consistent golden-signal metrics (request rate, error rate, latency percentiles), a per-hop topology map, and automatic distributed-trace span propagation — for every service, in one format, without touching app code. That is genuine value. But a mesh is not free, and for many teams it is the wrong tool.
Latency & resource cost — every call now traverses two extra proxy hops; sidecars add per-pod CPU/memory. On a small fleet this overhead can outweigh the benefit.
Operational complexity — you are now running and upgrading a distributed control plane, cert authority, and CRDs. That is a platform-team commitment, not a config toggle.
Debugging surface — an extra network layer means new failure modes (mTLS handshake errors, config that didn't propagate, ejected endpoints) that are opaque to app developers.
Rule of thumb: reach for the mesh when the same networking need — encryption, retries, canary routing, tracing — recurs across enough services that solving it per-app is more expensive than running the mesh. Below that threshold, a good HTTP client library, Kubernetes-native rolling deploys, and an ingress controller cover 90% of the need at a fraction of the operational cost. Start with a single capability (often mTLS or observability) rather than adopting the whole feature surface at once.