MyClothes.com - Stateful Web App

Overview

In the previous case study (WhatIsTheTime.com), we built a stateless application — the server didn't need to remember anything between requests. Now we're stepping into a much more realistic and challenging scenario: a stateful e-commerce web application.

MyClothes.com allows people to buy clothes online. It has a shopping cart, and hundreds of users are navigating the website simultaneously. The fundamental challenge is: how do we maintain horizontal scalability (many instances behind a load balancer) while preserving user state (the shopping cart)?

Key requirements:

• Users should never lose their shopping cart while navigating
• The web tier should remain as stateless as possible for easy scaling
• User details (address, name, etc.) should be stored in a durable database
• The architecture must scale reads efficiently
• Everything should survive AZ failures

Starting Point: The Problem

We start with the same architecture from WhatIsTheTime.com — Route 53, Multi-AZ ELB, Auto Scaling Group across three AZs. This worked great for a stateless app, but now we have a problem.

┌──────────┐    ┌─────┐    ┌────────────────────────────────┐
│          │    │     │    │  Instance A  ← Cart created    │
│   User   │─1─▶│ ELB │───▶│                                │
│          │─2─▶│     │───▶│  Instance B  ← Cart LOST!      │
│          │─3─▶│     │───▶│  Instance C  ← Cart LOST!      │
└──────────┘    └─────┘    └────────────────────────────────┘

The user adds something to their shopping cart on Instance A. The next request gets routed to Instance B — which has no knowledge of the cart. The user adds something again, gets routed to Instance C — cart lost again. The user is going crazy: "I keep losing my shopping cart! MyClothes.com is a terrible website!" And we're losing money.

This happens because the ELB distributes requests across instances, and each instance stores the cart in local memory. There's no shared state.

Solution 1: ELB Stickiness (Session Affinity)

The simplest fix: enable ELB Stickiness (also called session affinity). This tells the ELB to route all requests from the same user to the same instance.

┌──────────┐    ┌──────────────────┐    ┌────────────────────────────────┐
│          │    │       ELB        │    │                                │
│   User   │─1─▶│   Stickiness    │───▶│  Instance A  ← All requests   │
│          │─2─▶│    ENABLED      │───▶│  Instance A  ← go to same     │
│          │─3─▶│                  │───▶│  Instance A  ← instance       │
└──────────┘    └──────────────────┘    └────────────────────────────────┘

The ELB sets a cookie on the user's browser, and every subsequent request from that user goes to the same instance. The shopping cart is preserved!

This is a definite improvement, but it's fragile. If the ASG terminates an instance (during scale-in, or if it fails a health check), all users stuck to that instance lose their carts.

Solution 2: User Cookies (Client-Side State)

A completely different approach: instead of storing the cart on the server, let the user store it. The shopping cart content is sent as a web cookie with every HTTP request.

┌──────────────────────────┐    ┌─────┐    ┌────────────────────────────┐
│   User                   │    │     │    │  Instance A                │
│   Cookie: cart=          │───▶│ ELB │───▶│  Reads cart from cookie    │
│   {item1, item2, item3}  │───▶│     │───▶│  Instance B                │
│                          │───▶│     │───▶│  Reads cart from cookie    │
│                          │    │     │    │  Instance C                │
└──────────────────────────┘    └─────┘    │  Reads cart from cookie    │
                                           └────────────────────────────┘

Every time the user makes a request, they send the entire shopping cart content in the cookie. Any instance can read it — we've achieved true statelessness in the web tier. Each EC2 instance doesn't need to know what happened before; the user tells us.

This is actually a pattern that many web application frameworks use in practice, but the 4 KB limit and security concerns make it impractical for complex shopping carts.

Solution 3: Server Sessions with ElastiCache (Best Approach)

The best of both worlds: the user sends only a session ID (a small token) in the cookie, and the actual cart data is stored server-side in ElastiCache.

┌──────────────────────┐    ┌─────┐    ┌──────────┐    ┌──────────────────┐
│   User               │    │     │    │          │    │                  │
│   Cookie:            │───▶│ ELB │───▶│ Inst. A  │───▶│   ElastiCache    │
│   session_id=abc123  │    │     │    │          │    │                  │
│                      │    │     │    │ Inst. B  │───▶│   session_id →   │
│                      │    │     │    │          │    │   cart data      │
│                      │    │     │    │ Inst. C  │───▶│                  │
└──────────────────────┘    └─────┘    └──────────┘    │   Sub-ms perf   │
                                                        └──────────────────┘

Here's how it works:

1. User makes first request → EC2 instance creates a session, stores cart data in ElastiCache with a session ID as the key
2. EC2 instance sends the session ID back to the user as a cookie
3. User makes second request (to a different instance) → sends session ID in cookie
4. The new instance uses the session ID to look up the cart from ElastiCache
5. Cart data is retrieved in sub-millisecond time — the user never notices

This is the recommended pattern for stateful web applications. It's more secure than cookies, more resilient than stickiness, and keeps the web tier stateless.

Adding a Database Layer: RDS for User Data

Now that we've solved the shopping cart problem, we need to store durable user data — things like shipping addresses, names, order history. This data needs to persist long-term, not just for a session.

┌──────────────┐    ┌──────────────────────┐
│  Instance A  │───▶│                      │
│  Instance B  │───▶│    RDS (MySQL)       │
│  Instance C  │───▶│                      │
└──────────────┘    │  User addresses      │
                    │  Names, profiles     │
                    │  Order history       │
                    └──────────────────────┘

All EC2 instances can read from and write to the same RDS database. This gives us a centralized, durable data store that persists regardless of which instance handles the request.

Scaling Reads: Two Approaches

Our website is doing amazing — more and more users are browsing products, reading descriptions, viewing images. Most of the traffic is reads (browsing), not writes (purchasing). How do we scale the read capacity?

Approach A: RDS Read Replicas

┌──────────────┐         ┌──────────────────┐
│              │──WRITE──▶│   RDS Master     │
│  EC2         │         └────────┬─────────┘
│  Instances   │                  │ Replication
│              │──READ───▶┌──────▼──────────┐
│              │──READ───▶│  Read Replica 1 │
│              │──READ───▶│  Read Replica 2 │
│              │          │  ...up to 15    │
└──────────────┘          └─────────────────┘

Writes go to the RDS Master. Reads go to Read Replicas (up to 15). Replication happens asynchronously. This distributes the read load across multiple database instances.

Approach B: Lazy Loading with ElastiCache

┌──────────────┐    ┌──────────────────┐    ┌─────────────┐
│              │─1─▶│  ElastiCache     │    │             │
│  EC2         │    │  Cache MISS      │    │    RDS      │
│  Instance    │─2─▶│─────────────────▶│───▶│             │
│              │    │  Read from RDS   │    │             │
│              │─3─▶│  Write to cache  │    │             │
│              │    │                  │    │             │
│              │─4─▶│  Cache HIT! ✅   │    │             │
│              │    │  Return instantly │    │             │
└──────────────┘    └──────────────────┘    └─────────────┘

This is called lazy loading (or cache-aside pattern):

1. EC2 instance checks ElastiCache first: "Do you have this product data?"
2. Cache miss: ElastiCache doesn't have it → read from RDS → write the result to ElastiCache
3. Cache hit: ElastiCache has it → return immediately (sub-millisecond)
4. Subsequent requests for the same data get cache hits — no RDS query needed

This dramatically reduces RDS CPU usage and improves response times. But it requires application-side logic to manage the cache (what to cache, when to invalidate, TTL settings).

The Complete Multi-AZ Architecture

┌──────────┐   ┌──────────┐   ┌───────────────────┐   ┌──────────────────────────┐
│          │   │ Route 53 │   │    Multi-AZ ELB   │   │         ASG              │
│  Users   │──▶│  Alias   │──▶│  + Health Checks  │──▶│  AZ1: EC2  EC2          │
│          │   │          │   │                   │   │  AZ2: EC2  EC2          │
└──────────┘   └──────────┘   └───────────────────┘   │  AZ3: EC2              │
                                                        └───────────┬──────────────┘
                                                                    │
                              ┌─────────────────────────────────────┼──────────────────────┐
                              ▼                                     ▼                      ▼
                    ┌──────────────────┐                  ┌──────────────────┐   ┌──────────────────┐
                    │   ElastiCache    │                  │   RDS Master     │   │   RDS Standby    │
                    │   Multi-AZ       │                  │                  │   │   (Multi-AZ)     │
                    │   (Redis)        │                  │                  │   │   Auto-failover  │
                    │                  │                  └──────────────────┘   └──────────────────┘
                    │   Sessions +     │
                    │   Read Cache     │
                    └──────────────────┘

Every layer is now Multi-AZ:

• Route 53: Already highly available by default (100% SLA)
• ELB: Spans AZ1, AZ2, AZ3 with health checks
• ASG: Distributes EC2 instances across all AZs
• RDS: Multi-AZ with automatic failover (standby replica in another AZ)
• ElastiCache: Multi-AZ with Redis (automatic failover if primary node fails)

Security Group Chain

This is a critical security pattern — each layer only accepts traffic from the layer directly above it:

Internet
    │
    ▼
┌──────────────────────────────────────────┐
│  ALB Security Group                      │
│  Inbound: HTTP/HTTPS from 0.0.0.0/0     │  ← Anyone on the internet
└──────────────────┬───────────────────────┘
                   │
                   ▼
┌──────────────────────────────────────────┐
│  EC2 Security Group                      │
│  Inbound: from ALB Security Group ONLY   │  ← Only the load balancer
└──────────────────┬───────────────────────┘
                   │
          ┌────────┴────────┐
          ▼                 ▼
┌─────────────────┐  ┌─────────────────┐
│ ElastiCache SG  │  │    RDS SG       │
│ Inbound: from   │  │ Inbound: from   │
│ EC2 SG ONLY     │  │ EC2 SG ONLY     │
└─────────────────┘  └─────────────────┘

What We Learned — The 3-Tier Architecture

This case study introduced the classic 3-tier web architecture:

Questions & Answers