system-design · intermediate

Stateful vs. Stateless Architecture — Managing Session State

The Central Question

Consider an e-commerce platform running on the CoreLab platform (corelab.com):

• Architecture A (Stateful Memory): When a user logs in, the application server creates a session object in its local RAM (server_memory["session_8901"] = { user_id: 42 }). When the user clicks "Add to Cart", the HTTP load balancer routes the request to Server Node 2. Because Node 2's RAM does not contain the session object created on Node 1, the user is abruptly logged out and their shopping cart vanishes.
• Architecture B (Stateless Compute + Shared Store): When a user logs in, the application signs a cryptographic JWT token containing { user_id: 42 } or writes the session object to a shared Redis cluster. Every subsequent request carries the token or session key. Any healthy application node in the 100-node cluster handles any request instantly.

This lesson answers one central question: How do stateful and stateless architectures handle client session context, what are the trade-offs of sticky sessions versus shared state stores (Redis/JWT), and how do engineers design stateless compute tiers for linear horizontal scaling?

Core Definitions: Stateful vs. Stateless Services

System components fall into two fundamental architectural models based on how they handle session context across consecutive requests:

flowchart TB
  subgraph Stateful Architecture: Memory Dependent
    Client1[Client A: Request 1] -->|State Saved in Local RAM| Node1[Server Node 1]
    Client1 -.->|Request 2 Routed to Node 2| Node2[Server Node 2]
    Node2 --x|Session Missing!| Fail[HTTP 401 Unauthorized]
  end
subgraph Stateless Architecture: Shared State Store
    Client2[Client B: Request 1 (JWT Header)] --> Node3[Server Node 3]
    Client2 -->|Request 2 (JWT Header)| Node4[Server Node 4]
    Node3 & Node4 -->|Validate & Query Shared State| Redis[(Shared Redis / Database)]
  end

Figure 1: Architectural comparison between local memory state dependencies and shared state store designs.

1. Stateful Service

• Session Bound: Subsequent requests from the same client must route to the exact same server instance that initialized the session.
• Examples: Legacy Java Tomcat session memory, WebSockets connections bound to local server memory, embedded databases (SQLite).

2. Stateless Service

• Node Independent: Any healthy instance in the application pool can process any request from any client at any time.
• Examples: RESTful API microservices, AWS Lambda functions, Kubernetes application worker pods.

The Stateful Trap: Sticky Sessions and Failed Rolling Deploys

To make a stateful application work behind a load balancer, teams implement Sticky Sessions (Session Affinity):

sequenceDiagram
    autonumber
    actor Client as User Browser
    participant LB as Load Balancer (Cookie Sticky)
    participant NodeA as App Server Node A (Stateful)
    participant NodeB as App Server Node B (Stateful)
    
    Client->>LB: 1. POST /login
    LB->>NodeA: Route to Node A
    NodeA->>NodeA: Save Session in Local RAM (NodeA_Memory)
    NodeA-->>LB: Set Cookie: SERVERID=NodeA
    LB-->>Client: HTTP 200 OK + Cookie
    
    Client->>LB: 2. GET /cart (Cookie: SERVERID=NodeA)
    LB->>NodeA: Pin Route strictly to Node A!
    NodeA-->>Client: HTTP 200 OK (Cart Found)
    
    Note over NodeA: Node A crashes or is terminated for rolling deployment!
    
    Client->>LB: 3. GET /cart (Cookie: SERVERID=NodeA)
    LB->>NodeB: Node A Dead! Re-route to Node B
    NodeB-->>Client: HTTP 401 Unauthorized (Cart Vanished!)

Figure 2: Sequence diagram illustrating how sticky sessions fail during rolling deployments or instance crashes.

Why Sticky Sessions Break Down at Scale

1. Uneven Load Imbalance: High-traffic power users get pinned to Node A, driving Node A's CPU to 100% while Node B sits 95% idle.
2. Broken Rolling Deployments: During a continuous deployment, terminating Node A to upgrade its container drops all local sessions pinned to Node A, logging out thousands of active users.
3. In-effective Auto-Scaling: Adding 10 new nodes to an auto-scaling pool does not relieve load on existing nodes because existing users remain pinned to old nodes by sticky cookies.

The Stateless Solution: Offloading State to Shared Stores

Stateless services achieve scale by pushing state out of the compute tier into two primary target locations:

flowchart TD
  StateLoc[Where Does Session State Live?] --> ClientSide[1. Client-Side State: Signed JWTs]
  StateLoc --> SharedStore[2. Shared External Store: Redis Cluster]
  
  ClientSide --> CDesc["Cryptographically signed token carried in HTTP Authorization header.<br/>Validated 100% in RAM by any app node (Zero DB Seeks!)."]
  SharedStore --> SDesc["Centralized in-memory key-value cache (sub-ms lookups).<br/>Any app node queries Redis cluster over network."]

Figure 3: Taxonomies of stateless session offloading targets.

Pattern 1: Client-Side State via JSON Web Tokens (JWT)

package main
import (
	"context"
	"encoding/json"
	"fmt"
	"net/http"
	"strings"
	"time"
"github.com/redis/go-redis/v9"
)
type SessionData struct {
	UserID    string    json:"user_id"
	Email     string    json:"email"
	Role      string    json:"role"
	CreatedAt time.Time json:"created_at"
}
type StatelessSessionManager struct {
	redisClient *redis.Client
}
func NewSessionManager(redisAddr string) *StatelessSessionManager {
	rdb := redis.NewClient(&redis.Options{
		Addr:     redisAddr,
		Password: "", // No password for dev
		DB:       0,
	})
	return &StatelessSessionManager{redisClient: rdb}
}
// ServeHTTP handles requests statelessly across any node in the pool
func (m StatelessSessionManager) AuthenticateMiddleware(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, r http.Request) {
		ctx, cancel := context.WithTimeout(r.Context(), 2*time.Second)
		defer cancel()
sessionID := r.Header.Get("X-Session-ID")
		if sessionID == "" {
			http.Error(w, "Missing X-Session-ID Header", http.StatusUnauthorized)
			return
		}
// Fetch session state from SHARED Redis Store (Any node can query this!)
		val, err := m.redisClient.Get(ctx, "session:"+sessionID).Result()
		if err == redis.Nil {
			http.Error(w, "Session Expired or Invalid", http.StatusUnauthorized)
			return
		} else if err != nil {
			http.Error(w, "Shared Session Store Error", http.StatusInternalServerError)
			return
		}
var session SessionData
		json.Unmarshal([]byte(val), &session)
// Inject stateless session into context for downstream business logic
		reqWithCtx := r.WithContext(context.WithValue(r.Context(), "user_session", session))
		next.ServeHTTP(w, reqWithCtx)
	})
}