system-design · intermediate

API Gateway — Edge Entry for Microservices

The Central Question

Consider a microservices platform running on the APILab platform (apilab.com) consisting of 30 independent backend services (order-service, user-service, catalog-service, payment-service, shipping-service).

If public mobile and web applications connected directly to individual internal microservices:

• Clients would need to manage 30 distinct IP hostnames (orders.internal:8080, users.internal:8081).
• Every single microservice would be forced to independently implement HTTPS certificates, JWT token authentication, rate limiting counters, and Web Application Firewall (WAF) rule sets.
• Exposing 30 internal microservice network addresses directly to the public internet creates a massive attack surface.

An API Gateway is a specialized edge reverse proxy that serves as the single public entry point for external client traffic, routing incoming requests to internal microservices while centralizing cross-cutting platform concerns.

This lesson answers one central question: How does an API Gateway act as an edge reverse proxy to centralize cross-cutting concerns (authentication, TLS termination, rate limiting, routing, BFF aggregation) across microservice architectures without becoming a performance bottleneck or single point of failure?

Direct Client-to-Service Access vs. API Gateway Architecture

Deploying a unified gateway facade alters the edge boundary of a distributed architecture:

flowchart TB
  subgraph Anti-Pattern: Direct Client Access to Microservices
    C1[Mobile App] -->|HTTPS| S1[Order Service 172.16.1.10]
    C1 -->|HTTPS| S2[User Service 172.16.1.11]
    C1 -->|HTTPS| S3[Catalog Service 172.16.1.12]
    Note1["Drawbacks: 30 Public Hostnames, Duplicate Auth/TLS Code, Exposed VPC Topologies"]
  end
  
  subgraph Pattern: API Gateway Edge Facade
    C2[Mobile App] -->|Single Host: api.apilab.com| GW[API Gateway Pool]
    GW -->|Inspect Path & Auth Header| GWAuth[JWT Authentication & Rate Limits]
    GWAuth -->|Private HTTP| ServiceA[Order Service 172.16.1.10]
    GWAuth -->|Private HTTP| ServiceB[User Service 172.16.1.11]
    GWAuth -->|Private HTTP| ServiceC[Catalog Service 172.16.1.12]
  end

Figure 1: Comparing direct client-to-service access against a centralized API Gateway edge topology.

Architectural Benefits of an API Gateway

1. Single Entry Point: Clients connect to one domain (api.apilab.com), decoupling public client routing from internal microservice refactoring.
2. Centralized Cross-Cutting Concerns: Offloads authentication, TLS decryption, rate limiting, and CORS headers away from feature development teams.
3. Security Boundary Insulation: Hides private VPC IP addresses and internal port configurations from the public internet.

The 7-Stage Gateway Request Pipeline

When an HTTP request hits the API Gateway, it passes through a multi-stage execution pipeline before reaching the target upstream service:

flowchart LR
  Req[Public Client HTTP Request] --> Stage1[1. TLS Decryption]
  Stage1 --> Stage2[2. WAF & IP Filtering]
  Stage2 --> Stage3[3. Rate Limit Verification]
  Stage3 --> Stage4[4. JWT Token Auth Validation]
  Stage4 --> Stage5[5. Header Sanitization & Injection]
  Stage5 --> Stage6[6. Path-Based Upstream Routing]
  Stage6 --> Stage7[7. Private Network Forward]
  Stage7 --> Upstream[(Upstream Service)]

Figure 2: The sequential 7-stage processing pipeline inside an API Gateway.

Detailed Execution Stages

1. TLS Decryption: Terminates HTTPS encryption at the edge, converting public TLS traffic to un-encrypted HTTP (or internal mTLS) for high-speed VPC routing.
2. WAF & IP Filtering: Evaluates Web Application Firewall rules, blocking malicious SQL injection patterns and blacklisted IP addresses.
3. Rate Limit Verification: Checks client API keys or IP addresses against Redis token bucket counters. Returns HTTP 429 if quota is exceeded.
4. JWT Authentication & Validation: Validates client OAuth2/JWT signatures at the edge. If the token is expired or invalid, rejects the request with HTTP 401 Unauthorized.
5. Header Sanitization & Injection: Strips untrusted client headers (X-Forwarded-For, X-User-Id) and injects verified internal headers (X-Authenticated-User-Id: 9012, X-Tenant-Id: acme_corp).
6. Path-Based Upstream Routing: Matches URL patterns (/v1/orders/* $\rightarrow$ order-service-cluster).
7. Private Forwarding & Load Balancing: Forwards the sanitized HTTP request to a healthy upstream instance in the target private subnet.

Core Capabilities of Modern API Gateways

flowchart TD
  Gateway[API Gateway Platform Capabilities] --> C1[1. Edge Traffic Control]
  Gateway --> C2[2. Security & Identity]
  Gateway --> C3[3. Request Transformation & BFF]
  Gateway --> C4[4. Observability & Telemetry]
  
  C1 --> C1a[Path/Host Routing, Canary Releases, Rate Limits]
  C2 --> C2a[TLS Termination, JWT Auth, OAuth2 Token Exchange]
  C3 --> C3a[Backend-for-Frontend Aggregation, JSON Translation]
  C4 --> C4a[Distributed Trace Injection, Latency Telemetry]

Figure 3: Taxonomical breakdown of core API Gateway platform features.

1. Edge Traffic Control & Canary Releases

2. Security and Identity Header Transformation

3. Backend-for-Frontend (BFF) Pattern & Aggregation

Without a gateway, the mobile app makes 3 separate network calls. With a BFF Gateway, the mobile app issues a single GET /v1/mobile/dashboard call; the gateway calls all 3 internal services in parallel, aggregates the JSON responses, and returns a single optimized payload.

sequenceDiagram
    autonumber
    actor Mobile as Mobile App
    participant BFF as Gateway (BFF Tier)
    participant User as User Service
    participant Order as Order Service
    participant Notif as Notification Service
    
    Mobile->>BFF: GET /v1/mobile/dashboard
    par Parallel Fan-Out
        BFF->>User: GET /internal/users/me
        BFF->>Order: GET /internal/orders/recent
        BFF->>Notif: GET /internal/notifications/unread
    end
    User-->>BFF: 200 OK { userProfile }
    Order-->>BFF: 200 OK { recentOrders }
    Notif-->>BFF: 200 OK { unreadCount: 3 }
    Note over BFF: Aggregate & trim un-needed JSON fields
    BFF-->>Mobile: HTTP 200 OK { userProfile, recentOrders, unreadCount }

Figure 4: Sequence diagram illustrating parallel fan-out request aggregation inside a BFF Gateway.

Architectural Comparison: Load Balancer vs. API Gateway vs. Service Mesh

Engineers often confuse API Gateways with Load Balancers and Service Meshes. They operate at different architectural boundaries:

flowchart TB
  subgraph Public Internet
    Client[Client Browser]
  end
  subgraph Public Edge
    LB[Layer 4 Load Balancer] --> GW[Layer 7 API Gateway]
  end
  subgraph Private Mesh Network
    GW --> PodA[Order Service Pod (Envoy Sidecar)]
    PodA <-->|Service-to-Service mTLS| PodB[Payment Service Pod (Envoy Sidecar)]
  end

Figure 5: Network topology illustrating the relationship between Edge Load Balancer, API Gateway, and Service Mesh.

Gateway to Service Mesh Handoff

Complete Worked Example: Go Edge API Gateway Router

Let's inspect a complete Go implementation of an edge API Gateway router for the APILab platform (apilab.com) implementing JWT authentication, header sanitization, and path-based upstream routing.

package main
import (
	"fmt"
	"net/http"
	"net/http/httputil"
	"net/url"
	"strings"
)
type APIGateway struct {
	orderServiceURL   url.URL
	catalogServiceURL url.URL
}
func NewAPIGateway(orderURL, catalogURL string) *APIGateway {
	oURL, _ := url.Parse(orderURL)
	cURL, _ := url.Parse(catalogURL)
	return &APIGateway{
		orderServiceURL:   oURL,
		catalogServiceURL: cURL,
	}
}
func (gw APIGateway) ServeHTTP(w http.ResponseWriter, r http.Request) {
	// 1. Header Sanitization: Strip dangerous untrusted client identity headers
	r.Header.Del("X-Authenticated-User-Id")
	r.Header.Del("X-User-Roles")
// 2. JWT Authentication Validation Stage (Simulated)
	authHeader := r.Header.Get("Authorization")
	if !strings.HasPrefix(authHeader, "Bearer ") && r.URL.Path != "/v1/catalog" {
		http.Error(w, {"error": "Unauthorized"}, http.StatusUnauthorized)
		return
	}
// Inject Verified Internal Headers
	r.Header.Set("X-Authenticated-User-Id", "usr_9012")
	r.Header.Set("X-User-Roles", "customer")
// 3. Path-Based Upstream Routing Stage
	switch {
	case strings.HasPrefix(r.URL.Path, "/v1/orders"):
		fmt.Println("[GATEWAY ROUTER] Forwarding request to Order Service Cluster...")
		proxy := httputil.NewSingleHostReverseProxy(gw.orderServiceURL)
		proxy.ServeHTTP(w, r)
case strings.HasPrefix(r.URL.Path, "/v1/catalog"):
		fmt.Println("[GATEWAY ROUTER] Forwarding request to Catalog Service Cluster...")
		proxy := httputil.NewSingleHostReverseProxy(gw.catalogServiceURL)
		proxy.ServeHTTP(w, r)
default:
		http.Error(w, {"error": "Route Not Found"}, http.StatusNotFound)
	}
}

Failure Modes and Engineering Mitigations

What You Should Remember

1. API Gateways unify edge entry: A gateway provides a single public domain for clients, insulating public traffic from internal microservice restructuring.
2. Centralize cross-cutting concerns: Offload TLS termination, OAuth2/JWT validation, rate limiting, and CORS enforcement to the gateway tier.
3. Keep the gateway thin: Avoid putting business logic or SQL queries into the gateway. Thick gateways become fragile, un-maintainable monoliths.
4. Sanitize internal identity headers: Validate JWT tokens at the edge, strip client-supplied identity headers, and inject verified X-User-Id headers for upstream consumption.
5. Align timeout budgets: Gateway timeouts must be shorter than client timeouts and longer than upstream service execution goals.

Glossary of Terms

Practice Scenario and Self-Assessment

Architecture Scenario

1. Patient Mobile App (Requires fast JSON response aggregation)
2. Medical Records API (Strict HIPAA compliance, high security)
3. Billing Service (Slow processing, requires strict rate limiting)

1. Draw the edge request pipeline for POST /v1/billing/invoices, detailing where authentication, rate limiting, and routing occur.
2. Explain how a BFF Gateway pattern improves user experience for the Patient Mobile App when loading the medical dashboard.

Interactive Self-Assessment

It creates a fragile monolith at the edge that couples microservice teams and risks cascading outages.

API Gateways are physically incapable of executing code plugins.

Database drivers cannot run over TCP network connections.

Kubernetes prevents API Gateways from executing custom code plugins.

Unconditionally strip untrusted identity headers and inject verified internal headers.

Forward all incoming client HTTP headers directly without inspection.

Strip all Content-Type headers from request payloads.

Encrypt the HTTP header names using AES-256 encryption.

What to Learn Next

• REST vs RPC — Wire Protocol Trade-offs: Compare REST JSON vs gRPC Protobuf binary communication.
• REST vs GraphQL — Query Flexibility vs Over-Fetching: Explore GraphQL query flexibilities and DataLoader mechanics.
• Load Balancing — Algorithms and Layers: Revisit reverse proxy load balancing at Layer 4 and Layer 7.

Track: Software Design and Architecture

Next: Batch vs Stream Processing — When to Wait, When to Flow

By Shubham Jain

All articles · Study paths

Shubham Jain · Learning Lab