system-design · beginner

HTTP and HTTPS — How the Web Speaks

The Central Question

Consider a user logging into a web application over an open coffee-shop Wi-Fi network.

If the application uses un-encrypted HTTP, every packet transmitted between the browser and the server — including the user's plain-text password, session cookies, and personal credit card details — can be intercepted and read by anyone running a simple packet sniffer on the same Wi-Fi router.

Furthermore, an attacker can tamper with response packets mid-transit, injecting malicious JavaScript into the webpage.

To protect network communications, systems wrap application traffic inside cryptographic security channels.

HTTP (Hypertext Transfer Protocol) is the application-layer protocol used to transmit structured requests and responses. HTTPS (HTTP Secure) is HTTP operating over a TLS (Transport Layer Security) encrypted channel, providing confidentiality, data integrity, and server authentication.

This lesson answers one central question: How do HTTP request/response mechanics and TLS 1.3 encryption handshakes secure web and API communications, and how do HTTP version evolutions (HTTP/1.1, HTTP/2, HTTP/3 QUIC) optimize network transport latency?

The Network Stack: Where HTTP and HTTPS Live

In the TCP/IP networking model, HTTP operates at the Application Layer, relying on underlying transport and internet protocols to deliver bytes:

flowchart TD
  subgraph Application Layer
    HTTP["HTTP / HTTPS (App Semantics: Methods, Status Codes, Headers)"]
  end
  subgraph Security Sub-Layer
    TLS["TLS 1.3 (Encryption, Server Cert Auth, Data Integrity)"]
  end
  subgraph Transport Layer
    TCP["TCP (Reliable In-Order Byte Stream) / QUIC (UDP)"]
  end
  subgraph Network & Link Layers
    IP["IP (Packet Routing) & Ethernet / Wi-Fi (Physical Link)"]
  end
  
  HTTP --> TLS
  TLS --> TCP
  TCP --> IP

Figure 1: Protocol layering showing HTTP wrapped inside TLS, TCP, and IP.

The Three Core Guarantees of HTTPS (TLS 1.3)

HTTPS is not a separate protocol from HTTP. It is standard HTTP traffic executed over a negotiated TLS connection. TLS provides three non-negotiable security guarantees:

flowchart LR
  subgraph Guarantee 1: Confidentiality
    Conf["Eavesdroppers see only encrypted ciphertext bytes."]
  end
  subgraph Guarantee 2: Data Integrity
    Integ["MAC checksums detect mid-transit packet tampering."]
  end
  subgraph Guarantee 3: Authentication
    Auth["X.509 PKI Certificates verify server hostname identity."]
  end

Figure 2: The three foundational security pillars of TLS 1.3.

1. Confidentiality (Encryption)

2. Data Integrity

3. Server Authentication (X.509 Certificates)

Anatomy of the TLS 1.3 Handshake

Before any encrypted HTTP request data can be sent over a new socket, the client and server execute a TLS 1.3 Handshake to authenticate the server and derive shared symmetric encryption keys.

TLS 1.3 reduced the handshake from two round trips down to a single round trip (1-RTT):

sequenceDiagram
    autonumber
    actor Client as Client Browser
    participant Server as Server (api.checkoutlab.com)
    
    Note over Client,Server: Step 1: TCP 3-Way Handshake (1 RTT)
    Client->>Server: TCP SYN
    Server-->>Client: TCP SYN-ACK
    Client->>Server: TCP ACK
    
    Note over Client,Server: Step 2: TLS 1.3 Handshake (1 RTT)
    Client->>Server: ClientHello (Supported Ciphers, Diffie-Hellman Key Share)
    Server-->>Client: ServerHello (Selected Cipher, DH Key Share, X.509 Cert, Finished)
    Note over Client: Client verifies X.509 Cert & derives Session Keys
    
    Note over Client,Server: Step 3: Encrypted HTTPS Data Exchange
    Client->>Server: GET /v1/catalog HTTP/1.1 (Encrypted Payload)
    Server-->>Client: HTTP 200 OK (Encrypted Payload)

Figure 3: Sequence diagram detailing the TCP 3-way handshake and 1-RTT TLS 1.3 key exchange.

HTTP Version Evolution: HTTP/1.1 vs. HTTP/2 vs. HTTP/3

As web applications grew from static text documents to complex single-page apps making hundreds of asset requests, the HTTP protocol evolved to reduce transport latency:

flowchart TD
  H1["HTTP/1.1 (1997)<br/>• Text-Based Protocol<br/>• 1 Request per TCP Connection<br/>• Head-of-Line (HOL) Blocking at HTTP Layer"] --> H2["HTTP/2 (2015)<br/>• Binary Framing Layer<br/>• Multiplexed Streams over 1 TCP Connection<br/>• HPACK Header Compression"]
  H2 --> H3["HTTP/3 (2022)<br/>• Built on QUIC (UDP Transport)<br/>• Zero TCP Head-of-Line Blocking<br/>• 0-RTT Connection Resumption"]

Figure 4: The historical evolution of HTTP transport performance capabilities.

Architectural Comparison of HTTP Versions

Header Compression: HPACK (HTTP/2) vs. QPACK (HTTP/3)

In HTTP/1.1, headers like User-Agent, Cookie, and Authorization are transmitted as un-compressed ASCII text with every single HTTP request. For single-page applications dispatching dozens of API calls per second, redundant headers consume thousands of bytes of bandwidth.

HTTP/2 introduced HPACK, an algorithm that compresses headers by maintaining indexed lookup tables between client and server:

flowchart LR
  subgraph HPACK (HTTP/2)
    HStatic[Static Index Table: Common headers like :method GET]
    HDynamic[Dynamic Index Table: Connection-specific headers like Bearer JWT]
    HStatic --> HStream[Single In-Order Stream]
    HDynamic --> HStream
  end
  subgraph QPACK (HTTP/3)
    QStream[Out-of-Order Streams over QUIC Datagrams] --> QEncoder[Encoder / Decoder Streams prevent HOL blocking]
  end

Figure 5: Highlighting HPACK vs QPACK header compression indexing.

Why QPACK Succeeded HPACK in HTTP/3

Mutual TLS (mTLS) for Microservice Authentication

Standard HTTPS authenticates the server to the client. In zero-trust microservice architectures, backend services use Mutual TLS (mTLS) to authenticate both sides of the connection:

sequenceDiagram
    autonumber
    actor OrderService as Order Service Pod
    participant PaymentService as Payment Service Pod
    
    OrderService->>PaymentService: 1. ClientHello + Order Service Certificate
    PaymentService->>OrderService: 2. ServerHello + Payment Service Certificate
    
    Note over OrderService: 3. Verify Payment Service Cert against Private CA
    Note over PaymentService: 4. Verify Order Service Cert against Private CA
    
    Note over OrderService,PaymentService: 5. Bidirectional Mutual Authentication Complete!<br/>Encrypted mTLS Connection Open.

Figure 6: Bidirectional mTLS certificate exchange between microservices.

Why Enterprises Deploy mTLS

Essential HTTP Security Headers

Modern HTTPS production APIs configure mandatory security headers in responses to enforce browser security policies:

HTTP/1.1 200 OK
Content-Type: application/json
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Content-Security-Policy: default-src 'self'; script-src 'self' https://cdn.checkoutlab.com
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
Set-Cookie: session_id=xyz123; Secure; HttpOnly; SameSite=Strict

Production Security Headers Matrix

Complete Worked Example: CheckoutLab End-to-End Request Journey

Let's trace the complete network journey when a client fetches catalog data from the CheckoutLab platform (https://api.checkoutlab.com/v1/catalog).

sequenceDiagram
    autonumber
    actor Client as User Browser
    participant DNS as Recursive Resolver
    participant Edge as Edge CDN / Reverse Proxy
    participant App as App Instance (172.16.1.10)
    participant DB as PostgreSQL DB
    
    Client->>DNS: 1. Resolve api.checkoutlab.com
    DNS-->>Client: 2. Return A Record 203.0.113.10
    
    Client->>Edge: 3. TCP Handshake + TLS 1.3 Key Exchange (Port 443)
    Edge-->>Client: 4. TLS 1.3 Negotiated (Symmetric Key Derived)
    
    Client->>Edge: 5. Encrypted GET /v1/catalog HTTP/2
    Note over Edge: 6. Edge Decrypts TLS<br/>7. Checks Cache & WAF Rules
    Edge->>App: 8. Forward Private HTTP GET /v1/catalog
    App->>DB: 9. SELECT * FROM products WHERE active = true
    DB-->>App: 10. Product Data Rows
    App-->>Edge: 11. HTTP 200 OK (JSON Payload)
    Note over Edge: 12. Encrypt Response with Session Key
    Edge-->>Client: 13. Encrypted HTTP 200 OK Payload

Figure 7: End-to-end request lifecycle from DNS resolution through TLS edge termination to database execution.

Failure Modes and Engineering Mitigations

What You Should Remember

1. HTTPS is HTTP over TLS: HTTPS wraps standard HTTP requests/responses inside an encrypted, authenticated TLS channel.
2. TLS 1.3 provides 3 core guarantees: Confidentiality (encryption with Forward Secrecy), Data Integrity (HMAC checksums), and Server Authentication (X.509 certificates).
3. TLS 1.3 optimizes handshake latency: TLS 1.3 completes key exchange and authentication in a single round trip (1-RTT).
4. HTTP/2 & HTTP/3 eliminate socket bottlenecks: HTTP/2 multiplexes parallel streams over 1 TCP connection; HTTP/3 uses QUIC over UDP to remove TCP head-of-line blocking.
5. mTLS authenticates microservices: Mutual TLS enforces bidirectional X.509 certificate validation between backend microservices.

Glossary of Terms

Practice Scenario and Self-Assessment

Architecture Scenario

1. Production API domain api.health.com accepts both http:// and https:// requests on port 80 and 443.
2. TLS certificates are renewed manually by an engineer every 90 days.
3. API authentication tokens are stored in un-encrypted browser local storage.

1. Identify the security vulnerabilities present in this deployment architecture.
2. Formulate a 3-step remediation plan to upgrade this system to enterprise HTTPS standards.

Interactive Self-Assessment

It eliminates TCP Head-of-Line blocking by managing independent streams over UDP datagrams.

It eliminates the need for encryption and TLS security handshakes.

It replaces binary framing with plain text payloads.

It eliminates standard HTTP status codes (such as 200 OK and 404 Not Found).

It authenticates the server's identity, proving the client is connected to the legitimate domain.

It encrypts the payload data on the physical server hard drive.

It automatically rate-limits incoming HTTP client connections.

It resolves domain names to numeric IP addresses.

What to Learn Next

• TCP vs UDP — Reliable Streams vs Low-Latency Datagrams: Explore the transport layer protocols supporting HTTP.
• DNS — Domain Name System Resolution and Caching: Revisit how domain names are resolved before HTTPS connections open.
• Proxy vs Reverse Proxy — Forwarding vs Edge Ingress: Learn how edge proxies terminate TLS for application servers.

Track: Engineering Foundations

Previous: Gossip Protocol — Epidemic Membership and State Spread

Next: IP Addresses — How Machines Find Each Other

By Shubham Jain

All articles · Study paths

Shubham Jain · Learning Lab