system-design · beginner

Long Polling vs WebSockets — Realtime Over HTTP

Start here

Users want updates now: chat messages, live scores, collaborative cursors, order status.

Two common approaches over the web:

1. Long polling — client asks the server, and the server waits to answer until something happens (or a timeout). Client immediately asks again.
2. WebSockets — client and server upgrade the connection to a long-lived two-way channel and push frames either direction.

What you will learn

1. Define short polling, long polling, and WebSockets.
2. Compare latency, overhead, and complexity.
3. See how proxies and load balancers affect each.
4. Work a complete chat notification example.
5. Know failure modes: thundering reconnects, sticky sessions.
6. Choose a default for common product stages.

What you should know first

Words you need before we begin

Simple story: waiting at a bakery counter

• Short polling: walk up every 10 seconds “ready yet?”
• Long polling: stand at the counter until your order is ready or you give up after 30 minutes, then line up again.
• WebSocket: leave an open intercom so the baker calls you and you can ask questions anytime.

The problem with short polling only

Client requests every 2 seconds × 100,000 users = enormous QPS mostly saying “no news.” Latency is up to the poll interval. Battery and server CPU suffer.

Step-by-step: long polling

1. Client GET /events?since=cursor with long timeout.
2. Server parks the request (async) until event or 25–55s timeout.
3. Response returns events or empty; client reconnects immediately.
4. Use cursors so reconnects do not miss or double-deliver.

Step-by-step: WebSockets

1. Client connects with Connection: Upgrade and Upgrade: websocket.
2. Server accepts; connection stays open.
3. Either side sends frames (text/binary).
4. Heartbeats detect half-open sockets.
5. On drop, client reconnects with backoff and resumes from cursor/sequence.

Visual mental model

sequenceDiagram
  participant C as Client
  participant S as Server
  Note over C,S: Long polling
  C->>S: GET hold
  S-->>C: event or timeout
  C->>S: GET hold again
  Note over C,S: WebSocket
  C->>S: upgrade
  S-->>C: 101
  S-->>C: push frames
  C->>S: client frames

Learning question: Which approach needs more care with reverse proxies idle timeouts?

Caption: Both can work; WebSockets and long holds need timeout alignment end-to-end.

Complete worked example: order status for shoppers

Starting situation

Shoppers watch “preparing → out for delivery → delivered” live. Peak 50k concurrent watchers. Events are infrequent per user.

Constraints

• Mobile clients behind carrier NATs
• Existing HTTP API and CDN
• Team has limited WebSocket ops experience

Decisions

Long poll timeout 30s; CDN/proxy idle timeout raised above that; cursor = status version.

Failure behavior

• Proxy closes at 15s: client sees errors; align timeouts.
• Thundering herd on deploy: reconnect jitter.
• Multi-instance without pub/sub: event on instance A while long poll on B—need shared bus.

Outcome

MVP ships with long polling; add WebSockets when concurrent chattiness justifies ops cost.

How it works in production

• API gateways with WebSocket routes
• SSE for one-way feeds (dashboards)
• Message bus (Redis pub/sub, Kafka) to fan-out events to the instance holding the connection
• Metrics: active connections, reconnect rate, hold times

Failure modes

Trade-offs

Compare with related concepts

Common misunderstandings

1. “WebSockets are always better.” Not if you lack ops readiness and events are rare.
2. “Long polling is obsolete.” Still widely used and proxy-friendly.
3. “HTTP cannot do realtime.” Long poll and SSE exist.
4. “One connection per user forever is free.” Memory and file descriptors cost money.
5. “Reconnect will just work.” Without backoff and resume tokens, you melt the edge.

Check your understanding

1. How does long polling differ from short polling?
2. What does a WebSocket upgrade do?
3. Why do multi-instance deploys need a shared event bus?
4. Name one reason to pick long polling for an MVP.
5. What is a reconnect storm?

Practice

1. Design cursors for a chat long poll API.
2. List proxy settings to review for WebSockets.
3. Choose transport for live sports scores vs tax form auto-save.
4. Sketch reconnect with exponential backoff and jitter.
5. Estimate connection memory if each WS uses 10 KB server-side × 200k users.

Deeper production notes

Operational readiness

Before enabling this mechanism on a critical path, document detection signals, mitigation steps, and rollback. If on-call cannot answer "what do I click first?" at 3 a.m., the design is incomplete.

Measure user journeys alongside internal counters. Healthy-looking internals with broken customer flows mean the wrong dashboard is green.

Failure injection

In staging, inject latency, process kills, and partial network loss. Confirm the system degrades in the way the lesson describes—fail fast, elect, quarantine, or reject—not in a surprising new way.

Change management

Configuration for timeouts, thresholds, and fan-out is as dangerous as code. Review config diffs like code diffs. Gradual rollout and feature flags reduce blast radius when defaults are wrong.

Revision summary

• Long polling holds HTTP until events or timeout.
• WebSockets provide persistent bidirectional channels.
• Align timeouts, plan reconnects, fan-out via a bus.
• Match choice to event rate and operational maturity.

Glossary

Abbreviations and terminology

• SSE — Server-Sent Events
• WS — WebSocket
• QPS — Queries per second
• NAT — Network Address Translation

What to learn next

1. WebSockets
2. Push vs pull
3. Load balancing
4. Pub/sub

Additional teaching scenarios

Scenario A — busy day

Imagine traffic multiplies by ten for a marketing event. Re-read the failure modes section and mark which ones become likely first. Write the first mitigation you would take for each marked item. This exercise turns abstract lists into operational instincts.

Scenario B — partial deploy

Half of your instances run the new version and half run the old version. Which assumptions in this lesson break if the two versions disagree about protocols, message fields, or transaction boundaries? Prefer designs that tolerate mixed versions for at least one deploy window.

Scenario C — explain to a new teammate

In five sentences, teach the core idea of this lesson without acronyms. If you cannot, the mental model is not yet solid—revisit the simple story and worked example until the five sentences feel natural.

Scenario D — metric design

List three metrics and one alert threshold you would ship with this mechanism. Good metrics name the user impact or the resource that runs out, not only that a counter incremented.

Scenario E — deliberate non-goals

Write two problems this lesson's technique should not solve. Explicit non-goals prevent cargo-cult adoption where every service gets the same machinery whether it needs it or not.

Scenario F — capacity napkin math

Estimate requests or messages per second at peak, multiply by payload size, and ask whether your chosen design still holds. Napkin math catches fantasy architectures before they meet production invoices.

Scenario G — ownership checklist

Name the team that owns dashboards, the team that owns code changes, and the team that gets paged. If any is blank, fix ownership before enabling the feature broadly.

Additional teaching scenarios

Scenario A — busy day

Imagine traffic multiplies by ten for a marketing event. Re-read the failure modes section and mark which ones become likely first. Write the first mitigation you would take for each marked item. This exercise turns abstract lists into operational instincts.

Scenario B — partial deploy

Half of your instances run the new version and half run the old version. Which assumptions in this lesson break if the two versions disagree about protocols, message fields, or transaction boundaries? Prefer designs that tolerate mixed versions for at least one deploy window.

Scenario C — explain to a new teammate

In five sentences, teach the core idea of this lesson without acronyms. If you cannot, the mental model is not yet solid—revisit the simple story and worked example until the five sentences feel natural.

Scenario D — metric design

List three metrics and one alert threshold you would ship with this mechanism. Good metrics name the user impact or the resource that runs out, not only that a counter incremented.

Scenario E — deliberate non-goals

Write two problems this lesson's technique should not solve. Explicit non-goals prevent cargo-cult adoption where every service gets the same machinery whether it needs it or not.

FAQ from first-time learners

Q: Can I long poll through a CDN?
A: Sometimes, if timeouts and caching are configured not to break holds. Test explicitly.

Q: Are WebSockets only for chat?
A: No—any bidirectional low-latency messaging benefits, including multiplayer and collaborative editing.

Q: What about HTTP/2 streams?
A: Useful server push variants exist; browser and API ecosystems still often standardize on WS/SSE/long poll for app-level events.

Track: Engineering Foundations

Previous: Load Balancing — Algorithms and Layers

Next: OSI Model — Seven Layers as a Debugging Map

By Shubham Jain

All articles · Study paths

Shubham Jain · Learning Lab