system-design · beginner

Vertical vs Horizontal Scaling — Bigger Machine or More Machines?

Start here

When load grows, you can scale in two basic ways:

Vertical scaling (scale up): give one server more CPU, RAM, disk, or a bigger VM size.
Horizontal scaling (scale out): run more servers and split work among them.

You should care because interviews and production capacity planning both hinge on this choice. Vertical is simple until you hit hardware/cloud size limits or single-host failure risk. Horizontal enables huge scale but needs statelessness, load balancing, and often sharding.

What you will learn

Define vertical and horizontal scaling clearly.
See when each is appropriate.
Connect horizontal scale to load balancers and data partitioning.
Work a complete API tier example.
Discuss databases: vertical first, then replicas/shards.
Avoid “always microservices = horizontal.”

What you should know first

Topic	Why
Scalability	Goals of scaling
Load balancing	Horizontal app tier
System design foundations	Request path pieces

Words you need before we begin

Term	Plain English
Scale up	Vertical: bigger box.
Scale out	Horizontal: more boxes.
Stateless service	Any instance can handle any request without local sticky memory.
Sticky session	Forcing a user to one instance—hurts horizontal flexibility.
Shard	Horizontal split of data by key.
Replica	Copy of data for reads/HA.
Diminishing returns	Bigger machines get pricey/slow to grow.

Simple story: cafe capacity

Vertical: renovate one cafe with a giant kitchen.
Horizontal: open more cafe locations with shared recipes (stateless process) and a city map (load balancer).

One giant kitchen still burns down as one fire (SPOF). Many locations need coordination.

The problem of only vertical thinking

“Just buy a bigger database.” Eventually:

Cloud instance sizes max out.
Cost curves bend upward.
Maintenance windows affect everyone.
One host failure is a total outage.

Step-by-step explanation

Step 1 — Measure the bottleneck

CPU, memory, disk IO, network, or lock contention? Scaling the wrong dimension wastes money.

Step 2 — Try simple vertical for early stages

Often the best ROI before architecture rewrites.

Step 3 — Make app tier horizontally scalable

Stateless tokens/sessions in Redis/DB
Load balancer + health checks
Multiple instances

Step 4 — Scale reads on data

Replicas, caches—before complex write sharding.

Step 5 — Scale writes when needed

Sharding/partitioning; accept operational cost.

Step 6 — Automate

Autoscaling policies from CPU/RPS/lag—not vibes.

Step 7 — Re-evaluate cost

Sometimes a larger vertical DB + cache beats premature shards for a given company stage.

Visual mental model

flowchart TB
  subgraph v [Vertical]
    C1[Client] --> Big[One big server]
  end
  subgraph h [Horizontal]
    C2[Client] --> LB[Load balancer]
    LB --> S1[Server]
    LB --> S2[Server]
    LB --> S3[Server]
  end

Learning question: Which design survives one server disk death more gracefully?

Caption: Horizontal with redundancy—if data is also replicated.

Complete worked example: growing API

Starting situation

Monolith API on one VM at 70% CPU peak. Product expects 5× traffic in a year.

Phase 1 vertical

Move to larger VM; add better DB plan. Ships in days.

Phase 2 horizontal app

Containerize API; 3+ replicas; Redis sessions; LB. Deploy without downtime.

Phase 3 data

Read replicas for reporting; cache hot GETs; plan shard only if write IO saturates primary.

Failures discussed

Vertical only: host death still total.
Horizontal without statelessness: sticky mess.
Sharding too early: engineer time fire.

How it works in production

Cloud instance families and reserved capacity
Kubernetes HPA/VPA concepts
DB vertical plans vs Aurora-style storage separation
Cost dashboards by service

Failure modes

Mode	Impact	Mitigation
Vertical ceiling	Cannot grow	Horizontal strategy
Stateful app instances	Bad balancing	Externalize state
Autoscale thrash	Instability	Cooldowns, correct metrics
Premature sharding	Complexity outage	Evidence-based triggers
Ignoring data tier	App scales, DB melts	Measure end-to-end

Trade-offs

Approach	Pros	Cons
Vertical	Simple ops	Limits, SPOF, cost curve
Horizontal	Scale & HA potential	Coordination complexity
Hybrid	Pragmatic	Need clear playbook

Compare with related concepts

Concept	Notes
Scalability	Goal; vertical/horizontal are strategies
Elasticity	Auto add/remove capacity
Performance optimization	Sometimes better than more machines

Common misunderstandings

“Horizontal is always better.” Not at five users.
“Vertical is obsolete.” Still primary tool for many DBs early.
“More pods fix lock contention.” App locks may worsen.
“Microservices required to scale out.” A modular monolith can scale horizontally.
“Autoscaling replaces capacity planning.” Dependencies still saturate.

Check your understanding

Define scale up vs scale out.
Why statelessness matters for horizontal apps?
When scale a database vertically first?
Name a risk of premature sharding.
What should you measure before scaling?

Practice

Design a two-phase plan for a read-heavy blog.
Estimate cost: one huge VM vs four smaller ones (qualitative).
List stateful features that block scale-out.
Choose autoscale signals for API vs queue workers.
Explain SPOF differences with diagrams.

Deeper production notes

Connection pools

Horizontal app tiers multiply DB connections. Pool sizes × pods can knock over databases—coordinate.

Noisy neighbors

In multi-tenant horizontal systems, isolate heavy tenants so scale-out for one does not tax others unfairly.

Additional teaching scenarios

Scenario A — peak load day

Traffic multiplies by ten. Mark which failure modes appear first and the first mitigation for each.

Scenario B — mixed versions

Half the fleet runs an old build. Which assumptions break? Prefer one deploy window of compatibility.

Scenario C — five-sentence teach-back

Explain the core idea without acronyms.

Scenario D — metrics and alerts

List three metrics and one alert tied to user impact or scarce resources.

Scenario E — non-goals

Name two problems this technique should not solve.

Scenario F — ownership

Who owns dashboards, code, and pages?

Revision summary

Vertical = bigger machine; horizontal = more machines.
Start simple; scale with evidence.
Horizontal apps need statelessness + LB.
Data tier often needs its own plan (replicas/shards).
Cost and complexity are first-class trade-offs.

Glossary

Term	Definition
Vertical scaling	Adding resources to one node.
Horizontal scaling	Adding nodes.
Stateless service	No required local session affinity.

Abbreviations and terminology

SPOF — Single point of failure
HPA — Horizontal pod autoscaler
VPA — Vertical pod autoscaler
QPS — Queries per second

What to learn next

Extra teaching notes for first-time builders

Write the single bottleneck you are protecting before picking tools. Name the signal that tells you the design is working for users, not only that internal counters move. When reviewing a change related to this lesson, ask what happens when the component is slow for ten minutes, down entirely, or running twice. Prefer small explicit failure modes that operators can understand at 3 a.m.

Document ownership for dashboards, code, and pages. Undocumented mechanisms become folklore and then outages. Prefer designs that tolerate mixed versions for at least one deploy window so rollouts do not require perfect global simultaneity.

Napkin math helps: estimate peak rate, multiply by payload size, and ask whether the design still holds when a dependency is at half capacity. If the answer depends on luck, add bounds, backpressure, or shedding before production traffic arrives.

Extra teaching notes for first-time builders

FAQ from first-time learners

Q: Can I do both?
A: Yes—larger nodes and more of them is common.

Q: Is serverless horizontal?
A: Platforms scale out invocations for you, with different limits.

Q: Does horizontal always improve availability?
A: Only with redundancy and healthy data replication—not merely more app pods on one database.

Track: Software Design and Architecture

Previous: Sync vs Async Communication — Wait or Don’t Wait

By Shubham Jain

All articles · Study paths

Shubham Jain · Learning Lab