system-design · intermediate

Design a Notification Service — Push, Email, SMS

Start here

A notification service turns product events into user messages:

• Mobile push
• Email
• SMS
• (optional) in-app inbox

What you will learn

1. Define APIs for send and templated notifications.
2. Store user preferences and devices.
3. Pipeline: validate → enqueue → render → provider send.
4. Prioritize security alerts over marketing.
5. Rate-limit per user/channel and isolate providers.
6. Discuss idempotency and audit logs.

Words you need before we begin

Requirements

Functional

• Send notification API for internal services
• Templates + localization
• User preferences
• Delivery status callbacks (best effort)

Non-functional

• At-least-once toward providers with dedupe
• High throughput for campaigns
• Strict reliability for security messages
• Respect provider rate limits

High-level design

flowchart LR
  Svc[Product services] --> API[Notification API]
  API --> Val[Validation + prefs]
  Val --> QH[High priority queue]
  Val --> QL[Bulk queue]
  QH --> W[Workers]
  QL --> W
  W --> P1[Push provider]
  W --> P2[Email provider]
  W --> P3[SMS provider]
  W --> Log[(Delivery log)]

Step-by-step design

Step 1 — API

POST /v1/notifications with userId, templateId, data, channels[], idempotencyKey, priority.

Step 2 — Preferences & devices

Check opt-outs; load device tokens; drop disallowed channels.

Step 3 — Enqueue by priority

Security/otp → high lane; marketing → bulk lane with smoother rate.

Step 4 — Workers render & send

Render template; call provider; record attempt; schedule retries with backoff for transient errors.

Step 5 — Idempotency

Unique (idempotencyKey) or (userId, templateId, dedupeWindow) to stop double SMS codes when safe.

Step 6 — Observability

Per-channel success rate, provider latency, queue lag, opt-out hits.

Step 7 — Backpressure

If email provider throttles, slow bulk lane first; never starve OTP lane.

Failure modes

Trade-offs

Common mistakes

1. Product services each integrate Twilio directly.
2. Marketing and OTP share one queue without priority.
3. No idempotency for SMS.
4. Infinite retries.
5. Logging secrets/codes in plain text.

Check your understanding

1. Why a central notification service?
2. What goes in high vs bulk queues?
3. How do preferences apply?
4. Why idempotency keys for OTP?
5. What is provider isolation?

Practice

1. Design template schema with locale.
2. Write retry policy for email 429s.
3. Estimate QPS for 10M daily push.
4. Draw OTP vs newsletter paths.
5. Mock interview.

Deeper production notes

Compliance

SMS/email marketing needs legal opt-in rules by region—mention as constraint.

Inbox

In-app notifications can be a DB feed; push is best-effort wake-up.

Additional teaching scenarios

Scenario A — 10× peak

Scenario B — dependency down 30 minutes

Scenario C — interview wrap (5 sentences)

Revision summary

• Centralize channels behind one async service.
• Priority lanes, prefs, idempotency, DLQs.
• Protect users and providers with rate limits.
• Measure delivery health per channel.

Glossary

Abbreviations and terminology

• OTP — One-time password
• APNs/FCM — Apple/Google push
• DLQ — Dead-letter queue

What to learn next

1. Message queues
2. Rate limiting
3. Backpressure
4. Dead-letter queues

Extra teaching notes

When you apply this lesson, write the user-visible success metric first, then the failure mode you fear most. Design the smallest mechanism that protects that metric under partial failure. Prefer explicit timeouts, idempotency, and ownership over adding more infrastructure boxes.

In interviews or design reviews, narrate assumptions, request paths, and trade-offs out loud. A correct-enough design with clear failure handling beats a buzzword diagram without numbers. Revisit the worked example and restate it for a different domain to prove you own the ideas, not the template wording.

Extra teaching notes

When you apply this lesson, write the user-visible success metric first, then the failure mode you fear most. Design the smallest mechanism that protects that metric under partial failure. Prefer explicit timeouts, idempotency, and ownership over adding more infrastructure boxes.

In interviews or design reviews, narrate assumptions, request paths, and trade-offs out loud. A correct-enough design with clear failure handling beats a buzzword diagram without numbers. Revisit the worked example and restate it for a different domain to prove you own the ideas, not the template wording.

Extra teaching notes

When you apply this lesson, write the user-visible success metric first, then the failure mode you fear most. Design the smallest mechanism that protects that metric under partial failure. Prefer explicit timeouts, idempotency, and ownership over adding more infrastructure boxes.

In interviews or design reviews, narrate assumptions, request paths, and trade-offs out loud. A correct-enough design with clear failure handling beats a buzzword diagram without numbers. Revisit the worked example and restate it for a different domain to prove you own the ideas, not the template wording.

Extra teaching notes

When you apply this lesson, write the user-visible success metric first, then the failure mode you fear most. Design the smallest mechanism that protects that metric under partial failure. Prefer explicit timeouts, idempotency, and ownership over adding more infrastructure boxes.

In interviews or design reviews, narrate assumptions, request paths, and trade-offs out loud. A correct-enough design with clear failure handling beats a buzzword diagram without numbers. Revisit the worked example and restate it for a different domain to prove you own the ideas, not the template wording.

Extra teaching notes

When you apply this lesson, write the user-visible success metric first, then the failure mode you fear most. Design the smallest mechanism that protects that metric under partial failure. Prefer explicit timeouts, idempotency, and ownership over adding more infrastructure boxes.

In interviews or design reviews, narrate assumptions, request paths, and trade-offs out loud. A correct-enough design with clear failure handling beats a buzzword diagram without numbers. Revisit the worked example and restate it for a different domain to prove you own the ideas, not the template wording.

Interview and production field guide for this topic

Use this section as deliberate practice, not filler. Rewrite the worked example for a second domain you know well—fintech, education, logistics, or media. Keep the same skeleton: requirements, estimates, high-level diagram, request path, data model, scale lever, failure modes, and trade-offs. If you cannot fill every section without copying buzzwords, you do not yet own the design.

Numbers to force yourself to state

Always speak order-of-magnitude figures: peak QPS, storage growth per day, fan-out factor, connection counts, or queue depth. Wrong numbers that are explicit beat silent hand-waving. Correct the numbers when the interviewer or teammate challenges them; that is collaboration, not failure.

Failure minute

Set a timer for sixty seconds and list only failures: timeouts, duplicates, hot keys, dependency outages, bad deploys, and data corruption paths. For each, name detection and first mitigation. Designs that only describe the happy path are incomplete for production and weak in interviews.

Ownership and operability

Name the dashboard, the alert, the runbook section, and the team that pages. If any are blank, the system will train you during an incident. Prefer progressive delivery: canaries, flags, and rollback notes written before the change lands.

Consistency and retries

State whether the design assumes at-least-once delivery, whether handlers are idempotent, and where unique constraints live. Retries without idempotency are how double charges, double messages, and duplicate fan-out jobs appear. Timeouts without bounds are how thread pools die.

What good looks like in a review

A strong design review or interview answer clarifies scope, makes assumptions audible, draws a minimal path, deepens one or two bottlenecks, and closes with trade-offs and evolution. Use that bar on design-notification-service round 0 every time you revisit it.

Interview and production field guide for this topic

Use this section as deliberate practice, not filler. Rewrite the worked example for a second domain you know well—fintech, education, logistics, or media. Keep the same skeleton: requirements, estimates, high-level diagram, request path, data model, scale lever, failure modes, and trade-offs. If you cannot fill every section without copying buzzwords, you do not yet own the design.

Numbers to force yourself to state

Always speak order-of-magnitude figures: peak QPS, storage growth per day, fan-out factor, connection counts, or queue depth. Wrong numbers that are explicit beat silent hand-waving. Correct the numbers when the interviewer or teammate challenges them; that is collaboration, not failure.

Failure minute

Set a timer for sixty seconds and list only failures: timeouts, duplicates, hot keys, dependency outages, bad deploys, and data corruption paths. For each, name detection and first mitigation. Designs that only describe the happy path are incomplete for production and weak in interviews.

Ownership and operability

Name the dashboard, the alert, the runbook section, and the team that pages. If any are blank, the system will train you during an incident. Prefer progressive delivery: canaries, flags, and rollback notes written before the change lands.

Consistency and retries

State whether the design assumes at-least-once delivery, whether handlers are idempotent, and where unique constraints live. Retries without idempotency are how double charges, double messages, and duplicate fan-out jobs appear. Timeouts without bounds are how thread pools die.

What good looks like in a review

A strong design review or interview answer clarifies scope, makes assumptions audible, draws a minimal path, deepens one or two bottlenecks, and closes with trade-offs and evolution. Use that bar on design-notification-service round 1 every time you revisit it.

FAQ from first-time learners

Q: Should send be synchronous for OTP?
A: API can accept sync to the notification service, but provider I/O should still be carefully bounded; user needs fast success/fail.

Q: Kafka or SQS?
A: Either; justify ordering needs (usually per-user not global).

Track: Distributed Systems

Previous: Design a Job Scheduler — Delayed and Recurring Work

Next: Design Dropbox — File Sync and Storage

By Shubham Jain

All articles · Study paths

Shubham Jain · Learning Lab