kafka · intermediate

Kafka Partition Ordering and Delivery Guarantees

Start here

Apache Kafka is a distributed log: producers append records to topics, topics are split into partitions, and consumers read by advancing offsets.

Two questions dominate real designs:

1. Ordering: In what order will consumers see events?
2. Delivery: Might an event be lost, duplicated, or processed twice?

• Order is per partition (not global across a topic).
• Consumers usually see at-least-once delivery unless you carefully build stronger guarantees.
• “Exactly-once” marketing language usually means specific producer/broker/consumer configurations plus idempotent side effects—not magic.

What you will learn

1. Define topic, partition, offset, and consumer group.
2. Explain why keys choose partitions.
3. Contrast at-most-once, at-least-once, and practical exactly-once.
4. Work a complete order-events example.
5. See rebalance and duplicate pitfalls.
6. Avoid “Kafka guarantees order” without saying where.

What you should know first

Words you need before we begin

Simple story: multi-lane post office

A post office has many lanes (partitions). Letters in one lane stay in order. Letters in different lanes can be handled in any interleaving. If a clerk loses their place-marker (offset) and restarts a few letters earlier, some letters get processed twice—unless each letter has a unique id the system ignores on repeat.

Where the analogy stops: Kafka rebalances lanes among clerks automatically and can retain data for days for replay.

The problem without clear guarantees

Team assumes “Kafka is ordered and exactly once.” They process OrderCreated then OrderPaid for many orders on many partitions. Sometimes Paid is handled before Created for the same order because events landed on different partitions or consumers raced. Or a crash after side effect but before offset commit double-charges loyalty points.

Step-by-step explanation

Step 1 — Partition is the ordering unit

Within partition 3, offsets 10, 11, 12 appear in that order to a single-threaded consumer of that partition.

Across partitions 3 and 7, no global order.

Step 2 — Choose keys for entity order

Send all events for orderId=42 with key 42 so they hash to the same partition and preserve per-order order.

Step 3 — Scale with more partitions, not more global order

More partitions → more parallel consumers in a group. Trade-off: harder to get “total order of everything.”

Step 4 — Producer retries and duplicates

Network timeout after broker actually appended can cause retries. Enable idempotent producer for safer produce retries; still design consumers carefully.

Step 5 — Consumer commit timing defines delivery

• Commit before processing → risk at-most-once (loss on crash).
• Process then commit → at-least-once (duplicate on crash after process, before commit).

Step 6 — Rebalances interrupt

On rebalance, another consumer may resume near the last committed offset—duplicates possible.

Step 7 — Stronger processing needs more pieces

Transactional producers, exactly-once consumer semantics (read-process-write with Kafka transactions), and idempotent external side effects. Treat as advanced; do not claim casually.

Visual mental model

flowchart TB
  P[Producer] -->|key=order-42| T[Topic orders]
  T --> P0[Partition 0]
  T --> P1[Partition 1]
  T --> P2[Partition 2]
  P1 --> C[Consumer in group G]
  C -->|commit offset| O[(Offsets)]

Learning question: If OrderCreated and OrderPaid use different keys, can a consumer see Paid first?

Caption: Yes—different partitions destroy per-order ordering.

Complete worked example: order lifecycle stream

Starting situation

Services publish OrderCreated, PaymentCaptured, OrderShipped for millions of orders per day. Loyalty service awards points once per paid order.

Constraints

• Per-order event order required
• Loyalty must not double-award on redelivery
• Three consumers in one group for throughput

Decisions

Failure behavior

1. Crash after award before commit: redelivery; unique constraint no-ops second award.
2. Wrong key null: events spread randomly; order breaks—alert on null keys.
3. Rebalance mid-batch: duplicates; idempotency saves.

Outcome

Ordered per order, scalable across orders. Limitation: no single global timeline of all orders without a different design.

How it works in production

• Monitor consumer lag per partition
• Alert on stuck consumers and growing lag
• Schema registry for payloads
• Runbooks for poison messages and offset resets (dangerous)

Failure modes

Trade-offs

Compare with related concepts

Common misunderstandings

1. “Kafka is always globally ordered.” Only per partition.
2. “At-least-once means no duplicates.” It means the opposite risk.
3. “Idempotent producer equals exactly-once business effects.” External side effects still need care.
4. “More consumers than partitions helps.” Extra consumers in a group sit idle.
5. “Replaying a topic is free.” Side effects may not be.

Check your understanding

1. Where does Kafka guarantee order?
2. Why use orderId as key?
3. What delivery mode is process-then-commit?
4. What happens if consumers exceed partition count in a group?
5. How do you prevent double loyalty awards?

Practice

1. Design keys for multi-tenant events where order is per tenant.
2. Sketch consumer lag alerts for a payments topic.
3. Explain a duplicate after rebalance with a timeline.
4. Choose partition count for 50 MB/s with rough math.
5. List what “exactly-once” would require beyond defaults.

Deeper production notes

Head-of-line and slow keys

A hot key can overload one partition. Watch produce/consume imbalance. Design keys for both order and cardinality.

Poison records

Deserialization failures can block a partition consumer loop if not handled. Use dead-letter patterns and skip policies carefully.

Specialist caution

Kafka transactions and EOS configurations evolve by version. Verify against current Apache Kafka docs before promising regulatory-grade guarantees.

Additional teaching scenarios

Scenario A — peak load day

Scenario B — mixed versions

Scenario C — five-sentence teach-back

Scenario D — metrics and alerts

Scenario E — non-goals

Scenario F — ownership

Revision summary

• Order is per partition; use keys for entity order.
• Default consumer reality is at-least-once.
• Build idempotent side effects.
• Scale with partitions; do not invent global order casually.
• Treat “exactly-once” as an advanced, partial property—not a slogan.

Glossary

Abbreviations and terminology

• EOS — Exactly-once semantics (context-specific)
• ISR — In-sync replicas (broker durability topic)
• ack — Producer acknowledgment setting

What to learn next

1. Kafka architecture deep dive
2. Kafka exactly-once semantics
3. Idempotency, retries, backoff
4. Dead-letter queues

Extra teaching notes for first-time builders

Write the single bottleneck you are protecting before picking tools. Name the signal that tells you the design is working for users, not only that internal counters move. When reviewing a change related to this lesson, ask what happens when the component is slow for ten minutes, down entirely, or running twice. Prefer small explicit failure modes that operators can understand at 3 a.m.

Document ownership for dashboards, code, and pages. Undocumented mechanisms become folklore and then outages. Prefer designs that tolerate mixed versions for at least one deploy window so rollouts do not require perfect global simultaneity.

Napkin math helps: estimate peak rate, multiply by payload size, and ask whether the design still holds when a dependency is at half capacity. If the answer depends on luck, add bounds, backpressure, or shedding before production traffic arrives.

FAQ from first-time learners

Q: Can I get global order with one partition?
A: Yes for that topic, at the cost of throughput.

Q: Do multiple consumer groups each see all messages?
A: Yes—each group has its own offsets (pub/sub style fan-out).

Q: Is Kafka a database?
A: It is a durable log. Some patterns use it as a source of truth, but it is not a general relational store.

Track: Data, Storage and Messaging

Previous: Kafka Exactly-Once Semantics — What Is Actually Guaranteed

Next: Kafka Streams Introduction

Series: Kafka & Event Streaming

1. Message Queues — Hand Work Off Without Blocking the User
2. Kafka Architecture — Brokers, Partitions, ISR, and Consumers
3. Kafka Partition Ordering and Delivery Guarantees (this guide)
4. Kafka Exactly-Once Semantics — What Is Actually Guaranteed
5. Transactional Outbox and Saga Patterns — Reliable Multi-Step Work
6. Kafka Streams Introduction
7. Kafka with Spring Boot Code Walkthrough

All series

By Shubham Jain

All articles · Study paths

Shubham Jain · Learning Lab