Event-Driven Packing and Dispatch with Kafka and Redis

Packing and dispatch is where software meets physical reality. If your system lags, trucks wait. If your states are inconsistent, shipments get blocked.

Context

We needed to support high daily throughput in a multi-stage dispatch workflow:

• Pick confirmation
• Cartonization
• Label generation
• Dock validation
• Dispatch closure

The challenge was not only speed. It was consistency across multiple teams and integrations.

Why Event-Driven

A purely synchronous API chain created bottlenecks:

• A downstream timeout delayed the entire transaction.
• Reprocessing failed requests caused duplicate side effects.
• Peak-hour load created cascading failures.

We moved to an event-driven model with Kafka as the backbone.

System Pattern

• Command APIs accepted user actions and validated business rules.
• Commands emitted domain events (packed, labeled, validated, dispatched).
• Consumers updated read models and triggered integrations.
• Redis stored short-lived workflow context and anti-duplication keys.

Reliability Controls

1. Idempotency everywhere

Every externally visible operation used a stable idempotency key.

2. Consumer replay safety

Consumers were stateless where possible and deterministic where stateful.

3. Outbox pattern at critical boundaries

For write + publish consistency, we used transactional outbox in key modules.

4. Operational visibility

We tracked lag, retries, poison messages, and stage-specific throughput.

Business Impact

• Dispatch turnaround improved due to reduced blocking calls.
• Manual intervention decreased because retry flows became safe and predictable.
• Teams got near real-time visibility into each shipment stage.
• Failures became diagnosable with event lineage instead of guesswork.

Lessons

• Event-driven does not mean eventually inconsistent forever. Design clear read models per use case.
• Avoid mixing business logic in consumers and controllers. Keep it inside domain services.
• Invest early in observability. Invisible event systems fail quietly and expensively.

Executive Snapshot

In logistics systems, reliability comes from idempotent state transitions and observable event lineage, not just message throughput.

Dispatch Latency

ReducedAsynchronous flows removed blocking dependency chains.

Retry Safety

HighStable idempotency keys prevented duplicate side effects.

Incident Diagnosis

FasterLag, retries, and poison queues were visible at stage level.