Durable Workflows vs Message Queues

The Problem#

An order workflow may reserve inventory, charge payment, wait for shipping, send emails, and compensate failures. A queue can run each step, but it does not remember the business process by itself.

Why It Matters#

Teams often build a workflow engine accidentally: queues, cron jobs, status tables, retry counters, and cleanup scripts. Durable workflow systems make that state explicit.

Core Concepts#

A message queue moves work from producers to consumers. A durable workflow records the history of a process and replays it after failures. Activities perform side effects. Workflow code decides what happens next and must stay deterministic.

Implementation#

Use a queue for simple async work:

user_signed_up -> send_welcome_email

Use a workflow when the process has timers, human waits, compensation, or multiple dependent side effects:

create_order
  -> reserve_inventory
  -> charge_payment
  -> wait_for_fulfillment
  -> release_inventory_if_cancelled

Real Project Scenario#

A travel booking flow may reserve a flight, reserve a hotel, wait for payment, and release reservations if payment fails. Building this with only queues usually creates status tables, cron cleanup, retry counters, and manual recovery scripts. A durable workflow engine keeps the process history and recovery behavior in one place.

Production Setup#

Use queues for simple background work and workflows for business processes with state, timers, compensation, or human waits. Expose workflow status to support teams, because the most common production question is not "did a job run?" It is "where is this customer order stuck?"

Common Mistakes#

• Using one giant queue message as hidden workflow state.
• Retrying payment or fulfillment steps without idempotency keys.
• Putting non-deterministic behavior inside workflow decision code.
• Choosing a workflow engine for simple fire-and-forget tasks.

Production Considerations#

Model each business process with a stable id. Make every activity idempotent or non-retryable. Expose workflow status to support teams so failures do not require database archaeology.

Security#

Workflow histories can contain sensitive business data. Define retention and avoid storing secrets in event payloads.

Performance#

Queues are cheaper and faster for simple fan-out. Durable workflows pay more overhead but reduce operational complexity for processes that live across minutes, days, or months.

Summary#

Queues are transport. Durable workflows are process state. Use each where it fits instead of rebuilding workflow semantics out of scattered queue consumers.