Queues, Async, and Microservices

Synchronous calls are brittle: the caller blocks until the callee responds. If the payment service takes 3 seconds, your checkout API takes 3 seconds. If the payment service is down, checkout is down. Scale them independently? You can't — they're coupled at the protocol level.

A message queue breaks that coupling. The producer writes a message and returns immediately. The consumer reads it whenever it's ready. Suddenly:

• Latency: checkout responds in 20 ms, not 3 s.
• Resilience: payment service restarts? Messages wait in the queue, not dropped.
• Scale independently: add payment workers without touching checkout.
• Traffic shaping: a queue absorbs a 10x spike; consumers work at their own pace.

Real numbers: AWS SQS processes over 1 million messages/second at peak. A single Kafka partition sustains ~100 MB/s throughput. For an e-commerce platform handling 50k orders/hour (~14 orders/sec), a queue makes the payment worker fleet the scale knob, not the checkout API.

Two distinct delivery models:

Point-to-point (queue): one producer, one consumer group. A message is delivered to exactly one consumer and then deleted. AWS SQS, RabbitMQ classic queues. Use this when one worker should process each job — sending an email, resizing an image, processing an order.

Pub/sub: one producer, multiple independent subscriber groups. Each subscriber group gets its own copy of every message. Apache Kafka, Google Cloud Pub/Sub, AWS SNS. Use this when multiple downstream services need the same event — an order.placed event consumed by inventory, billing, notifications, and analytics simultaneously.

Kafka can act as both: topics with one consumer group = queue semantics; topics with multiple groups = pub/sub.

Distributed systems can't guarantee exactly-once delivery without coordination overhead. Most queues guarantee at-least-once: the message will be delivered, possibly more than once. SQS visibility timeout: if a consumer takes the message but doesn't delete it within 30 s (configurable), SQS re-delivers it — even if the consumer did process it and then crashed before deleting.

This means your consumers MUST be idempotent: processing the same message twice has the same effect as processing it once.

def process_order(message: dict) -> None:
    order_id = message["order_id"]
    # Idempotency: check if already processed
    if db.exists("processed_orders", order_id):
        return  # already done, skip
    with db.transaction():
        charge_payment(order_id)
        update_inventory(order_id)
        db.insert("processed_orders", order_id)  # mark done atomically

The idempotency key (here order_id) must be written atomically with the business action. If they're separate transactions, a crash between them leaves the system inconsistent.

Order placement is synchronous; fulfillment steps run asynchronously via a message queue.

Backpressure is what happens when consumers can't keep up with producers. Without it, the queue grows unboundedly and consumers eventually OOM or fall so far behind that messages are hours stale.

Mechanisms:

• Queue depth limit: SQS, Kafka, and RabbitMQ can be configured to reject new messages when the queue exceeds a threshold. Producers must handle the rejection (retry, drop, or alert).
• Consumer-side rate limiting: process N messages/sec per worker. Autoscale workers based on queue depth (SQS → CloudWatch → Lambda/ASG).
• Flow control: producers explicitly check queue depth and slow down or pause. Kafka producers can set max.block.ms to block when buffer is full.

Dead-letter queues (DLQ): when a message fails processing N times (e.g., 3 retries), move it to a DLQ instead of blocking the main queue. Engineers inspect the DLQ to diagnose poison-pill messages (malformed payloads, unexpected data). Without a DLQ, one bad message can halt all processing indefinitely. Configure maxReceiveCount=3 in SQS to enable automatic DLQ routing.

The industry overcorrected toward microservices. Here's an honest comparison:

Monolith: one deployable unit. In-process function calls (microsecond latency vs milliseconds for RPC). Atomic transactions across all data. Easy to debug — one log stream, one debugger. The right choice for teams under ~20 engineers or products pre-product-market-fit.

Microservices: independent deploy, independent scale, independent language choice. The penalty is severe: distributed tracing across 20 services, partial failure handling everywhere, network latency on every cross-service call, eventual consistency instead of ACID transactions, and a drastically more complex CI/CD pipeline.

Rule: start with a monolith. Extract a service only when (a) it needs to scale independently at a rate that justifies the operational overhead, or (b) different teams own it and deploy conflicts become painful.

Distributed systems fail in partial ways. Three patterns to build resilience:

Timeouts: every network call must have a timeout. Without one, a slow downstream service consumes your thread pool indefinitely. Set timeouts based on P99 latency + buffer: if payment service P99 = 500 ms, set timeout at 1 s. Too tight = false timeouts; too loose = cascading slowdowns.

Retries with exponential backoff + jitter: retry transient failures (503, network timeout), but not all errors (400 Bad Request is not transient). Backoff: wait 100 ms, then 200 ms, then 400 ms. Add jitter (random ±50 ms) to prevent retry storms where all callers retry simultaneously.

import time, random

def call_with_retry(fn, max_retries=3):
    for attempt in range(max_retries):
        try:
            return fn()
        except TransientError:
            if attempt == max_retries - 1:
                raise
            wait = (2 ** attempt) * 0.1 + random.uniform(0, 0.05)
            time.sleep(wait)

Circuit breaker: after N consecutive failures, open the circuit — stop calling the downstream service entirely. Return a fast fallback (cached response, error) instead. After a timeout (e.g., 30 s), try one probe request. If it succeeds, close the circuit. Netflix Hystrix, Resilience4j, Envoy's outlier detection all implement this. Prevents one failing service from consuming all threads in a cascade.

Microservices killed ACID. When an order touches Inventory, Payment, and Notification services on three separate databases, there's no single transaction that spans them. Enter the saga.

A saga is a sequence of local transactions, each publishing an event that triggers the next. If any step fails, compensating transactions roll back completed steps.

Choreography (event-driven): each service listens to events and reacts. No central coordinator. Simple, but hard to follow the flow across services.

Orchestration (command-driven): a saga orchestrator sends commands and waits for replies. Flow is explicit and visible in one place; the orchestrator is a coordination bottleneck.

Example — order placement saga:

1. ReserveInventory → success → 2
2. ChargePayment → success → 3. If failure → ReleaseInventory (compensate)
3. SendConfirmation → success → done

Sagas introduce complexity: compensating transactions must be idempotent, and partial failures leave the system in intermediate states that can be visible to users (order shows as "processing" for seconds). Use sagas only when strong consistency across services is genuinely required — often, eventual consistency with a DLQ for failures is simpler.