Microservices Communication Patterns: Sync, Async, and the Saga Pattern
Service boundaries, the transactional outbox pattern, and compensating transactions — the parts of microservices architecture that break in production, not the diagram.
Drawn from building Kuybi, a NestJS microservices framework where main.ts (HTTP API) and worker.ts (BullMQ queue processing) run as genuinely separate processes communicating asynchronously — not a theoretical treatment of the pattern.
Microservices communication design comes down to one recurring decision, repeated at every service boundary: does this interaction need an immediate answer, or can it be decoupled? Get that wrong and you build a distributed monolith — one where every service still has to be up at once, just with network calls where function calls used to be. This guide covers sync vs async trade-offs, the transactional outbox pattern, the saga pattern for distributed transactions, API gateway design, and tracing a request across service boundaries.
1. Service Boundaries: The Decision That's Hard to Undo Later
Every microservices failure story starts the same way: services were split along the org chart instead of along data ownership. A distributed monolith — services that are technically separate deployables but still call each other synchronously for every operation and share a database — combines microservices' operational overhead (network calls, service discovery, distributed deployment) with the monolith's tightest coupling (every service still has to be up at once), without either architecture's actual benefit.
The Domain-Driven Design concept of a bounded context is the right unit of decomposition: a boundary within which a specific domain model and its terminology stay consistent. "Order" means something different to the fulfillment service (a shipment to schedule) than it does to the billing service (an invoice line item) — trying to force one shared "Order" model across both is what creates the tightest, most brittle coupling in a microservices system.
In practice, this is why a modular monolith — one deployable, internally split into modules with enforced boundaries — is often the right starting point rather than a failure to "properly" adopt microservices. Kuybi, the NestJS framework this pattern is drawn from, structures its modules/ directory this way: acl, attachments, audit, auth, categories, stories, tags, and users are separate bounded contexts today, deployed as one process — extractable into real services later, if and when a specific one actually needs independent scaling or a separate deploy cadence, without a rewrite.
The test that actually matters
Before splitting a service, the question isn't "is this a distinct business capability?" — almost anything qualifies under that test. The question is: does this boundary need to scale, deploy, or fail independently of its neighbors? If the honest answer is no, a module boundary inside one deployable gets you the same design discipline without the network hop.
2. Synchronous Communication: REST and gRPC, and When Each One Bites You
Synchronous request/response is the natural first choice because it maps directly onto how a monolith's function calls already worked. The cost shows up under failure, not under normal load: a synchronous call chain is only as available as the least available service in it. If service C is down, and B calls C to answer A's request, A is now down too — an outage that started in one service just became three services' outage.
| Protocol | Payload | Best fit | Failure mode to watch |
|---|---|---|---|
| REST/JSON | Text, human-readable | Public APIs, browser clients | Timeout cascades with no built-in deadline propagation |
| gRPC | Binary (Protocol Buffers) | Internal service-to-service, high volume | Requires a shared .proto contract — schema drift breaks silently without codegen discipline |
| GraphQL federation | Text, client-shaped queries | API gateway aggregating multiple services | A single slow subgraph can stall an entire federated response |
gRPC's binary encoding and HTTP/2 multiplexing make it measurably faster for internal service-to-service traffic than JSON over REST, which is why it's the common default for internal calls even in stacks that expose REST or GraphQL externally. But the deeper issue with any synchronous protocol is the same regardless of encoding: a timeout is not a failure-handling strategy, it's a delay before the failure becomes visible. A circuit breaker (open the circuit and fail fast after N consecutive failures, instead of letting every caller wait out the full timeout) turns a slow cascading failure into a fast, contained one — it doesn't eliminate the coupling, but it stops one struggling service from dragging its callers down with it.
3. Asynchronous Communication and the Transactional Outbox Pattern
Moving an interaction from synchronous to asynchronous — a message queue or event bus instead of a direct call — removes the availability coupling from the previous section: the publisher doesn't need the consumer to be up at the same instant. It introduces a different, less obvious problem instead: the dual-write problem.
Consider a service that needs to save a new order to its database *and* publish an "OrderCreated" event so the fulfillment service can react. Written as two separate steps — save to the database, then publish to the broker — there's a window where one succeeds and the other fails: the database write commits but the process crashes before publishing, or the publish succeeds but the database transaction rolls back. Either way, the two systems are now inconsistent, silently.
The transactional outbox pattern
The fix is to never treat them as two operations. Write the event to an outbox table in the same database, in the same transaction as the business write — so both commit atomically or neither does — then have a separate process relay unpublished outbox rows to the broker:
// Inside the same DB transaction as creating the order
await this.dataSource.transaction(async (manager) => {
const order = await manager.save(Order, newOrder);
await manager.save(OutboxEvent, {
aggregateId: order.id,
eventType: "OrderCreated",
payload: JSON.stringify(order),
publishedAt: null,
});
});A relay process — a polling job or a change-data-capture stream reading the outbox table — then publishes unpublished rows and marks them sent. If the relay crashes mid-publish, it retries the same row on restart, which means consumers need to be idempotent (safe to process the same event twice), but that's a far easier problem than reconciling two systems that silently drifted apart.
This is the same shape of decoupling behind Kuybi's own main.ts / worker.ts split: the HTTP API process (main.ts) enqueues a BullMQ job and returns immediately, while a completely separate worker process consumes it — so a slow or failing background job (sending an email, processing an upload) never blocks or crashes the request path that queued it.
4. The Saga Pattern: Distributed Transactions Without Two-Phase Commit
A single database transaction guarantees atomicity: every statement commits, or none do. Once "the transaction" spans multiple services, each with its own database, that guarantee is gone — there's no cross-service COMMIT. Two-Phase Commit (2PC) technically exists to solve this, but it requires every participant to hold locks until every other participant votes, which means one slow or unavailable service blocks all of them. It doesn't survive contact with services that need to scale and fail independently — which is the entire premise of choosing microservices in the first place.
The saga pattern replaces one distributed transaction with a sequence of local transactions, each in its own service, each paired with a compensating transaction that undoes its effect if a later step fails.
Orchestration: a central coordinator explicitly calls each service in sequence and issues compensations on failure.
OrderSaga.start()
→ PaymentService.charge(order) [ok]
→ InventoryService.reserve(order) [fails: out of stock]
→ PaymentService.refund(order) [compensation]
→ OrderService.markFailed(order)Choreography: there is no central coordinator — each service publishes an event when it completes its step, and the next service reacts to that event.
OrderService: publishes OrderCreated
PaymentService: consumes OrderCreated, charges, publishes PaymentCompleted
InventoryService: consumes PaymentCompleted, reserves stock —
on failure, publishes InventoryReservationFailed
PaymentService: consumes InventoryReservationFailed, refundsChoreography is more decoupled — no service needs to know about the others, only about the events it consumes and produces — but that decoupling has a real cost: there's no single place to read the whole business process, and debugging "why did this order end up half-refunded" means reconstructing the flow from logs across four services. Orchestration keeps the full sequence, including every compensation, in one place, at the cost of the orchestrator having to know about every participant. For sagas with more than three or four steps, or any compliance requirement to audit *why* a transaction failed, that traceability usually outweighs the extra coupling.
Either way, the property that makes this work is not "distributed atomicity" — that doesn't exist here — it's that every step has a compensating action. If a step can't realistically be undone (an email that's already been sent, for instance), it either needs to move to the end of the saga, after every other step has succeeded, or it needs its own separate confirmation step before it becomes irreversible.
5. API Gateway: One Front Door, Not One Bottleneck
An API gateway sits in front of a set of services and handles the concerns every one of them would otherwise duplicate: TLS termination, authentication, rate limiting, and request routing. For client-facing traffic, it's also frequently the layer that aggregates multiple backend calls into one response shaped for a specific client — the Backend-for-Frontend (BFF) variant of the pattern, where a mobile app and a web app each get a gateway tuned to their own payload shape instead of both consuming the same generic internal API.
The failure mode to actively guard against is scope creep: a gateway that starts making business decisions — "if this user's plan is Pro, call the analytics service; otherwise skip it" — has quietly become a second copy of your domain logic, just one that isn't versioned or tested the way the services underneath it are. Keep the gateway's job mechanical: authenticate, route, rate-limit, aggregate. Every conditional that depends on *what the business rule is*, not *who's asking*, belongs in a service, not the gateway in front of it.
6. Observability Across Service Boundaries: Correlation IDs and Distributed Tracing
A request that touches five services produces five separate sets of logs, in five different processes, with no inherent relationship between them — unless something explicitly ties them together. That something is a correlation ID: a unique identifier generated at the edge (the API gateway, or the first service that receives the request) and propagated through every subsequent call, HTTP header and message-broker metadata alike.
The W3C Trace Context specification standardizes this as the traceparent header, which OpenTelemetry and most modern tracing systems implement directly — using the standard instead of a home-grown X-Correlation-Id header means traces interoperate with any W3C-compliant tool without custom translation code.
Two details are easy to get right for HTTP and easy to forget for messaging:
- Propagate through the broker, not just HTTP. A trace context header on an HTTP call is often implemented via a library's auto-instrumentation and forgotten about. A message published to a queue needs the same trace context carried in its metadata — otherwise the trace silently ends at the last synchronous hop, and everything the saga's compensating actions do downstream becomes untraceable.
- One trace ID per business operation, not per network call. The trace ID that starts at the gateway should still be present on the compensating transaction that fires three services later — that's what makes it possible to look at one trace and see the whole saga, including its failure and rollback, instead of piecing it together from unrelated fragments.
Kuybi's own stack pairs Prometheus (metrics), OpenTelemetry (tracing) and Sentry (error tracking) for exactly this reason — a slow request and a failed request are different signals, and neither one, on its own, explains what happened across a chain of services.
Summary Checklist
- ✔ Service boundaries follow data ownership (DDD bounded contexts), not the org chart.
- ✔ Synchronous calls only where the caller needs an immediate answer, with a circuit breaker on the ones that remain.
- ✔ Every async publish goes through a transactional outbox — never a database write and a broker publish as two independent steps.
- ✔ Cross-service transactions are modeled as sagas, with an explicit compensating action for every step.
- ✔ The API gateway authenticates, routes, and rate-limits — it doesn't make business decisions.
- ✔ A W3C Trace Context correlation ID rides along on every HTTP call and every queued message.
Frequently Asked Questions
Should every service-to-service call be asynchronous, or is REST between services ever fine?
Synchronous REST or gRPC is fine when the caller genuinely can't proceed without an immediate answer — a checkout flow validating that a promo code exists, for example. The default should still lean asynchronous for anything that changes state across a service boundary, because a synchronous call chain is only as available as its least available link: if a downstream service is down, every synchronous caller in the chain is now down too.
What's the difference between orchestration and choreography sagas?
Orchestration uses a central coordinator that explicitly calls each service in sequence and issues compensating actions on failure — the whole business process is readable in one place. Choreography has no central coordinator: each service publishes an event when it finishes its step, and the next service reacts to that event. Choreography is more decoupled since no service needs to know about the others, but debugging a failed transaction means reconstructing the sequence from logs across every service involved, rather than reading it from one orchestrator.
How do you avoid the dual-write problem when a service needs to update its database and publish an event?
Use the transactional outbox pattern: write the event to an outbox table in the same database transaction as the business write, so both commit or neither does. A separate relay process then reads unpublished rows from the outbox and publishes them to the message broker, retrying on failure. This guarantees the event is eventually published if and only if the business write actually committed, which writing directly to the database and the broker as two independent steps cannot guarantee.