Microservices vs Monolith Performance: Latency Trade-offs We Measured

A team does not usually split a monolith because one endpoint is slow.

It happens more quietly.

One part of the codebase starts changing faster than the rest. Deployments become tense. Ownership gets blurry. A small change in billing requires a full regression pass through orders, users, reporting, notifications, and admin workflows.

At that point, microservices start to look attractive.

But after the first migration, another problem appears: the new system is easier to deploy, but the request path is slower.

The monolith was not elegant, but it was fast because most calls never left the process.

This article documents the latency trade-offs we measured when comparing a monolithic Spring Boot service with a microservices-style request path. The point is not that monoliths are always better. The point is that architecture boundaries have a runtime cost, and that cost is easy to underestimate.

The Architecture We Compared

The monolith path was simple. The HTTP request entered a Spring Boot application, business logic executed through in-process method calls, and the application talked to the database.

Client
  ↓
Spring Boot Application
  ↓
Database

The microservices path looked more like the kind of system many teams move toward over time. The same user request passed through an API gateway and several services before reaching the database-backed operation that actually produced the response.

Client
  ↓
API Gateway
  ↓
Service A
  ↓
Service B
  ↓
Service C
  ↓
Database

On a diagram, this looks like a clean separation of responsibilities. In production, every arrow becomes something the runtime has to pay for.

• A network hop
• A serialization and deserialization boundary
• A connection pool
• A timeout policy
• A retry decision
• A log and trace propagation point
• A place where partial failure can occur

Initial Assumption

The first assumption was familiar: microservices would not materially affect latency because the individual service calls were small.

That is how the cost hides. One HTTP call inside the same region or cluster feels cheap. JSON serialization feels cheap. A trace header feels cheap. A gateway hop feels cheap.

Individually, those statements can be true.

But user-facing latency is not paid individually. It is paid cumulatively across the whole request path.

The performance question is not: is one service call cheap? The question is: how many service calls exist on the critical path?

What We Measured

We compared the two shapes under the same workload pattern: a request that performs application logic and returns a response only after the downstream work is complete.

The numbers that mattered were not only average latency. Average latency is useful for a quick baseline, but it hides the exact pain users notice when a system gets busy.

We cared more about:

• P95 latency
• P99 latency
• Throughput at stable error rate
• Timeout count
• Retry count
• Thread pool and connection pool pressure
• Trace spans on the critical path

The monolith spent most of its time inside application logic and database access. The microservices version spent additional time at the boundaries between services.

Where The Latency Came From

The extra latency did not come from one dramatic bottleneck. That was the important finding.

There was no single terrible query. No obviously overloaded JVM. No one service consuming all CPU. Instead, the additional latency came from several small costs that compounded.

1. Network Hops

In the monolith, one component called another component through memory. In the microservices version, the same logical operation became an HTTP call.

That introduced connection acquisition, network transit, request parsing, response parsing, and error handling. Even when each hop was fast, the request path paid that cost repeatedly.

2. Serialization

Inside the monolith, a Java object can move through the call stack without becoming JSON. Across service boundaries, the object becomes a payload.

That conversion is not usually the biggest cost in isolation, but it becomes visible when the request path has several service-to-service calls and the payloads grow over time.

3. Connection Pools

Microservices replace method calls with client calls. That means connection pools, max concurrent requests, keep-alive behavior, and queueing behavior now matter.

A service can look healthy while its outbound client pool is quietly becoming the bottleneck.

4. Retries

Retries are useful when they absorb short transient failures. They are dangerous when they multiply load during a slowdown.

In the monolith, a failed internal call is usually just an exception in the same process. In a microservices system, a failed downstream request might trigger a retry, which might trigger another retry upstream, which might increase load on the dependency that was already struggling.

5. Observability Overhead

Tracing, structured logging, and correlation IDs are necessary in microservices. They are not optional if you want to debug production incidents.

But they also add more work to every boundary. The overhead is usually worth it, but it is still part of the system budget.

The Timeout Problem

The most dangerous latency bug was not the extra network call itself. It was the timeout behavior around that call.

A monolith often fails quickly because the call stack is local. A microservices request can wait at each boundary unless every client has deliberate timeout settings.

This is the kind of configuration that looked small during implementation but mattered during load testing:

orders:
  client:
    connect-timeout: 300ms
    read-timeout: 800ms
    retries: 1
    retry-on: 5xx, timeout

The exact values are not the point. The point is that every service-to-service call needs an explicit latency budget. Without one, a slow dependency can hold request threads, fill queues, and turn a small downstream problem into a wider incident.

Why The Monolith Was Faster

The monolith won on raw request latency because it had fewer boundaries.

That sounds obvious after the measurement, but it is easy to forget during design discussions. We often compare architectures by ownership, release cadence, scalability, and deployment independence. Those are real advantages. They are just not free.

In the monolith:

• Method calls stayed in process
• No HTTP client was needed between internal modules
• No payload serialization was needed between internal modules
• Only one application had to be scheduled for most of the work
• Most failure handling stayed inside one runtime

That does not make the monolith better in every dimension. It only explains why it had a lower latency floor.

Why Microservices Were Still Attractive

The microservices version was slower, but it solved different problems.

This is where architecture discussions often go wrong. Teams ask whether microservices are faster or slower. That is not the best question.

The better question is: what problem are we buying this latency overhead to solve?

Those advantages can be worth the latency cost. But they should be treated as a trade, not as a free upgrade.

The Investigation Pattern

When the microservices path became slower, the useful investigation was not to blame the network immediately.

We broke the request down by span and asked which parts were on the critical path.

• How much time was spent in the gateway?
• How much time was spent waiting for each downstream service?
• Did any service call happen sequentially when it could be parallel?
• Were retries happening during normal load?
• Were connection pools queueing requests?
• Were thread pools saturated?
• Was the payload larger than the operation required?

That investigation changed the conversation. The problem was no longer “microservices are slow.” The problem became “this request path has too many synchronous boundaries.”

What We Changed

The solution was not to collapse everything back into the monolith. It was to treat request-path design as a performance problem.

The most useful changes were architectural, not cosmetic.

• Removed service calls that existed only because the domain boundary was drawn too aggressively
• Moved non-critical work out of the synchronous request path
• Introduced explicit timeout budgets per dependency
• Stopped retrying operations that were not safe or useful to retry
• Reduced payload size between services
• Added tracing around outbound calls and connection pool wait time

The biggest improvement came from removing unnecessary synchronous hops. Tuning timeouts helped stability. Reducing payloads helped throughput. But the cleanest latency win was making the request path shorter.

Design Rules I Would Use Again

• Do not split a service boundary just because the code has a package boundary.
• Do not put every domain interaction on the user-facing critical path.
• Measure P95 and P99 before and after the split.
• Give every downstream call a timeout, retry policy, and owner.
• Prefer fewer synchronous hops for latency-sensitive APIs.
• Use events for work that does not need to finish before the response.
• Make observability part of the migration plan, not a cleanup task.

Key Insight

Microservices improve organizational scalability more directly than they improve request latency.

They can improve system scalability when the boundaries are right and the workload benefits from independent scaling. But if the request path becomes a chain of synchronous calls, latency usually gets worse before it gets better.

A microservice boundary is not just a code boundary. It is a runtime boundary.

That means every boundary needs to justify its cost.

Conclusion

The monolith was faster in the measured request path because it had fewer moving parts.

The microservices version gave us cleaner ownership and deployment flexibility, but it also introduced network hops, serialization, timeouts, retries, connection pools, and distributed debugging.

That trade-off can absolutely be worth it. But it should be measured, not assumed.

Before splitting a Spring Boot monolith, trace the request path you are about to create. Count the synchronous hops. Define latency budgets. Decide which work really needs to happen before the user gets a response.

The architecture diagram may look cleaner after the split. The latency chart might not.