API Slow? Database Fine — Real Performance Case Study

When an API becomes slow, the database usually gets blamed first. It is a reasonable instinct. Most backend bottlenecks eventually touch storage, indexes, locks, connection pools, or query plans.

But this case was different. PostgreSQL was healthy, the query plan was fine, and the direct database response time stayed around 40-50 ms. The slow part was hiding in the Java application layer.

The interesting lesson was not just that the API was slow. The lesson was that the average response time looked harmless while the tail latency told a very different story.

The Symptom

The API returned transaction records for an account. During normal testing, most requests looked fast. Some responses completed in only a few milliseconds, which made the endpoint appear healthy at first glance.

Under load, however, a small group of requests stretched close to two seconds. That is the kind of latency pattern that frustrates users and confuses engineers, because the system is not slow all the time. It is slow just often enough to be a production problem.

If I had only looked at the average, I could have closed the investigation too early. A 23 ms average sounds excellent. But a 1926 ms maximum response time says some requests were taking a completely different execution path or were paying a hidden cost.

The Initial Assumption

The endpoint shape looked like a classic database-performance issue. A request entered the Spring Boot API, went through the service layer, called a repository, and returned data from PostgreSQL.

Client
  ↓
API Gateway
  ↓
Spring Boot Controller
  ↓
Service Layer
  ↓
Repository (JPA)
  ↓
PostgreSQL

This is where many investigations go wrong. The query is visible, the table is large, and the database is easy to suspect. Application CPU work, object allocation, mapping, serialization, and accidental loops are less obvious unless you measure them directly.

Step 1: Verify the Database Directly

The first check was to isolate PostgreSQL from the API. Instead of testing through HTTP, I ran the query directly against the database and inspected the execution time.

EXPLAIN ANALYZE
SELECT *
FROM transaction_record
WHERE account_id = 100;

The result was consistent: the database returned in roughly 40-50 ms. That did not prove the application was clean, but it strongly suggested the database was not responsible for the two-second API responses.

At this point, the investigation changed direction. The question was no longer, “Why is PostgreSQL slow?” It became, “What happens after PostgreSQL returns?”

Step 2: Reproduce the Slow Path Under Load

The API was tested with JMeter using concurrent traffic. The dataset contained around 200,000 records, and the test ran on a Ryzen 5 5600X machine.

The load test was useful because it reproduced the real symptom: most requests were fast, but a smaller number became much slower. That pattern usually means you need to look at tail latency, not just throughput and averages.

Step 3: Split the Timing by Layer

The next step was simple but effective: log the database time separately from the application processing time. This is one of the fastest ways to stop guessing during a backend performance investigation.

long start = System.currentTimeMillis();

List<Transaction> data = repository.findByAccountId(id);
long dbTime = System.currentTimeMillis();

process(data);
long end = System.currentTimeMillis();

log.info("DB time={} ms", dbTime - start);
log.info("Processing time={} ms", end - dbTime);
log.info("Total time={} ms", end - start);

The logs made the problem obvious. PostgreSQL was consistently fast, while the application processing step was sometimes extremely expensive.

DB time = 19 ms
Processing time = 1112 ms
Total time = 1131 ms

DB time = 24 ms
Processing time = 1291 ms
Total time = 1315 ms

That changed the root-cause narrative completely. The database was not the bottleneck. The API was slow because the Java process was doing too much work after the database returned.

The Real Root Cause

The problematic path contained expensive application-side work. In the simplified example below, the service performs CPU-heavy computation during request processing. In real systems, the same pattern often appears as repeated mapping, nested loops, unnecessary enrichment calls, oversized DTO construction, or expensive serialization.

for (int i = 0; i < 10_000_000; i++) {
    Math.sqrt(i * id);
}

This kind of code can be deceptive because it does not look like I/O. There is no slow network call, no blocked database query, and no obvious external dependency. The request simply burns CPU inside the application.

Under light traffic, that cost may be hidden. Under 200 concurrent users, the cost becomes visible because CPU scheduling, allocation pressure, garbage collection, and thread contention amplify the slow path.

Why Only Some Requests Were Slow

One confusing part of this case was that not every request was slow. That is common in performance incidents. A backend does not need to be slow on every request to be unhealthy.

• Some requests hit larger result sets or more expensive transformation paths.
• Concurrent traffic increases CPU contention between request threads.
• Garbage collection can pause or delay unlucky requests.
• JIT compilation and warm-up can make early measurements noisy.
• Serialization cost grows with object graph size, not just row count.

This is why percentiles matter. P50 describes the typical request. P95 and P99 describe the experience of users who are unlucky enough to hit the expensive path. Production incidents often live in that gap.

The Fix

The optimization was not a database index. It was application cleanup. The heavy loop was removed, transformation logic was simplified, unnecessary work was taken out of the request path, and object creation was reduced.

• Remove CPU-heavy loops from synchronous request handling.
• Avoid repeated transformations over the same collection.
• Return only the fields the endpoint actually needs.
• Keep DTO mapping predictable and cheap.
• Measure serialization time when returning large payloads.

After the change, the database time remained about the same, but the API response time improved dramatically. That is exactly what you expect when the root cause is above the repository layer.

Performance Chart

What This Case Teaches

The main mistake would have been optimizing the database simply because the endpoint reads from PostgreSQL. The database was involved in the request, but it was not the source of the latency.

A better performance investigation separates each layer and measures the time spent in each one. Once you know the database is 40 ms and the total request is 1300 ms, the missing 1260 ms has to be explained somewhere else.

• Do not trust average latency alone.
• Measure database time and application processing time separately.
• Treat P95, P99, and max latency as first-class signals.
• Look for CPU-heavy work inside service methods.
• Remember that a fast query can still produce a slow API.

Final Thought

A slow API is not always a database problem. Sometimes PostgreSQL is doing its job, and the application is wasting the time afterward.

The most useful habit is to stop guessing early. Add timing around the repository call, around the service processing, and around serialization. Once the request is split into measurable parts, the real bottleneck has fewer places to hide.

FAQ

Why is the API slow if PostgreSQL is fast?

The database is only one segment of the request. Slow object mapping, JSON serialization, remote calls, connection waits, and thread contention can dominate total latency.

How should this be debugged in production?

Split timing by request stage: controller, service, database call, mapping, serialization, downstream calls, and queue or pool waits.

What metric matters most for users?

Use percentile latency such as P95 or P99 for the full endpoint, then correlate it with database timing and application-stage timings.