The Database Was Fast. Our Spring Boot API Still Ran Out of Memory.

Hibernate loaded 100,000 records into a managed object graph, turning a fast report query into an exhausted JVM heap.

The SQL query finished quickly.

The HTTP request did not.

Our Spring Boot report-preview endpoint kept working long after the database returned its result. Heap usage climbed, garbage collection became more frequent, and the service eventually crashed with an OutOfMemoryError.

At first, we suspected the database.

The real problem was inside the application.

Hibernate was converting approximately 100,000 transaction records into a large managed object graph. The preview endpoint and full report shared the same retrieval method, even though they had completely different latency and memory requirements.

The lasting lesson was not “never use JOIN FETCH.”

It was this:

Design retrieval boundaries around the workload, not around code reuse.

The Query Finished, but the Request Did Not

The endpoint prepared a preview before generating a transaction report.

Users selected a date range and filters. The application returned matching transaction details so they could verify the data before starting the complete report.

The endpoint looked like a normal read operation:

GET /api/reports/transactions/preview
    ?start=2026-06-01T00:00:00Z
    &end=2026-07-01T00:00:00Z

It worked for small date ranges.

For a busy month, the query returned approximately 100,000 transactions. The API became slow and unstable.

Eventually, the application logged an error similar to this:

java.lang.OutOfMemoryError: Java heap space
    at org.hibernate.sql.results.graph.entity...
    at org.hibernate.sql.results.jdbc.internal...

The database execution plan looked healthy. Indexes were used, there were no significant locks, and running the SQL directly completed much faster than the API request.

The database had done its work.

The missing time was between SQL completion and HTTP response completion.

We Initially Blamed the Database

A large report query naturally puts the database under suspicion.

We checked the usual areas:

• Query execution plan
• Missing or unused indexes
• Table scans
• Database locks
• Connection-pool usage
• Network latency
• Query timeouts
• Number of rows returned

These checks were necessary. A bad execution plan can make a reporting endpoint slow.

But the evidence did not support that explanation.

A simplified trace showed the real shape of the request:

Database execution:        420 ms
JDBC result reading:     2,800 ms
Hibernate hydration:    10,900 ms
DTO/response mapping:    2,700 ms
JSON serialization:      3,100 ms
GC pauses:               1,900 ms
Total request time:     21,820 ms

The exact values differ between systems, but the pattern was clear.

SQL execution represented only a small part of the request. Most of the time was spent after the database returned the result set.

Optimizing the SQL alone would not solve the incident.

What Hibernate Does After SQL Returns

A SQL client displays rows.

Hibernate does much more.

After the JDBC driver receives the result set, Hibernate may need to:

1. Read every column from every row.
2. Convert SQL values into Java values.
3. Create entity instances.
4. Resolve entity identifiers.
5. Match repeated rows to existing entities.
6. Create associated entities.
7. Build collections.
8. Register managed objects in the persistence context.
9. Maintain dirty-checking state.
10. Pass the resulting graph to mapping or serialization code.

One database row does not become one lightweight Java object.

A row can contribute to several objects, references, collection entries, and internal Hibernate structures.

Our simplified entity model looked like this:

Transaction
|-- Customer
|   `-- CustomerAddress
|-- PaymentMethod
|-- Merchant
`-- List<TransactionItem>
    `-- Product

The repository fetched several relationships because the same data was also used by full report generation.

A simplified query looked like this:

@Query("""
    select distinct t
    from Transaction t
    join fetch t.customer c
    join fetch t.merchant m
    left join fetch t.paymentMethod pm
    left join fetch t.items i
    left join fetch i.product p
    where t.transactionTime >= :start
      and t.transactionTime < :end
    order by t.id
    """)
List<Transaction> findTransactionsForReport(
        Instant start,
        Instant end
);

There is nothing inherently wrong with JOIN FETCH.

It can be the right solution when the application intentionally needs related entities and wants to avoid N+1 queries.

The problem was that this method had no retrieval limit. It also fetched a one-to-many collection.

If one transaction contained five items, 100,000 transactions could produce hundreds of thousands of SQL rows before Hibernate reconstructed the distinct transaction entities.

The result-set row count and final entity count were both large, but they were not the same number.

The Persistence Context Increased the Memory Cost

Hibernate stores managed entities in the persistence context for the duration of the transaction or session.

Conceptually, it maintains structures similar to this:

Persistence Context
|-- Transaction entity 1
|-- Transaction entity 2
|-- Transaction entity 3
|-- ...
|-- Transaction entity 100,000
|-- Customer entities
|-- Merchant entities
|-- Product entities
|-- Collection wrappers
`-- Entity state snapshots

These objects remain reachable while Hibernate processes the request.

The garbage collector cannot release reachable objects, even when heap usage is high. It can only spend more time examining and promoting them.

That produces a common failure pattern:

Request starts
Heap rises quickly
Young GC runs repeatedly
Objects survive into old generation
Full GC becomes more frequent
Request still holds the object graph
Heap cannot recover
OutOfMemoryError occurs

This was not a traditional memory leak where objects remained forever.

One request simply required more live memory than the application could safely provide.

Using @Transactional(readOnly = true) is still appropriate for read paths and may reduce unnecessary persistence work, depending on the Hibernate configuration.

It does not make an unbounded entity graph safe.

What the Heap Evidence Showed

When operationally safe, capture a heap dump before restarting an affected JVM.

For example:

java \
  -XX:+HeapDumpOnOutOfMemoryError \
  -XX:HeapDumpPath=/var/log/report-service/heapdump.hprof \
  -jar report-service.jar

The destination must have enough disk space, and the process or container must be allowed to write there. Heap dumps can be nearly as large as the configured JVM heap.

They can also contain sensitive business data. Restrict access and define a short retention period.

A simplified object histogram from this incident pattern might look like this:

 num       instances          bytes  class
---------------------------------------------------------
   1         620,000      148,800,000  Object[]
   2         510,000      122,400,000  TransactionItem
   3         100,000       88,000,000  Transaction
   4         100,000       51,200,000  PersistentBag
   5         340,000       43,520,000  HashMap$Node
   6         180,000       31,680,000  Product
   7         100,000       24,000,000  EntityEntry

The exact class names depend on the Hibernate version and entity model.

What matters is the ownership path.

If the request thread, Hibernate session, or persistence context retains a large graph of report entities, the database is no longer the main suspect.

Useful investigation tools include:

• Java Flight Recorder
• Java Mission Control
• Eclipse Memory Analyzer
• jcmd GC.class_histogram
• jcmd GC.heap_dump
• APM distributed traces
• Hibernate statistics
• Garbage-collection logs

The Root Cause: One Retrieval Shape for Two Workloads

The immediate cause was clear:

One unbounded request loaded approximately 100,000 transactions and their relationships into Hibernate.

But the deeper problem was architectural.

The preview and full report reused the same repository method.

At the business level, both operations used the same transaction data. Reusing the method therefore looked reasonable.

Operationally, they were different workloads.

A preview is interactive:

• It should respond quickly.
• It should display a limited number of rows.
• It needs predictable memory usage.
• Multiple users may call it concurrently.
• It operates within an HTTP timeout.

A full report is exhaustive:

• It must process every matching record.
• It may take minutes.
• It needs progress tracking.
• It needs retry and failure handling.
• It should process records in bounded chunks.

The data was shared.

The resource requirements were not.

By reusing one retrieval method, the preview inherited the full report’s data volume. It performed exhaustive work to display a small screen.

The code looked reusable, but the workloads were incompatible.

Fix Part 1: Build a Bounded Preview

The preview did not need complete transaction entities.

It needed only:

• Transaction ID
• Transaction time
• Amount
• Status

We created a DTO specifically for the screen:

public record TransactionPreview(
        Long id,
        Instant transactionTime,
        BigDecimal amount,
        String status
) {
}

This object is not a managed JPA entity.

Hibernate does not need to create the complete transaction graph, register relationships, or maintain dirty-checking state for it.

Preview DTOs should remain close to their use case. They do not need every field that might eventually appear in the complete report.

Select Only the Required Fields

The repository now uses a DTO projection:

@Query("""
    select new com.example.report.TransactionPreview(
        t.id,
        t.transactionTime,
        t.amount,
        t.status
    )
    from Transaction t
    where t.transactionTime >= :start
      and t.transactionTime < :end
    order by t.id
    """)
Page<TransactionPreview> findPreview(
        @Param("start") Instant start,
        @Param("end") Instant end,
        Pageable pageable
);

The query no longer loads Customer, Merchant, TransactionItem, or Product entities.

It also returns a bounded page instead of an unbounded list.

DTO projection reduces ORM overhead, but it does not make an unlimited query safe. Returning 100,000 DTOs can still consume substantial memory and produce a large JSON response.

Projection and pagination solve different parts of the problem.

Enforce Limits on the Server

The client should not control the resource boundary.

A request such as size=100000 must not be allowed to recreate the original failure.

The service enforces a maximum page size:

private static final int DEFAULT_PAGE_SIZE = 50;
private static final int MAX_PAGE_SIZE = 100;

@Transactional(readOnly = true)
public Page<TransactionPreview> findPreview(
        Instant start,
        Instant end,
        int requestedPage,
        Integer requestedSize) {

    int page = Math.max(requestedPage, 0);

    int size = requestedSize == null
            ? DEFAULT_PAGE_SIZE
            : Math.min(Math.max(requestedSize, 1), MAX_PAGE_SIZE);

    Pageable pageable = PageRequest.of(
            page,
            size,
            Sort.by(Sort.Direction.ASC, "id")
    );

    return repository.findPreview(start, end, pageable);
}

Pagination is not only a user-interface feature.

Here, it places an upper limit on database rows, Java objects, mapping work, serialized bytes, and request memory.

The date range should also be validated. A page limit controls returned rows, but a broad date range can still make the count query expensive.

Return Only the Pagination Data the Screen Needs

The API can return rows with navigation metadata:

{
  "content": [
    {
      "id": 981501,
      "transactionTime": "2026-06-15T09:14:22Z",
      "amount": 1250.00,
      "status": "COMPLETED"
    }
  ],
  "page": 0,
  "size": 100,
  "totalElements": 100000,
  "totalPages": 1000,
  "hasNext": true
}

A Page normally requires a count query to calculate totalElements and totalPages.

If the screen only needs to know whether another page exists, return a Slice instead:

Slice<TransactionPreview> findPreview(
        Instant start,
        Instant end,
        Pageable pageable
);

This can avoid an expensive count over a large filtered dataset.

Use Page only when the exact total is a genuine screen requirement.

Do Not Paginate a Collection Fetch Join Blindly

Paginating a query that fetch-joins a collection is dangerous.

For example:

select distinct t
from Transaction t
left join fetch t.items

The database applies pagination to SQL rows, not necessarily distinct Transaction entities.

One transaction with many items may occupy several rows. Depending on the query and Hibernate version, pagination can produce inconsistent page sizes or cause Hibernate to paginate in memory.

A common warning is:

HHH90003004: firstResult/maxResults specified with collection fetch;
applying in memory

Do not ignore that warning.

In non-production environments, configure Hibernate to fail fast:

spring.jpa.properties.hibernate.query.fail_on_pagination_over_collection_fetch=true

For paginated screens, safer options include:

• Use a DTO projection without collection fetches.
• Page root IDs first, then load details in a second bounded query.
• Use batch fetching for selected relationships.
• Create a read-optimized query for the screen.

JOIN FETCH remains useful, but its retrieval shape must match the workload.

Fix Part 2: Move Full Reports Outside the HTTP Request

The complete report still needed every matching transaction.

We did not solve that requirement by pretending the dataset was smaller. We changed how the work was executed.

The preview became a bounded synchronous read.

Full report generation became an asynchronous batch process:

PENDING -> RUNNING -> COMPLETED
                 `-> FAILED

A user starts the report:

POST /api/reports/transactions
Content-Type: application/json

{
  "start": "2026-06-01T00:00:00Z",
  "end": "2026-07-01T00:00:00Z"
}

The API creates a report request and returns immediately:

HTTP/1.1 202 Accepted

{
  "reportId": "8f43b7c1-280a-41fd-9d49-a6422d87bcc4",
  "status": "PENDING",
  "statusUrl": "/api/reports/8f43b7c1-280a-41fd-9d49-a6422d87bcc4"
}

The client can poll the status endpoint or receive progress through another notification mechanism.

When generation finishes, the status response includes the report file location.

Make the Report Lifecycle Visible

A simple status model exposes the lifecycle:

public enum ReportStatus {
    PENDING,
    RUNNING,
    COMPLETED,
    FAILED
}

The report request should also store operational information:

report_id
requested_by
requested_at
started_at
completed_at
status
processed_rows
last_processed_id
output_location
failure_message

This supports monitoring, controlled retries, and investigation.

Moving work to a background job helps only when the team can see whether that job is progressing.

Process Full Reports in Bounded Batches

A batch job must not replace one unbounded query with another.

The complete report can use a read-specific projection and keyset pagination:

public record ReportRow(
        Long id,
        Instant transactionTime,
        BigDecimal amount,
        String status,
        String customerName
) {
}

@Query("""
    select new com.example.report.ReportRow(
        t.id,
        t.transactionTime,
        t.amount,
        t.status,
        c.name
    )
    from Transaction t
    join t.customer c
    where t.id > :lastId
      and t.transactionTime >= :start
      and t.transactionTime < :end
    order by t.id
    """)
List<ReportRow> findNextBatch(
        @Param("lastId") Long lastId,
        @Param("start") Instant start,
        @Param("end") Instant end,
        Pageable pageable
);

The processor requests a limited number of records at a time:

long lastId = 0L;
int batchSize = 1_000;

while (true) {
    Pageable limit = PageRequest.of(0, batchSize);

    List<ReportRow> rows = repository.findNextBatch(
            lastId,
            reportStart,
            reportEnd,
            limit
    );

    if (rows.isEmpty()) {
        break;
    }

    reportWriter.write(rows);

    lastId = rows.get(rows.size() - 1).id();
    reportProgress.update(lastId, rows.size());
}

At most 1,000 report rows are held by this loop at a time, excluding buffers owned by the writer.

The report writer must also flush incrementally. Collecting every generated row in a list before writing the file would recreate the memory problem in another component.

Why Keyset Pagination Helps

Offset pagination is easy to implement:

ORDER BY id
OFFSET 90000 ROWS
FETCH NEXT 1000 ROWS ONLY

Large offsets can become expensive because the database still needs to locate or skip the earlier rows.

Keyset pagination continues from the last processed key:

WHERE id > :lastId
ORDER BY id
FETCH FIRST 1000 ROWS ONLY

This is usually more stable for large ordered scans.

The selected key must provide deterministic ordering. If the report is ordered by a non-unique timestamp, include a unique tie-breaker:

WHERE transaction_time > :lastTime
   OR (transaction_time = :lastTime AND id > :lastId)
ORDER BY transaction_time, id

For reports that read changing data, establish a stable cutoff or database snapshot. Otherwise, inserted or updated rows can move between pages while the report is running.

Clear Managed Entities Between Chunks

DTO projections avoid managed entities and are usually preferable for read-heavy report generation.

If a batch must update or process entities, clear the persistence context after every chunk:

@Service
public class TransactionChunkProcessor {

    private final TransactionRepository repository;
    private final EntityManager entityManager;

    public TransactionChunkProcessor(
            TransactionRepository repository,
            EntityManager entityManager) {
        this.repository = repository;
        this.entityManager = entityManager;
    }

    @Transactional
    public void processChunk(List<Long> transactionIds) {
        List<Transaction> transactions =
                repository.findAllById(transactionIds);

        for (Transaction transaction : transactions) {
            process(transaction);
        }

        entityManager.flush();
        entityManager.clear();
    }
}

Without clear(), managed entities can accumulate when chunks share one long-lived persistence context.

Spring Batch provides chunk-oriented processing, transaction boundaries, restart information, and job metadata.

Those features are useful, but Spring Batch does not automatically make an unbounded reader memory-safe. The reader and writer still need bounded behavior.

Use Diagnostics Carefully

Hibernate statistics can be enabled temporarily during an investigation:

spring.jpa.properties.hibernate.generate_statistics=true
logging.level.org.hibernate.stat=DEBUG

SQL logging can also help in development:

logging.level.org.hibernate.SQL=DEBUG

Avoid permanently enabling detailed SQL and bind-parameter logging in a high-volume production system. It can increase I/O, expose sensitive values, and make the original performance problem worse.

GC logging provides stronger memory evidence:

java \
  -Xms2g \
  -Xmx2g \
  -Xlog:gc*:file=/var/log/report-service/gc.log:time,uptime,level,tags \
  -XX:+HeapDumpOnOutOfMemoryError \
  -XX:HeapDumpPath=/var/log/report-service/heapdump.hprof \
  -jar report-service.jar

The JVM heap must remain below the container memory limit.

Thread stacks, metaspace, direct buffers, JIT-compiled code, and other native allocations also need memory. Setting -Xmx2g inside a container limited to exactly 2 GB leaves no room for those allocations.

Increasing the heap may delay the crash.

It does not correct an unbounded retrieval boundary.

Verify the Fix End to End

The new design needs more evidence than one successful request.

First, verify the preview:

curl "http://localhost:8080/api/reports/transactions/preview?page=0&size=100&start=2026-06-01T00:00:00Z&end=2026-07-01T00:00:00Z"

Confirm that:

• No more than 100 rows are returned.
• The selected fields match the screen.
• Pagination metadata is correct.
• A requested size above the maximum is reduced or rejected.
• The query does not fetch unnecessary collections.

Next, test concurrency.

One preview request may look safe while 50 concurrent requests create enough allocation pressure to destabilize the JVM.

Measure:

• Response-time percentiles
• Request allocation rate
• Peak heap usage
• Full GC count and duration
• Database connections
• Rows returned per request
• Response size
• Error rate

Finally, run the full report against realistic data volume.

Confirm that:

• Every expected transaction appears exactly once.
• Processing memory stays within a stable range.
• Progress is stored.
• Failed work can be retried safely.
• The output writer does not retain all rows.
• Concurrent data changes cannot make pagination inconsistent.

Before and After

After separating the workloads, the design looked like this:

Representative measurements might look like:

Preview response time:     18 seconds -> 450 milliseconds
Peak request allocation:       1.4 GB -> 20 MB
Records loaded per request:   100,000 -> 100

These numbers should come from actual tracing and profiling before publication or before being used as performance targets.

The important outcome is not one impressive latency number.

It is predictable resource usage under realistic concurrency.

The Trade-Off: Less Reuse, Better Boundaries

The new design contains more use-case-specific code.

The preview has its own DTO and repository query. The full report has a separate reader, writer, status model, and monitoring path.

That is less reuse than returning the same entity graph everywhere.

But code reuse is not automatically beneficial when it combines workloads with different constraints.

A shared query can hide:

• Different latency expectations
• Different memory limits
• Different transaction lifetimes
• Different failure-handling requirements
• Different consistency requirements
• Different data volumes

A small amount of duplication made the operational boundaries clear.

We could still reuse filters, validation rules, date-window calculation, and business definitions. We did not need to reuse the same physical retrieval method.

Production Warnings

Do not return JPA entities directly from a public API.

Entity serialization can trigger lazy loading, expose fields unintentionally, create recursive relationships, and couple the API contract to persistence concerns.

Do not assume DTO projection makes unlimited retrieval safe.

A million DTOs are still a million Java objects.

Do not trust the client’s page size.

Server-side limits protect the application from mistakes and abusive requests.

Do not paginate collection fetch joins without understanding where Hibernate applies the limit.

Inspect the generated SQL and fail fast on in-memory pagination during testing.

Do not move exhaustive work to a batch job and then load everything at once.

Batch processing still needs chunks, pages, cursors, streams, or stable key ranges.

Do not test only one request.

Concurrent previews multiply allocation, database traffic, serialization work, and connection usage.

Lessons Learned

• Fast SQL does not guarantee a fast API. Measure JDBC reading, ORM hydration, mapping, serialization, and garbage collection.
• A database row does not represent the full memory cost of the Java object graph created from it.
• JOIN FETCH was not inherently harmful. The unbounded retrieval shape was the problem.
• Pagination is a backend resource control, not only a screen-navigation feature.
• Shared data does not require a shared retrieval strategy.
• Moving work to a batch job helps only when batch processing is also bounded.
• Heap dumps and traces are more reliable than guessing which component is slow.

Conclusion

The database was fast because it was not doing the work that exhausted the JVM heap.

After SQL completed, Hibernate still had to read rows, hydrate entities, resolve relationships, build collections, maintain persistence-context state, and prepare the response.

Our preview and complete report read the same transactions.

But they were never the same operation.

We fixed the incident by giving each workload its own retrieval boundary. The preview now uses a paginated DTO projection with a server-enforced limit. The full report runs asynchronously and processes data in bounded batches with explicit progress and retry state.

The architecture improved when we stopped optimizing for repository-method reuse and started designing for predictable resource usage.