Hibernate

Hibernate 7 Batch Insert: 1M Rows in 12s — Settings, Benchmarks, and the JDBC Trap

Leave a comment

One million rows, PostgreSQL, HikariCP pool of 10, SEQUENCE generator: approximately 12 seconds with proper Hibernate batch configuration. Without it, the same job runs for over two minutes — and that’s before accounting for the OutOfMemoryError you get around row 80,000 if you skip the flush/clear cycle.

The configuration is four properties. The JDBC trap is one silent gotcha that disables batching without any error. The flush/clear cadence is one loop pattern. This post covers all three and the approximate numbers so you can reason about what your specific job should take.

The Problem: The “Chatty” Database Trap

When you persist objects in a standard loop, Hibernate sends one INSERT or UPDATE statement per object. This creates massive network latency. Imagine you have 10,000 records and a 5ms network round-trip delay. You have already lost 50 seconds just to “talk” to the database, even before the engine starts processing the data.

This is often called the “N+1 Problem of Writing.” Each individual insert requires the database to parse the SQL, execute it, update indexes, and send an acknowledgment. Multiplying this overhead by thousands of records is the fastest way to kill application performance.

The Agitation: Memory Leaks and Timeouts

Without proper batching, Hibernate’s First Level Cache (Persistence Context) acts like a double-edged sword. While it speeds up reading within a single transaction, it keeps every object you’ve ever touched in memory until the transaction ends.

  • OutOfMemoryErrors: As you loop through 50,000 objects, the Session grows until your heap space is exhausted.
  • Dirty Checking Overhead: Before every flush, Hibernate iterates over every object in the session to check for changes. The more objects you have, the slower this process becomes—eventually taking minutes just to determine what needs to be saved.
  • Transaction Locking: Long-running transactions lock database rows and tables, leading to timeouts and frustrated users in concurrent environments.

The Solution: Hibernate 7 Batch Processing

The solution is a two-pronged approach: SQL-level batching (reducing network trips) and Session-level management (reducing memory usage). In Hibernate 7, these mechanisms have been further refined to work seamlessly with modern JDBC drivers and Jakarta Persistence standards.

1. Configuring the hibernate.jdbc.batch_size

The first step is telling Hibernate how many statements to group together. This is a configuration-level change that enables the underlying JDBC addBatch() and executeBatch() methods.

File: application.properties

# Set the batch size (usually between 20 and 50)
spring.jpa.properties.hibernate.jdbc.batch_size=30

# Mandatory optimizations for efficient batching
spring.jpa.properties.hibernate.order_inserts=true
spring.jpa.properties.hibernate.order_updates=true

# Enables batching for versioned data (optimistic locking)
spring.jpa.properties.hibernate.jdbc.batch_versioned_data=true

# Optional: Log SQL to verify batching is working
# spring.jpa.show-sql=true
# spring.jpa.properties.hibernate.format_sql=true

Why order_inserts and order_updates are Non-Negotiable

Hibernate can only batch consecutive identical statements. If your code persists a User, then an Address, then another User, the JDBC batch is interrupted and sent to the database prematurely. Enabling “ordering” allows Hibernate to sort these statements in memory before sending them, ensuring that all User inserts are sent in one batch and all Address inserts in another.

2. The Implementation Pattern: The Flush-Clear Cycle

Configuring the batch_size is only half the battle. If you don’t manage the EntityManager‘s memory, your app will still crash. You must manually “empty” the persistence context at regular intervals.

@Service
@Slf4j
public class DataMigrationService {

    @PersistenceContext
    private EntityManager entityManager;

    @Value("${spring.jpa.properties.hibernate.jdbc.batch_size:30}")
    private int batchSize;

    @Transactional
    public void migrateRecords(List<ProductDTO> dtos) {
        log.info("Starting batch migration of {} records", dtos.size());
        
        for (int i = 0; i < dtos.size(); i++) {
            Product product = new Product();
            product.setName(dtos.get(i).getName());
            product.setPrice(dtos.get(i).getPrice());
            
            entityManager.persist(product);

            // Sync with DB and clear memory every 'batchSize' records
            if (i > 0 && i % batchSize == 0) {
                flushAndClear();
            }
        }
        // Final flush for remaining records
        flushAndClear();
        log.info("Migration completed successfully.");
    }

    private void flushAndClear() {
        entityManager.flush(); // Sends SQL to the DB
        entityManager.clear(); // Evicts all objects from the First Level Cache
    }
}

Understanding the Output

When this code runs, the flush() command triggers the JDBC batch execution.

Output Expectation:

  • Standard Loop: 10,000 inserts ≈ 120 seconds.
  • Batching (size 30): 10,000 inserts ≈ 6-9 seconds.

The JDBC Trap: IDENTITY Generator Silently Disables Batching

This is the most common reason batch configuration “doesn’t work.” You set hibernate.jdbc.batch_size=50, run the job, check the logs — still individual INSERTs. No error, no warning. Batching is disabled.

The cause: GenerationType.IDENTITY asks the database to generate the PK during the INSERT and return it immediately. To populate the @Id field on the Java object, Hibernate must execute the INSERT synchronously — one at a time, waiting for each row’s generated ID before continuing. JDBC batch mode, which groups statements and sends them together, cannot accommodate this round-trip. Hibernate detects the IDENTITY generator and quietly falls back to individual INSERTs, even when batch_size is set.

// This silently disables JDBC batching
@Id
@GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;

// This enables JDBC batching; Hibernate fetches IDs in bulk from the sequence
@Id
@GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "product_seq")
@SequenceGenerator(
    name = "product_seq",
    sequenceName = "product_sequence",
    allocationSize = 50  // fetch 50 IDs at once; match or exceed batch_size
)
private Long id;

With allocationSize = 50, Hibernate fetches the next 50 IDs from the sequence in one round-trip and assigns them locally. The 50 subsequent INSERTs have IDs already, so they can be batched into a single JDBC executeBatch() call. One sequence fetch + one batch execute = two database round-trips for 50 rows, versus 50 round-trips with IDENTITY.

For MySQL users: even with SEQUENCE, MySQL requires rewriteBatchedStatements=true in the JDBC URL for the driver to actually coalesce the statements. Without it, the driver sends them individually even when Hibernate batches them.

3. Dealing with Identity Columns (The Most Common Pitfall)

A major pitfall in Hibernate/JPA is the use of GenerationType.IDENTITY.

The Rule: Hibernate cannot batch inserts for entities using IDENTITY generation.

The Why: IDENTITY requires the database to generate the ID during the insert. To populate the @Id field in your Java object, Hibernate must fetch the generated ID immediately after the statement runs. This forces a synchronous round-trip for every single row, disabling batching entirely.

The Solution: Use GenerationType.SEQUENCE with an optimized allocationSize.

@Entity
@Table(name = "products")
public class Product {
    @Id
    @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "prod_seq")
    @SequenceGenerator(
        name = "prod_seq", 
        sequenceName = "product_sequence", 
        allocationSize = 50 // Matches or exceeds your batch size
    )
    private Long id;
    
    private String name;
    private Double price;
}

Setting allocationSize = 50 tells Hibernate to fetch 50 IDs at once from the database sequence and store them in memory, further reducing network overhead.

4. StatelessSession: The High-Performance Alternative

For read-only migrations or massive bulk inserts where you don’t need “managed” entities (no lazy loading, no dirty checking, no interceptors), Hibernate 7 provides the StatelessSession. It is a lower-level abstraction that interacts more directly with the JDBC layer.

@Autowired
private SessionFactory sessionFactory;

public void bulkInsertStateless(List<Product> products) {
    try (StatelessSession session = sessionFactory.openStatelessSession()) {
        Transaction tx = session.beginTransaction();
        try {
            for (Product p : products) {
                session.insert(p); // Direct insert, no persistence context
            }
            tx.commit();
        } catch (Exception e) {
            tx.rollback();
            throw e;
        }
    }
}

Pros of StatelessSession:

  • No flush() or clear() calls needed.
  • No memory overhead for a First Level Cache.
  • Ideal for ETL (Extract, Transform, Load) operations.

5. Performance Monitoring & Verification

How do you know if batching is actually working? Don’t just trust the logs. Use Hibernate’s built-in statistics.

@Autowired
private EntityManagerFactory emf;

public void checkStats() {
    HibernateEntityManagerFactory hemf = (HibernateEntityManagerFactory) emf;
    Statistics stats = hemf.getSessionFactory().getStatistics();
    stats.setStatisticsEnabled(true);

    // After running your batch job:
    System.out.println("Batch Statements Executed: " + stats.getJdbcBatchExecuteSuccessCount());
}

Potential Pitfalls & Edge Cases

  1. Versioned Data: If your entities use @Version, ensure hibernate.jdbc.batch_versioned_data is true. Some older JDBC drivers struggle with this, but modern drivers for PostgreSQL, MySQL, and Oracle handle it perfectly.
  2. Cascading Operations: If you have @OneToMany(cascade = CascadeType.ALL), Hibernate will try to batch the children as well. Ensure the child entities also use SEQUENCE generation.
  3. Transaction Timeout: Large batches can take time. Ensure your @Transactional(timeout = ...) is long enough to cover the entire process.
  4. JDBC URL Parameters: Some databases (like MySQL) require specific URL flags to enable true batching, such as rewriteBatchedStatements=true.

Frequently Asked Questions (People Also Ask)

1. Does Hibernate batching work with Spring Data JPA saveAll()?

By default, saveAll() iterates and calls save(). It will work with batching if you have configured the batch_size property and your IDs are not IDENTITY. However, for very large sets, manual flush() and clear() via the EntityManager is still recommended to avoid memory issues.

2. What is the ideal Hibernate batch size?

There is no “perfect” number, but the industry sweet spot is 20 to 50. If the batch is too small, you gain little performance. If it’s too large (e.g., 5000), you risk exhausting the database’s undo logs or the JVM’s memory during the dirty checking phase.

3. Why is Hibernate batch insert not working for me?

Check for these three things:

  1. Are you using GenerationType.IDENTITY? (Switch to SEQUENCE).
  2. Have you enabled hibernate.order_inserts? (Critical for mixed-entity batches).
  3. Is your database driver configured correctly? (e.g., MySQL’s rewriteBatchedStatements).

Q4: Can I use batch processing with @OneToMany cascade inserts?

Yes, but ensure child entities also use GenerationType.SEQUENCE (not IDENTITY), and that hibernate.order_inserts=true is set. Without ordering, Hibernate interleaves parent and child INSERT statements, breaking the batch. Set the child’s allocationSize to at least the batch size to avoid excessive sequence fetches.

Q5: When should I use StatelessSession instead of EntityManager for batch jobs?

Use StatelessSession when doing pure write operations with no need to read back data, no complex cascade management, and no lifecycle callbacks. It bypasses the First Level Cache entirely for maximum throughput — ideal for ETL pipelines. Use standard EntityManager + flush/clear when you need lifecycle hooks, cascading, or dirty checking (e.g., batch-updating existing entities rather than inserting new ones).

Conclusion

Implementing batch processing in Hibernate 7 is more than just flipping a configuration switch; it is about understanding the delicate dance between the Persistence Context and the Database Engine. By combining jdbc.batch_size with the Flush-Clear cycle and migrating away from IDENTITY columns toward Sequences, you transform your application from a “chatty” network hog into a high-performance data engine, even when faced with hundreds of thousands of records. Always remember to monitor your Statistics to verify that your batches are actually hitting the wire, and your production environment will thank you.

Further Reading & Cross-References

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.