Cache performance degradation after migration from Hibernate 5.6 > 6.6

Hey, Team

After migrating from Hibernate 5.6 > 6.6 version we faced performance degradation when retrieving data from cache for JPQL Queries that return a List, regardless of whether the result is empty or not.

In both Hibernate versions, we use the same version of Ehcache 3 provider. All other stuff, like Java version, DB version, etc., is the same.

We used JProfiler to detect that issue, also we enabled Hibernate statistics to be sure that the L2 cache works properly.

We started from Load Tests on our application, but in the simplest variant, we were able to recreate the issue on a simple controller just to run some JPQL Query (that returns a list) in a loop, e.g., with 1000 iterations.

In the 5.6 query.getResultList() takes 30-50 nanoseconds

In the 6.6 query.getResultList() takes 300-500 nanoseconds
This is 10 times slower

For query.getSingleResult(), I have not faced the above behavior.

Is there any explanation for such performance degradation?
Let me know if you need some additional data from our side.

Thank you!

We have a bunch of benchmarks that we used to verify the performance of Hibernate ORM 6, but I haven’t seen such a slowdown before.

If you want us to look into it, I would suggest you to maybe create a PR against that hibernate-orm-benchmarks repository that reproduces the problem.

Hello,

I was able to recreate this performance issue in test case template .

On the “test case template, the JProfiler shows that in 6.6, the fetch operation from the query cache for the query.getResultList() performs 2 times slower than in the 5.6 version.

In the UnitTest the overall difference ~50%, but it is difficult to isolate only the fetch from cache operation.

On my project, the difference is 10 times slower.

The steps to recreate:

  1. Use Jcache and Ehcache
  2. Enable L2 cache and query caching
  3. Create a simple Entity
  4. In the test method, persist this entity and then fetch it in the query in the loop (e.g. 10k times)

Here are my classes/methods. I hope this will help to recreate the issue:

@Entity
@Cacheable
public class Department {

    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    private String name;

    // Getters and Setters
    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

}
@Test
void hhh123Test() throws Exception {
   EntityManager entityManager = entityManagerFactory.createEntityManager();
   entityManager.getTransaction().begin();

   Department department = new Department();;
   department.setName("name");
   entityManager.persist(department);
   entityManager.getTransaction().commit();

   entityManager.getTransaction().begin();
   long startTime = System.currentTimeMillis();

   for (int i = 1; i <= 10000; i++) {
      List<Department> departments = readAllDepartments(entityManager);
   }

   long endTime = System.currentTimeMillis();
   long timeElapsed = endTime - startTime;

   System.out.println("timeElapsed: " + timeElapsed);
   entityManager.getTransaction().commit();

   entityManager.close();
}

public List<Department> readAllDepartments(EntityManager entityManager) {

   TypedQuery<Department> query = entityManager.createQuery(
         "SELECT d FROM Department d",
         Department.class
   );

   query.setHint(QueryHints.HINT_CACHEABLE, true);
   query.setHint(QueryHints.HINT_CACHE_REGION, "query.Department");

   return query.getResultList();
}

We have benchmarks that exercise the query cache with an entity that even contains an association: hibernate-orm-benchmark/basic/src/test/java/org/hibernate/benchmark/queryl2hit/QueryCacheEntityWithAssociation.java at main · hibernate/hibernate-orm-benchmark · GitHub

If you can reproduce the problem with a similar benchmark, please open a PR against that repo and then we will look into it.

Hi, Team! (cc:@beikov)

We came back to the performance tests for our application after
Hibernate 5.6.15 > Hibernate 7.2.7 migration.

We used the project/test that you suggested
hibernate-orm-benchmark/basic/src/test/java/org/hibernate/benchmark/queryl2hit/QueryCacheEntityWithAssociation.java at main · hibernate/hibernate-orm-benchmark · GitHub

And the results are the same as we had for the 5.6.15 > 6.6.x migration half a year ago:

50% performance degradation for getResultList() operation for cached entity

On our real data, the performance degradation for such operations is up to 400%

I don’t have the permissions to create a PR in that repo, but my code changes are super straightforward.

  1. Increased the count of iterations from 1000 to 10000 - to be more precise
  2. Added “Time elapsed” only for the loop where call getResultList() - to isolate the getResultList() operation

Here is the 'Time elapsed" for 5.6.x, 6.6.x, 7.2.x Hibernate versions

5.6.15

Time elapsed: 125
Time elapsed: 68
Time elapsed: 57
Time elapsed: 54
Time elapsed: 47

6.6.44

Time elapsed: 200
Time elapsed: 106
Time elapsed: 92
Time elapsed: 80
Time elapsed: 75

7.2.7

Time elapsed: 229
Time elapsed: 105
Time elapsed: 88
Time elapsed: 80
Time elapsed: 75

@Benchmark
public void single(Blackhole bh, EventCounters counters) {
    em = entityManagerFactory.createEntityManager();
    em.getTransaction().begin();
    final Author author = em.createQuery( "from Author", Author.class ).setMaxResults( 1 ).getSingleResult();
       //add start time
       long startTime = System.currentTimeMillis();
       //increase count of iterations to 10000
       for (int i = 0; i < 10000; i++) {
       final List<Book> books = em.createQuery( "from Book b where b.author = :author", Book.class )
             .setParameter( "author", author )
             .setHint( "org.hibernate.cacheable", "true" )
             .getResultList();
       for ( Book book : books ) {
          if ( bh != null ) {
             bh.consume( book );
          }
       }
       if ( counters != null ) {
          counters.books += books.size();
       }
    }
       long endTime = System.currentTimeMillis();
       long timeElapsed = endTime - startTime;
       //show Time elapsed
       System.out.println("Time elapsed: " + timeElapsed);
    em.getTransaction().commit();
    em.close();
}

Maybe @mbladel wants to look into that as he recently worked on some caching related bug fixes. Some degradation is certainly possible, because Hibernate ORM 6 does more than ORM 5 in terms of fetching and has other defaults, but I agree that this looks a bit odd.

You mentioned that
"and has other defaults"

Any suggestions on what we can change to make 6.x defaults similar to 5.x version?

Thank you

Well, Hibernate ORM 5 and before by default behaved as if you configure hibernate.cache.query_cache_layout=shallow: Hibernate ORM User Guide

But then Hibernate ORM 6+ will also fetch data according to the configured laziness of the entity class, even if data is not in the cache. I believe ORM 5 just took what was there and you could end up with LazyInitializationException later. I don’t know your exact model, but I would suggest you to print the SQL queries that are executed as well to see if more queries are being executed with Hibernate ORM 6+.