Query performance regressions in Hibernate 6

Hibernate ORM
21

Hi @beikov: I just re-ran my test against the current main branch and it is still significantly slower than Hibernate 5. Did you set some properties when you ran the benchmark on your machine?

22

I just tried with these settings but there are no changes in performance for me:

integrationSettings = {
		@Setting(name = AvailableSettings.DEFAULT_BATCH_FETCH_SIZE, value = "10"),
		@Setting(name = AvailableSettings.QUERY_PLAN_CACHE_ENABLED, value = "true"),
		@Setting(name = AvailableSettings.CRITERIA_COPY_TREE, value = "false"),
}
23

You have to use the same CriteriaQuery instance to see a benefit.

24

OK thanks!

Then this optimization is not really relevant for my use-case. In my project I canā€™t re-use the same query. Queries are constructed in stateless DAOs, called from the service layer with different parameters.

I will rewrite my tests to HQL next week to see if translation of criteria queries really is the main factor for the performance regressions.

25

I just updated my PRs to use HQL instead of criteria to rule out any issues with query translation. The results are almost identical:

Hibernate 5.6:

Test-case 1: Query took 15.8S
Test-case 2: Query took 6.8S

Hibernate 6 (Main):

Test-case 1: Query took 21.6S
Test-case 2: Query took 11.9S

I would be great it if someone could profile these test-cases some more to find the real root cause of these regressions.

26

@beikov: Is there anything I can do to further help with this issue? Should I create another Jira ticket?

These regressions are currently blocking us from upgrading to Hibernate 6. I believe this is not some edge-case issue but something that potentially affects a lot of users.

27

We have identified a few unnecessary JDBC interactions that will degrade performance in cases when you are relying on 2L cache hits for entities, which I believe you are, right?
Please try the latest 6.3.0-SNAPSHOT which comes with this fix: HHH-16790 Performance issue when retrieving a large bulk of entity in bidirectionnal association by mbladel Ā· Pull Request #7009 Ā· hibernate/hibernate-orm Ā· GitHub

28

The change has no effect on the test-cases in my PR.

I will try to run our mass-indexer against the latest snapshot later today.

29

What are the maven coordinates for 6.3.0-SNAPSHOT?

I couldnā€™t find it here: https://repository.jboss.org/nexus/content/repositories/snapshots

30

This is the new repository for snapshots: https://oss.sonatype.org/content/repositories/snapshots/

1 Like
31

Thanks @mbladel!

Maven currently fails to build my project against 6.3.0-SNAPSHOT:

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.11.0:compile (default-compile) on project XY: Fatal error compiling: java.lang.NoClassDefFoundError: org/antlr/v4/runtime/ANTLRErrorListener: org.antlr.v4.runtime.ANTLRErrorListener ā†’ [Help 1]

I see that ANTLR has recently been made a dependency of the jpamodelgen module. Is there any additional Maven configuration required? Iā€™m currently simply declaring my dependency like this:

<dependency>
    <groupId>org.hibernate.orm</groupId>
    <artifactId>hibernate-jpamodelgen</artifactId>
    <version>${version.hibernate}</version>
    <scope>provided</scope>
</dependency>

Maybe there is something wrong with the generated POM for the module?

32

From what I can tell, ANTLR is defined in the pom as runtime dependency, so this should all be fine. No idea what Maven is doing here, but seems like the ANTLR JAR is not added to the classpath.

33

I managed to work around this by explicitly marking ANTLR as a compile dependency in my POM:

<dependency>
    <groupId>org.antlr</groupId>
    <artifactId>antlr4-runtime</artifactId>
    <version>${version.antlr}</version>
    <scope>compile</scope>
</dependency>

This should not be necessary though. There might be something special about my setup, but its probably worth creating a small test Maven Project with jpamodelgen enabled before releasing 6.3.0.

Iā€™ll report back when I have run our indexer.

34

After running some tests I figured out that H2 is pretty bad at handling SQL array contains checks, which is what ORM 6.2 introduced.
Using this sort of predicate is supposed to improve statement cache hit rates without significant performance penalty, but it seems for H2 this is not the case.

Disabling the array strategy for batch loading helps though. Iā€™ll discuss my findings with the team and let you know our next steps.

35

Interesting, but I donā€™t think this is the root cause either:

The performance differences are nearly exactly the same when batching is disabled. And in that case there arenā€™t any array_contains statements. I just added batching to the test, because it more closely reflects my real use case.

Maybe its better to run the tests without batching to have the simplest possible reproducer.

36

Our indexer is still 10-20% slower using 6.3.0-SNAPSHOT.

37

@beikov: I saw your performance improvements on the main branch. My original test looks much better and is on par with Hibernate 5.

Our indexer is slightly faster now, but still significantly slower than with Hibernate 5.

I adjusted my tests to more closely resemble one of our core entities and discovered another major regression:

Hibernate 5.6:

Test-case 1: Query took 12.9S
Test-case 2: Query took 18.6S

Hibernate 6 (Main):

Test-case 1: Query took 27.9S
Test-case 2: Query took 36.9S

All I did was add an eager one-to-one AuthorDetails association to the Author entity.

Could you do one more round of profiling on the updated test-cases?

38

I created [HHH-17030] - Hibernate JIRA and will look into it as soon as I find the time.

1 Like
40

For reference: I added some more benchmarks and feedback to HHH-17030.

1 Like