How to make lazy loading truly lazy (V5.4)

faeze_kosari · January 13, 2021, 12:04pm

Just remove the org.hibernate.orm.tooling plugin or at least disable enableLazyInitialization.
I had the same issue and after testing a lot of things, I found this plugin was the issue .

Learner.Evermore · March 5, 2021, 6:58pm

I just noticed this after hitting (apparently) related roadblock I posted a question about here:

Here’s a copy:

I am learning how to fully control Hibernate behaviour and have hit a roadblock: if I annotate my @ManyToOne entity properties one way they aren’t lazy “enough” but if I annotate them differently then entity graphs don’t have a desired effect. Specifically:

Common to both
Hibernate 5.4 is used with bytecode enhancement. Entity class is annotated as follows:

@javax.persistence.Entity
@Table(name = "test")
@org.hibernate.annotations.DynamicInsert(false)
@org.hibernate.annotations.DynamicUpdate(false)
@org.hibernate.annotations.SelectBeforeUpdate(false)
@org.hibernate.annotations.Proxy(lazy = false)
@org.hibernate.annotations.OptimisticLocking(type = org.hibernate.annotations.OptimisticLockType.VERSION)
@org.hibernate.annotations.Polymorphism(type = org.hibernate.annotations.PolymorphismType.IMPLICIT)
@javax.persistence.Access(javax.persistence.AccessType.FIELD)
public class TestEntity {

All @ManyToOne properties look like:

    @ManyToOne // or @ManyToOne(fetch = FetchType.LAZY) - no difference
    @JoinColumn(name = "test_link_id", nullable = true)
    @OptimisticLock(excluded = false)
    @NotFound(action = NotFoundAction.EXCEPTION)
    private TestEntity testLink;

Way #1:

HAVE the following annotation:

    @LazyToOne(LazyToOneOption.NO_PROXY)

but do NOT have the following together with it (if I do it becomes “Way 2”):

    @Fetch(FetchMode.JOIN) // Other fetch modes are OK

GOOD: If I do not specify an entity graph at all Hibernate does not load ANYTHING about these ManyToOne properties (i.e. SQL query does not select them) - not even ids that are in the same table/rows as other basic properties of the entity/ies I am getting. That is an overkill. Attempt to access will cause an extra query to get the ids first, then individual queries to lazily get the rest…

BAD: If I do specify the entity graph to load these ManyToOne properties, the SQL query that Hibernate executes does get them but Hibernate reports the properties as uninitialized. On access Hibernate lazily executes a second query which just gets the ids of those ONLY (again?) … It seems to have the other properties already … so why loading the ids again?

Way #2:

Either do NOT have the following annotation:

    @LazyToOne(LazyToOneOption.NO_PROXY)

OR DO HAVE the following together with it (otherwise it becomes “Way 1”):

    @Fetch(FetchMode.JOIN) // other fetch modes are NOT OK

BAD: If I don’t specify an entity graph at all Hibernate issues TWO queries before I get the chance to do anything. The first query just gets the ids. Then the second query then gets the properties of the ManyToOne linked entities even though I didn’t ask/want that.

GOOD: If I do specify the entity graph it behaves as expected - the single executed SQL query joins in all the data needed and no subsequent queries are issued on access.

Test code

    final CriteriaBuilder = session.getCriteriaBuilder()
    final CriteriaQuery<TestEntity> query = builder().createQuery(TestEntity.class);
    final Root<TestEntity> root = query.from(TestEntity.class);
    final Path<Long> idPath = root.get("id");
    query.where(builder.equal(idPath, someChosenId));
    final TypedQuery<TestEntity> tq = session().createQuery(query);
    tq.setHint("javax.persistence.fetchgraph", entityGraph); // as/only when wanted
    final TestEntity testEntity = tq.getSingleResult();
    System.out.println(Hibernate.isPropertyInitialized(testEntity, "testLink"));
    // access testEntity's testLink property, watch Hibernate-generated/executed SQL

Question

How can I get the “good” both without and with entity graphs specified? I.e. no eager/extra loading without the graph (not even the more complex query) and still have the ability to say that I want more/eagerly using the graph without IT then causing additional queries for id? Note: I tried using .fetch(…) methods instead of graphs with exactly the same effect.

(Almost) the entire test code can be found at pastebin.com/1iFiPTL4 - start it with testLoading() method. You do have to supply your own Hibernate Session.

beikov · March 8, 2021, 1:46pm

As far as I understand, what you want is the following:

@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "test_link_id", nullable = true)
@OptimisticLock(excluded = false)
@NotFound(action = NotFoundAction.EXCEPTION)
private TestEntity testLink;

This will load the FK (which seems ok for you) and cause a bytecode enhanced proxy to be used for the association. On access of the first non-id property, it will lazy load the association.

If that doesn’t work, please let me know how you configured the enhancement. Maybe you need to enable hibernate.enhancer.enableLazyInitialization. Also see Hibernate ORM 5.4.29.Final User Guide

Learner.Evermore · March 8, 2021, 6:33pm

Thanks for responding!

Yes, loading the FK is fine (inexpensive, same row). Also yes/good to issue lazy load when association is accessed (at that point its id is already known, though). enableLazyInitialization is, in fact, set to true for me, with Hibernate 5.4.14 - perhaps a version update would help.

EDIT: No, it would not. Just tried with 5.4.29.Final. No difference.

If you have a look at the test code I referenced via pastebin (is there a better way?) you can see that it includes that case with a different name:

@ManyToOne(fetch = FetchType.LAZY)
@JoinColumn(name = "linkLazy_id", nullable = true)
@OptimisticLock(excluded = false)
@NotFound(action = NotFoundAction.EXCEPTION)
public TestEntity linkLazy;

Merging my two posts into one to work around the three posts limit:

I have a log of the execution of that test. You can see it here:

If you care about the DB setup I can share that too (didn’t end up needing all as subselects aren’t meant to be for ToOnes)

create table public.loading_test (
    id bigint NOT NULL,
    version integer NOT NULL,
    name text,
	link_id bigint,
	linkSelect_id bigint,
	linkJoin_id bigint,
	linkSubselect_id bigint,
	linkNoProxy_id bigint,
	linkNoProxySelect_id bigint,
	linkNoProxyJoin_id bigint,
	linkNoProxySubselect_id bigint,
	linkLazy_id bigint,
	linkLazySelect_id bigint,
	linkLazyJoin_id bigint,
	linkLazySubselect_id bigint,
	linkLazyNoProxy_id bigint,
	linkLazyNoProxySelect_id bigint,
	linkLazyNoProxyJoin_id bigint,
	linkLazyNoProxySubselect_id bigint
);

insert into public.loading_test values(1, 1, 'link', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(2, 1, 'linkSelect', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(3, 1, 'linkJoin', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(4, 1, 'linkSubselect', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(5, 1, 'linkNoProxy', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(6, 1, 'linkNoProxySelect', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(7, 1, 'linkNoProxyJoin', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(8, 1, 'linkNoProxySubselect', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(9, 1, 'linkLazy', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(10, 1, 'linkLazySelect', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(11, 1, 'linkLazyJoin', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(12, 1, 'linkLazySubselect', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(13, 1, 'linkLazyNoProxy', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(14, 1, 'linkLazyNoProxySelect', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(15, 1, 'linkLazyNoProxyJoin', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(16, 1, 'linkLazyNoProxySubselect', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);

insert into public.loading_test values(100, 1, 'no-graph', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(101, 1, 'shallow-graph', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(102, 1, 'id-graph', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(103, 1, 'id-name-graph', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(104, 1, 'all-graph', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);

insert into public.loading_test values(200, 1, 'no-fetch', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(201, 1, 'shallow-fetch', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(202, 1, 'id-fetch', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(203, 1, 'id-name-fetch', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);
insert into public.loading_test values(204, 1, 'all-fetch', 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16);

Also one more thing - I manually put up this table showing different outcomes:

UPDATE: I cannot add more posts so answering here… After @beikov noted what I originally missed:

Could you try setting the property hibernate.bytecode.allow_enhancement_as_proxy as suggested in the answer I posted?

… I applied that setting and reran the tests. The results are different. Now nothing is lazy, everything loads all the time, the only question whether Hibernate splits the loading into two queries or joins everything in one - same for all properties.

New log here:

beikov · March 9, 2021, 8:27am

I think this is similar to this question: ManyToOne Field Loads Eagerly When Lazy Initialization Enabled - #4 by sebersole

Could you try setting the property hibernate.bytecode.allow_enhancement_as_proxy as suggested in the answer I posted?

Learner.Evermore · March 10, 2021, 4:16pm

OK… now I can add a message… after I deleted my previous one to “make room” (I have a limit of three messages per thread, total). I did respond yesterday already, though, by editing the message above. I initially missed the hibernate.bytecode.allow_enhancement_as_proxy setting but, after trying it out, I noticed that everything became almost equal and essentially eager, just two different kinds of eager. Details in my previous post, including new logs.

Please let me know how to respond next time as this limit of three won’t really let me add any more messages…

Our (Gradle) bytecode enhancement settings:

hibernate {
    sourceSets = [project.sourceSets.main]
    enhance {
        enableLazyInitialization = true;
        enableDirtyTracking = true;
        enableAssociationManagement = false;
        enableExtendedEnhancement = false;
    }
}

Our hibernate.properties:

hibernate.bytecode.use_reflection_optimizer=true
hibernate.order_inserts=true
hibernate.order_updates=true
hibernate.enhancer.enableLazyInitialization=true
hibernate.bytecode.allow_enhancement_as_proxy=true
hibernate.enhancer.enableDirtyTracking=true

I also noticed that we have @org.hibernate.annotations.Proxy(lazy = false):

@javax.persistence.Entity
@Table(name = "loading_test")
@org.hibernate.annotations.BatchSize(size = 100)
@org.hibernate.annotations.DynamicInsert(false)
@org.hibernate.annotations.DynamicUpdate(false)
@org.hibernate.annotations.SelectBeforeUpdate(false)
@org.hibernate.annotations.Proxy(lazy = false)
@org.hibernate.annotations.OptimisticLocking(type = org.hibernate.annotations.OptimisticLockType.VERSION)
@org.hibernate.annotations.Polymorphism(type = org.hibernate.annotations.PolymorphismType.IMPLICIT)
@javax.persistence.Access(javax.persistence.AccessType.FIELD)
public class TestEntity

I tried having it set to true and/or removing it, in combination with other suggested property options, but it does not seem to have any effect.

beikov · March 11, 2021, 10:10am

Discourse limits non-admin users to 3 replies in topics they don’t own. You should have created a new topic as the creator of a topic can comment as often as he likes. Anyway, could you please create an issue in the issue tracker(https://hibernate.atlassian.net) with a test case(hibernate-test-case-templates/JPAUnitTestCase.java at master · hibernate/hibernate-test-case-templates · GitHub) that reproduces the issue? This is a very complicated topic involving lots of different features, so we need to be able to see this failing in a reproducible way to give you further guidance.

Learner.Evermore · March 11, 2021, 3:25pm

Will have a look. Not sure if I am able to do this right. Are there any similar tests you can point me to that I can start with, that use bytecode enhancement with field access? My main challenges:

Ensuring test environment / setup (bytecode enhancement and its settings)
Capturing SQL issues by Hibernate - the thing “works” but not efficiently and not lazily/eagerly as it should. Not sure how to check/assert these things. I looked at logs… I could try to wrap a JDBC Connection to intercept calls and test for that but that would be both finicky and very much “involved”.

(Otherwise I am perfectly willing and ready to do this)

beikov · March 11, 2021, 3:40pm

It would be best if you create a test in the ORM repository if you are willing to do the work. See org.hibernate.test.bytecode.enhancement.lazy.proxy.LazyToOnesProxyWithSubclassesTest for an example.

Learner.Evermore · March 11, 2021, 4:10pm

Thanks! Will do. Not today but possibly as early as tomorrow or early next week.

Learner.Evermore · March 12, 2021, 2:51pm

See [HHH-14500] Eager vs lazy loading control of link properties is broken in some cases - Hibernate JIRA

Learner.Evermore · March 12, 2021, 7:03pm

Test is now attached to HHH-14500 - LazyToOnesControlTest.java.

Topic		Replies	Views
ManyToOne - always EAGER loading Hibernate ORM	9	9742	April 7, 2022
Hibernate 5.2.18 cannot get a lazy OneToOne anymore Hibernate ORM	7	938	December 30, 2020
@ManyToOne Lazy fetch works on select, but is treated as Eager for update Hibernate ORM	1	1497	May 31, 2021
Lazy loading of ManyToOne does not work when hibernate.jpa.compliance.proxy is set and L2 cache is enabled Hibernate ORM	0	1871	October 16, 2019
Bytecode enhancement Hibernate ORM	2	1596	July 13, 2020

How to make lazy loading truly lazy (V5.4)

Related topics