Hibernate Batch Fetch style recommendations

Hibernate ORM
21

Hi,

I’m a @caristu’s coworker.

I’ve extracted Openbravo Hibernate’s model and created a test case checking retained heap on SessionFactory with legacy and dynamic batch fetch styles with different (10 and 50) batch sizes.

The numbers I got are:

Style         Batch Size      Heap
------------------------------------
Legacy                10     788MB
Legacy                50    1000MB
Dynamic               10     149MB
Dynamic               50     150MB

Here you can find the repo with the test cases (fetch-style branch).

If you want to check Openbravo complete sources:

Note Openbravo generates Hibernate mapping classes from meta-model defined in DB.

I hope this helps.

22

Hi @alostale ,

Very nice thanks. It’s helping a lot.

23

Follow-ups:

Discussions on the mailing list:

24

Thanks a lot, that’s really great!

Btw, I guess here you meant Openbravo, not OpenConcerto :wink:

Thanks again!

25

I just tested the same again with 5.3.0.Final, and the difference is awesome:

Style         Batch Size      Heap
------------------------------------
Legacy                10     228MB
Legacy                50     269MB
Dynamic               10     119MB
Dynamic               50     120MB

@gsmet and the rest, thanks for your great job!

1 Like
26

Glad to see a happy user :).

Thanks for taking the time to prepare the test case, it helped a lot validating we were on the right path. And fixing that will probably help a lot of our users.

1 Like
27

For the record, there are 2 patches:

  • the first one was not risky at all so we also included it into the next 5.2: it reduces the memory used by the LoadPlan based entity loaders (you have 13 of them with a batch size of 50) by sharing what could be shared: it reduces the memory used from 1 GB to ~470 MB in your case
  • the second one was only applied to 5.3 and lazy load the per lock mode loaders (it loads eagerly the two most commonly used and loads lazily the others). This accounts for the rest of the gain but obviously if you are using some exotic lock modes, more memory will be used once they get used in your application. It should help anyway as there is very little chance that you use the 11 lock modes involved for all your entities.
29

@caristu The best way to control the amount of data you fetch is at query-time because the fetch strategy is given by the current business user requirements.

Default batch fetching strategy or mapping-time constructs like @Subselect, @BatchSize or @LazyCollection are like applying a band-aid on a broken foot. They don’t really fix the actual problem, they only bring some relief in the short-run.

If performance is important to you, you want to:

  • switch to LAZY fetching for all association types,
  • use entities only if you want to later modify them and benefit from optimistic locking or optimistic locking
  • use DTOs for read-only projections (e.g. trees, tables, reports)
  • avoid anti-patterns like Open-Session in View or enable_lazy_load_no_trans

If you do that, you will see that there is no need for stuff like @Subselect or @BatchSize, and, not only that you’ll get better memory utilization on the JVM size, but you avoid lots of processing on the DB side as well (CPU, memory, IO) as well on the networking layer.