Batch Fetch Style Recommendations


#21

Hi,

I’m a @caristu’s coworker.

I’ve extracted Openbravo Hibernate’s model and created a test case checking retained heap on SessionFactory with legacy and dynamic batch fetch styles with different (10 and 50) batch sizes.

The numbers I got are:

Style         Batch Size      Heap
------------------------------------
Legacy                10     788MB
Legacy                50    1000MB
Dynamic               10     149MB
Dynamic               50     150MB

Here you can find the repo with the test cases (fetch-style branch).

If you want to check Openbravo complete sources:

Note Openbravo generates Hibernate mapping classes from meta-model defined in DB.

I hope this helps.


#22

Hi @alostale ,

Very nice thanks. It’s helping a lot.


#23

Follow-ups:

Discussions on the mailing list:


#24

Thanks a lot, that’s really great!

Btw, I guess here you meant Openbravo, not OpenConcerto :wink:

Thanks again!


#25

I just tested the same again with 5.3.0.Final, and the difference is awesome:

Style         Batch Size      Heap
------------------------------------
Legacy                10     228MB
Legacy                50     269MB
Dynamic               10     119MB
Dynamic               50     120MB

@gsmet and the rest, thanks for your great job!


#26

Glad to see a happy user :).

Thanks for taking the time to prepare the test case, it helped a lot validating we were on the right path. And fixing that will probably help a lot of our users.


#27

For the record, there are 2 patches:

  • the first one was not risky at all so we also included it into the next 5.2: it reduces the memory used by the LoadPlan based entity loaders (you have 13 of them with a batch size of 50) by sharing what could be shared: it reduces the memory used from 1 GB to ~470 MB in your case
  • the second one was only applied to 5.3 and lazy load the per lock mode loaders (it loads eagerly the two most commonly used and loads lazily the others). This accounts for the rest of the gain but obviously if you are using some exotic lock modes, more memory will be used once they get used in your application. It should help anyway as there is very little chance that you use the 11 lock modes involved for all your entities.

Assertion error in org.hibernate.engine.internal.BatchFetchQueueHelper