Hi, since we process many messages we decided to create some custom queries to perform some complex operation once as batch.
After struggling very much to create an array of objects from our type have decided to pass many arrays to the query and using unnest in the query.
What we’ve saw is that after a while our java process memory is raising and causing our service to crash.
After investigating we’ve saw that the leak came from hibernate - In the class JdbcTypeRegistry.java there a map (called typeConstructorDescriptorMap) for caching the typeDescriptors. I’ve saw that because the elementType passed to this function (resolveTypeConstructorDescriptor) is all the time new instance the cache key is never the same. what casues this map to fill with the same types over and over sinch the hash code is different of the element type.
Simply in the some repository adding a custom query as followed:
@Modifying
@Query(value =
" INSERT INTO some_table (id, name)" +
" SELECT id, name" +
" FROM UNNEST(:ids, :names) as data(id,name)", nativeQuery = true)
void memoryLeakFunction(UUID[] ids, String[] names);
Ideal would be calling this method like this
@Modifying
@Query(value =
" INSERT INTO some_table (id, name)" +
" SELECT data.id, data.name" +
" FROM UNNEST(:data) as data(id ,name)", nativeQuery = true)
void withUserType(MyType[] data);
Would love to get some help here regarding this issue? maybe workaround ?
or an example of a UserType for an object - and array of this object…
And would it help if i would create a custom UserType ? or perhaps a JdbcType ?
There are still instances that will end up creating too many entries in the caches. Would be good to track them down.
Hibernate uses the BoundedConcurrentHashMap implementation that was copied from Infinispan at some point for creating bounded caches. I am preparing a PR that would limit the number of entries to 1000. Will create a ticket for it and see if I can get it merged.
For a quick solution, I implemented a very ugly way of replacing the instances of the caches in JdbcTypeRegistry with that same implementation through reflection (the solution is extra-hacky as I had to create a subclass of ConcurrentHashMap - the fields reference the implementation directly) and have seen a completely stable memory profile after that. I think it makes sense to have a limit to the size of the caches even if they never get hit.
Thanks for trying to fix this. The solution you’re proposing is just a remedy, not a definitive solution unfortunately.
You said you were able to reproduce this problem, so can you please try to create a reproducer with our test case template and if you are able to reproduce the issue, attach it to the Jira case?
I completely agree that the core issue should also be resolved, will track it down and create a reproducer for it.
I still believe that including the changes I proposed is cautious long term and does not have any downsides from my perspective, but I would love to also hear your concerns if you have any.
@beikov what do you think about introducing Bounded implementations of the ConcurrentMap in the class anyway? It is a good practice that would prevent any further unintended consequences.
Did you have specific concerns about that change that you could explain? I would love to understand the concerns if possible.
The fix was released in ORM 6.6.2 last week, so you can all update now.
Since we’re talking about a type registry and not about a cache, there is no need for a bounded map implementation. The amount of types should be bounded by design, unless of course there is a bug like this one. Evicting entries from that registry is just wrong.
)…
