Hi, I have a problem where my program is running out of heap.
Hibernate-core 5.4.29
Postgres 11 with postgresql-42.2.10.jar
It runs fine for quite some time, and then suddenly the heap goes on a upward trend for a while. It goes back to more-or-less flat, and then another upward trend. Repeat until I eventually get an out of memory error.
After the sudden rise in the heap, using JMX I see that it is filled with org.hibernate.hql.internal.ast.tree.ParameterNode objects. Each of these has originalText = java.lang.String = "serverIDs_XXXXX" where XXXXX is some integer.
For example, in very short order, my heap goes from ~1,000,000 ParameterNode objects to something like 13,000,000 objects.
What does the “serverIDs_XXX” mean? I suspect it must be internal to Hibernate.
I have a periodic cleanup thread. It runs a native query something like this:
DELETE FROM {table} WHERE id IN (SELECT id FROM (SELECT id, ROW_NUMBER() OVER w AS rnum FROM {table} WINDOW w AS (PARTITION BY account_id, conversation_id ORDER BY client_timestamp DESC)) as sub WHERE rnum > :maxMsgs);
Would this query actually create parameter nodes? I was under the impression that as a native query, it would be all handled inside postgres.
My guess, you use a multi-valued parameter named serverIDs somewhere and the instances you see are what Hibernate generates when expanding the parameter for each value in the final SQL.
You are right! Yes. I was searching the code for serverIDs_ with the underscore and found nothing. However, serverIDs without the underscore did show up, thanks!
I will be exploring that query now, to see if I can get it to leak.
There is no leakage here, you just seem to have a high or varying cardinalities of parameter numbers that you use which causes multiple query plan cache entries to be created. You can use parameter padding hibernate.query.in_clause_parameter_padding though to reduce the cache trashing: Hibernate ORM 5.6.1.Final User Guide
The think I don’t understand is that I have anywhere from 1 to 200 items in my list, which should, I think, have a maximum of 200 of these in the cache. I seem to recall there is a limit of 1000 items in a parameter list. So I am still trying to figure out why I am seeing 13 million of these!
If you don’t use list padding, there will be a query plan with 200 items if you provide 200 value and a query plan with 199 items if you provide 199 items etc. I don’t know why you are seeing 13 million, maybe you have different HQL representations that use this kind of list parameter?