Possible memory leak in Envers due to Narayana Transaction Reaper


In a JTA multi-threaded environment we are working on a project where we are processing large amounts of data. But we ran into a problem where it seems that our software is leaking memory. We analyzed the heapdumps and determined that it was Envers’s AuditProcessManager that was keeping data from being garbage collected.

Perhaps this issue should be posted in the Jboss/Narayana forums. I have been debugging and testing and determined that the cause lies in the fact that the transaction times out and the Transaction Reaper aborts the transaction at a point where Hibernate is not getting a chance to cleanup.

The issue seems to be related and/or originated in the following issue/fix: Transaction timeout can cause non-threadsafe session access by reaper thread

I have tried to write it down as simple as possible. The following is happening (Note that the lock I am talking about is the _syncLock in TwoPhaseCoordinator):
Thread: work
Thread: flush. Envers registers beforeCompletion and afterCompletion on Session’s actionQueue (afterCompletion will do the cleanup)
Thread: doCommit
Thread: beforeCompletion -> lock -> Envers does its work
Reaper: abort transaction
Reaper: afterCompletion -> wait for lock
Thread: finds out transaction is rolled back and runs delayedAfterCompletion
Thread: ends beforeCompletion -> unlock
Reaper: gets lock, calls afterCompletion and finds out it is not the original Thread, thus sets delayedAfterCompletion.
Thread: afterCompletion -> wait for lock
Reaper: finishes afterCompletion -> unlock
Thread: gets lock and finds out afterCompletion has already been called (_afterCalled in TwoPhaseCoordinator) -> nothing to do and unlock

Now Envers still has a reference to the audit-work in the Map in AuditProcessManager it is supposed to cleanup during afterCompletion, but it will never be called.

This issues has been found using an older version of Hibernate and Narayana, but I have tested this in a small example application with the latest version of Hibernate (5.2.15.Final) and the latest version of Narayana (5.8.0.Final) and I can reproduce the issue.

I have created a github-repository where I have pushed my test application: https://github.com/ronaldmik/memoryleak

Any help would be appreciated. If this should (also) be posted in the Jboss/Narayana forums, let me know.

Thanks in advance!

edit: added link to github and fixed link to issue HHH-7910

Thanks for providing a replicating example. It sounds like a bug.

Since HHH-7910 is marked as Fixed, we need a new issue. You should create a new issue and add a link to the replicating test case on GitHub.

@Naros you might be interested in this issue too.

Wow, that was a quick response!

I have copied the text and created the following issue: https://hibernate.atlassian.net/browse/HHH-12448

Thank you. We’ll have to investigate it.

Thank you for the test case & issue. That will definitely help in the investigation.

Just following up here, the issue HHH-12448 has been resolved.

The solution was to make sure that we attempt to fire the AfterTransactionCompletion callbacks at the time of close to ensure that if an applications depends on that callback to release resources that those callbacks happen to avoid any potential resource/memory leak as in the case of Envers and AuditProcessManager.