# Possible memory leak in Envers due to Narayana Transaction Reaper

**URL:** <https://discourse.hibernate.org/t/possible-memory-leak-in-envers-due-to-narayana-transaction-reaper/463>\
**Category:** Hibernate ORM\
**Created:** [March 30, 2018, 7:51am UTC](https://discourse.hibernate.org/t/possible-memory-leak-in-envers-due-to-narayana-transaction-reaper/463 "2018-03-30T07:51:41Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![kimdlanor](https://avatars.discourse-cdn.com/v4/letter/k/3bc359/32.png) [@kimdlanor](https://discourse.hibernate.org/u/kimdlanor)\
**Post date:** [March 30, 2018, 7:51am UTC](https://discourse.hibernate.org/t/possible-memory-leak-in-envers-due-to-narayana-transaction-reaper/463/1 "2018-03-30T07:51:41Z")

</div>

Hi,

In a JTA multi-threaded environment we are working on a project where we are processing large amounts of data. But we ran into a problem where it seems that our software is leaking memory. We analyzed the heapdumps and determined that it was Envers’s AuditProcessManager that was keeping data from being garbage collected.

Perhaps this issue should be posted in the Jboss/Narayana forums. I have been debugging and testing and determined that the cause lies in the fact that the transaction times out and the Transaction Reaper aborts the transaction at a point where Hibernate is not getting a chance to cleanup.

The issue seems to be related and/or originated in the following issue/fix: [Transaction timeout can cause non-threadsafe session access by reaper thread](https://hibernate.atlassian.net/browse/HHH-7910)

I have tried to write it down as simple as possible. The following is happening (Note that the lock I am talking about is the \_syncLock in TwoPhaseCoordinator):  
Thread: work  
Thread: flush. Envers registers beforeCompletion and afterCompletion on Session’s actionQueue (afterCompletion will do the cleanup)  
Thread: doCommit  
Thread: beforeCompletion -\> lock -\> Envers does its work  
Reaper: abort transaction  
Reaper: afterCompletion -\> wait for lock  
Thread: finds out transaction is rolled back and runs delayedAfterCompletion  
Thread: ends beforeCompletion -\> unlock  
Reaper: gets lock, calls afterCompletion and finds out it is not the original Thread, thus sets delayedAfterCompletion.  
Thread: afterCompletion -\> wait for lock  
Reaper: finishes afterCompletion -\> unlock  
Thread: gets lock and finds out afterCompletion has already been called (\_afterCalled in TwoPhaseCoordinator) -\> nothing to do and unlock

Now Envers still has a reference to the audit-work in the Map in AuditProcessManager it is supposed to cleanup during afterCompletion, but it will never be called.

This issues has been found using an older version of Hibernate and Narayana, but I have tested this in a small example application with the latest version of Hibernate (5.2.15.Final) and the latest version of Narayana (5.8.0.Final) and I can reproduce the issue.

I have created a github-repository where I have pushed my test application: [https://github.com/ronaldmik/memoryleak](https://github.com/ronaldmik/memoryleak)

Any help would be appreciated. If this should (also) be posted in the Jboss/Narayana forums, let me know.

Thanks in advance!

edit: added link to github and fixed link to issue HHH-7910

---

<div class="post-metadata">

**Author:** ![vlad](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/vlad/32/658_2.png) [@vlad](https://discourse.hibernate.org/u/vlad)\
**Post date:** [March 30, 2018, 8:25am UTC](https://discourse.hibernate.org/t/possible-memory-leak-in-envers-due-to-narayana-transaction-reaper/463/2 "2018-03-30T08:25:34Z")

</div>

Thanks for providing a replicating example. It sounds like a bug.

Since [HHH-7910](https://hibernate.atlassian.net/browse/HHH-7910) is marked as Fixed, we need a new issue. You should create a new issue and add a link to the replicating test case on GitHub.

@Naros you might be interested in this issue too.

---

<div class="post-metadata">

**Author:** ![kimdlanor](https://avatars.discourse-cdn.com/v4/letter/k/3bc359/32.png) [@kimdlanor](https://discourse.hibernate.org/u/kimdlanor)\
**Post date:** [March 30, 2018, 8:32am UTC](https://discourse.hibernate.org/t/possible-memory-leak-in-envers-due-to-narayana-transaction-reaper/463/3 "2018-03-30T08:32:17Z")

</div>

Wow, that was a quick response!

I have copied the text and created the following issue: [https://hibernate.atlassian.net/browse/HHH-12448](https://hibernate.atlassian.net/browse/HHH-12448)

---

<div class="post-metadata">

**Author:** ![vlad](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/vlad/32/658_2.png) [@vlad](https://discourse.hibernate.org/u/vlad)\
**Post date:** [March 30, 2018, 8:43am UTC](https://discourse.hibernate.org/t/possible-memory-leak-in-envers-due-to-narayana-transaction-reaper/463/4 "2018-03-30T08:43:13Z")

</div>

Thank you. We’ll have to investigate it.

---

<div class="post-metadata">

**Author:** ![Naros](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/naros/32/19_2.png) [@Naros](https://discourse.hibernate.org/u/Naros)\
**Post date:** [April 2, 2018, 1:34pm UTC](https://discourse.hibernate.org/t/possible-memory-leak-in-envers-due-to-narayana-transaction-reaper/463/5 "2018-04-02T13:34:26Z")

</div>

Thank you for the test case & issue. That will definitely help in the investigation.

---

<div class="post-metadata">

**Author:** ![Naros](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/naros/32/19_2.png) [@Naros](https://discourse.hibernate.org/u/Naros)\
**Post date:** [April 17, 2018, 6:14pm UTC](https://discourse.hibernate.org/t/possible-memory-leak-in-envers-due-to-narayana-transaction-reaper/463/6 "2018-04-17T18:14:13Z")

</div>

Just following up here, the issue [HHH-12448](https://hibernate.atlassian.net/browse/HHH-12448) has been resolved.

The solution was to make sure that we attempt to fire the `AfterTransactionCompletion` callbacks at the time of close to ensure that if an applications depends on that callback to release resources that those callbacks happen to avoid any potential resource/memory leak as in the case of Envers and AuditProcessManager.