Wildfly crashing with hibernate-core-6.5.2.Final

Hibernate ORM
1

Hi team,
We have seen connection leaks happening after we upgraded from Hibernate3 to hibernate-core-6.5.2.Final. This crashes the wildfly (version 31) after some time and only a restart of the server helps. we are using jdk 17.
The queries which used to work earlier have stopped working after the upgrade. They just hang and dont return results for a long time. This is impacting our production application. Any tips or guidance will be greatly appreciated.
FYI - we are using same hibernate.cfg.xml from hibernate3. Do we need to change anything to make it compatible with hibernate6.5?

Thanks
CM

hibernate-cfg
hibernate-cfg1709×761 45.4 KB

2

Hello @cmalayil, Hibernate 3 was released in 2005. A lot has changed in the past 19 years, and with this little information is going to be pretty much impossible to help you in any significant way.

You can try isolating the problem around these “long running” queries, extracting the entity mappings that are involved and looking at the generated SQL and the database logs to try understanding what’s going on.

If you can provide a reproducer test case that shows hanging queries we’ll be more than happy to help you. Otherwise, your only option is looking into commercial support.

Your configuration looks pretty simple, and I believe the properties you are using didn’t chance.

3

Thanks for the reply mbladel!
looks like widlfly is crashing due to connection leaks.
We were trying to understand the code using jprofiler to know where exactly connections leaks are happening and noticed that this happens only when an exception is thrown in below code which results in somehow connection not getting closed. We are not able to understand what is wrong is this. I have also attached HibernateUtil.java class for reference. Kindly help. Thanks.

HibernateUtil1
HibernateUtil1690×912 152 KB

HibernateUtil2
HibernateUtil2880×905 126 KB

HibernateUtil3
HibernateUtil3751×901 123 KB

HibernateUtil4
HibernateUtil4896×888 116 KB

********** Main method which is causing connection leak when exception is thrown in this **********

@Override
	@WebMethod(exclude = true)
	public RequestGenerationResponseDto createOcRequestWithoutDataWithRecptContext(NoticeRequest noticeRequest, User user) throws Exception
	{
        if (_logger.isDebugEnabled())
        {
            _logger.debug("createOcRequestWithoutDataWithRecptContext(NoticeRequest noticeRequest=" + noticeRequest + ", " +
            								"User user) - entered");
        }
 
        RequestGenerationResponseDto rgr = ResponseUtil.createRequestGenerationResponse();                        
 
        DALAccessor accessor = null;
        Session sess = null;
        try (OCTempFiles tempFolder = OCTempFiles.openNewTemporaryFolder())
        {
            accessor = HibernateUtil.getDALAccessor(user);                        
            sess = HibernateUtil.getSession(user);
            HibernateUtil.beginTransaction();
 
            OcGenerationDelegate bd = new OcGenerationDelegate(accessor);
 
            //set the updatedBy to the requestor
            HibernateUtil.setSessionUserId(noticeRequest.getRequestor());
 
            RequestGenerationResponseDto response = bd.createOcRequestWithoutData(noticeRequest.getCorrespondenceName(),
            								-1, 
            								Long.parseLong(noticeRequest.getCaseID()), 
            								Long.parseLong(noticeRequest.getContextKey()), 
            								noticeRequest.getContextType(), 
            								noticeRequest.getRequestor(), 
            								noticeRequest.getRequestSource(), 
            								null, 
            								noticeRequest.getRecipientContextType(), 
            								noticeRequest.getRecipientContextKey());
 
            if(!StringUtil.isNullOrEmpty(response.getFailureReason()))
    		{
            	rgr.setFailureReason(response.getFailureReason());
            	rgr.setResolutionCaseId(response.getResolutionCaseId());
                HibernateUtil.rollbackTransaction();            
    		}
            else
            {
            	HibernateUtil.commitTransaction();
            }
        }
        catch(Exception e)
        {
            _logger.error("createOcRequestWithoutDataWithRecptContext(NoticeRequest noticeRequest=" + noticeRequest + ", " +
            								"User user)", e);
 
            HibernateUtil.rollbackTransaction();            
            rgr.addMessage(ExceptionUtil.getMessage(e), OcConstants.OC_REQUEST_ERROR, null, null);
        }
        finally
        {
        	HibernateUtil.cleanUp(accessor);
            HibernateUtil.cleanUp();            
        }
 
 
        if (_logger.isDebugEnabled())
        {
            _logger.debug("createOcRequestWithoutDataWithRecptContext(NoticeRequest, User) - end");
        }
 
        return rgr;
	}

***** DalEntryPointManager ********

public DALAccessor getAccessor(User dtUser) throws DALException
	{
        GenericDALAccessor dalac = null;
        DALConnection con = null;
 
    	try 
    	{
    		DataSource ds = null;
    		_logger.error("In DALEPM:" + dtUser.getUsername() + dtUser.getDecryptedPassword());
	  		if (dsPool.containsKey(dtUser.getUsername() + dtUser.getDecryptedPassword()) == false)
	        {	  
	  			_logger.error("In GetAccessor false");
	  			ds = this.setupDataSource(dtUser);
	  			con = new DALConnection(ds.getConnection(), dtUser);
	  			dsPool.put(dtUser.getUsername()+dtUser.getDecryptedPassword(), ds);
	        }
	  		else
	  		{	
	  			_logger.error("In GetAccessor true");
	  			ds = dsPool.get(dtUser.getUsername() + dtUser.getDecryptedPassword());
	  			con = new DALConnection(ds.getConnection(), dtUser);
	  		}
    		//con = new DALConnection(setupJNDIDataSource().getConnection(), dtUser);
	        dalac = new GenericDALAccessor(con, this, dtUser);
            synchronized(_allDALAccessors)
            {
                this._allDALAccessors.add(dalac);
            }
    	}
    	catch (Exception a)
    	{
			_logger.error(a.getMessage(), a);
    		_dalExcept = new DALException(a.getMessage(), a);
    		throw _dalExcept;
    	}    	
    	catch (Error er)
    	{
    		_logger.error(er.toString());
    	}
	    return dalac;
	}

**** Connection Leak Error in Logs *****
03:00:58,769 INFO [org.jboss.jca.core.api.connectionmanager.ccm.CachedConnectionManager] (default task-6) IJ000100: Closing a connection for you. Please close them yourself: org.jboss.jca.adapters.jdbc.jdk8.WrappedConnectionJDK8@8d16ef7: java.lang.Throwable: STACKTRACE
at org.jboss.ironjacamar.impl@3.0.8.Final//org.jboss.jca.core.connectionmanager.ccm.CachedConnectionManagerImpl.registerConnection(CachedConnectionManagerImpl.java:308)
at org.jboss.ironjacamar.impl@3.0.8.Final//org.jboss.jca.core.connectionmanager.AbstractConnectionManager.allocateConnection(AbstractConnectionManager.java:819)
at org.jboss.ironjacamar.jdbcadapters@3.0.8.Final//org.jboss.jca.adapters.jdbc.WrapperDataSource.getConnection(WrapperDataSource.java:152)
at org.jboss.as.connector@31.0.1.Final//org.jboss.as.connector.subsystems.datasources.WildFlyDataSource.getConnection(WildFlyDataSource.java:47)
at deployment.dtaxws.war//com.revenuesolutionsinc.dali.entry.DALEntryPointManager.getAccessor(DALEntryPointManager.java:317)
at deployment.dtaxws.war//com.rsi.oc.util.HibernateUtil.getConnection(HibernateUtil.java:448)
at deployment.dtaxws.war//com.rsi.oc.util.HibernateUtil.getManualSession(HibernateUtil.java:361)
at deployment.dtaxws.war//com.rsi.oc.util.HibernateUtil.getSession(HibernateUtil.java:259)
at deployment.dtaxws.war//com.rsi.oc.ws.operation.OCOperationWebServices.createOcReqAndDoc(OCOperationWebServices.java:1170)

4

I would strongly advise you to use container managed transactions and entity managers. Implementing this stuff by yourself is asking for trouble as you see for yourself.
You probably forget to close a session or commit/rollback a transaction somewhere.

5

Hi @beikov ,

thanks for your reply. we are actually closing connection and session properly (pasted code in previous thread). Also wanted to mention that this code worked perfectly fine before the upgrade hence we are confused on why its not working now. Any thoughts?

Thanks
cmalayil

6

Your transaction handling looks a bit fishy to me. If you don’t want to use declarative transaction management, at least use a dedicated method taking a lambda to abstract this properly e.g.

public class HibernateUtil {

    ...

	public static void inTransaction(User user, Consumer<Session> function) {
		try ( Session session = getSession( user ) ) {
			final Transaction txn = session.getTransaction();
			txn.begin();
			try {
				function.accept( session );
			}
			catch (Throwable e) {
				try {
					txn.rollback();
				}
				finally {
					throw e;
				}
			}
			if ( !txn.getRollbackOnly() ) {
				txn.commit();
			}
			else {
				txn.rollback();
			}
		}
	}
7

Hi @beikov , can you please help me understand which part of the code is fishy so that i can pay close attention to it?
We tried doing the logic which you suggested but still gettign same problem. Thanks for your help!

8

Hibernate ORM will not close the connection that you provide. You will have to close it

public class HibernateUtil {

    ...

	public static void inTransaction(User user, Consumer<Session> function) {
		Connection c = null;
		try ( Session session = getSession( user ) ) {
			c = ( (SessionImplementor) session ).getJdbcCoordinator().getLogicalConnection().getPhysicalConnection();
			final Transaction txn = session.getTransaction();
			txn.begin();
			try {
				function.accept( session );
			}
			catch (Throwable e) {
				try {
					txn.rollback();
				}
				finally {
					throw e;
				}
			}
			if ( !txn.getRollbackOnly() ) {
				txn.commit();
			}
			else {
				txn.rollback();
			}
		}
		finally {
			if ( c != null ) {
				try {
					c.close();
				}
				catch (SQLException e) { }
			}
		}
	}