I’m having an issue with streaming large files to the oracle DB using Spring Boot and Hibernate.
This issue occurs after a Spring Boot Upgrade to 3.4.1 (from 3.2.5), which includes the new minor version Hibernate 6.6.4. I’m using oracle driver ojdbc8-21.3.0.0.
In order to stream a large input file to the database I’m using BlobProxy.generateProxy(in, length) to set up the blob proxy with my input stream.
Upon transaction commit I now get the following exception:
Caused by: java.lang.ClassCastException: class org.hibernate.engine.jdbc.BlobProxy cannot be cast to class oracle.jdbc.internal.OracleBlob (org.hibernate.engine.jdbc.BlobProxy and oracle.jdbc.internal.OracleBlob are in unnamed module of loader 'app')
at oracle.jdbc.driver.OraclePreparedStatement.setBlob(OraclePreparedStatement.java:6528)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.setBlob(OraclePreparedStatementWrapper.java:153)
at com.zaxxer.hikari.pool.HikariProxyPreparedStatement.setBlob(HikariProxyPreparedStatement.java)
at org.hibernate.type.descriptor.jdbc.BlobJdbcType$4$1.doBind(BlobJdbcType.java:173)
at org.hibernate.type.descriptor.jdbc.BlobJdbcType$2$1.doBind(BlobJdbcType.java:115)
at org.hibernate.type.descriptor.jdbc.BasicBinder.bind(BasicBinder.java:61)
at org.hibernate.engine.jdbc.mutation.internal.JdbcValueBindingsImpl.lambda$beforeStatement$0(JdbcValueBindingsImpl.java:87)
at java.base/java.lang.Iterable.forEach(Iterable.java:75)
at org.hibernate.engine.jdbc.mutation.spi.BindingGroup.forEachBinding(BindingGroup.java:51)
at org.hibernate.engine.jdbc.mutation.internal.JdbcValueBindingsImpl.beforeStatement(JdbcValueBindingsImpl.java:85)
at org.hibernate.engine.jdbc.mutation.internal.AbstractMutationExecutor.performNonBatchedMutation(AbstractMutationExecutor.java:130)
at org.hibernate.engine.jdbc.mutation.internal.MutationExecutorSingleNonBatched.performNonBatchedOperations(MutationExecutorSingleNonBatched.java:55)
at org.hibernate.engine.jdbc.mutation.internal.AbstractMutationExecutor.execute(AbstractMutationExecutor.java:55)
at org.hibernate.persister.entity.mutation.InsertCoordinatorStandard.doStaticInserts(InsertCoordinatorStandard.java:194)
at org.hibernate.persister.entity.mutation.InsertCoordinatorStandard.coordinateInsert(InsertCoordinatorStandard.java:132)
at org.hibernate.persister.entity.mutation.InsertCoordinatorStandard.insert(InsertCoordinatorStandard.java:104)
at org.hibernate.action.internal.EntityInsertAction.execute(EntityInsertAction.java:110)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:644)
at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:511)
at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:414)
at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:41)
at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:127)
at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1429)
at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:491)
at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:2354)
at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:1978)
at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:439)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:169)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:267)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)
Is there an alternative ways to create a blob proxy now from an input stream? Or maybe an alternative way to stream large objects to the DB that works for oracle?
Please try to create a reproducer with our test case template and if you are able to reproduce the issue, create a bug ticket in our issue tracker and attach that reproducer.
Initially it worked fine, but the template helped me to narrow down the reproducing conditions. It seems that the exception occurs specifically when disabling the hibernate.boot.allow_jdbc_metadata_access configuration property.
I also noticed, that due to the changes in HHH-18206 Switch to JDBC LOB APIs for Oracle Dialect by loiclefevre · Pull Request #8486 · hibernate/hibernate-orm · GitHub the stream-writing of the lobs was disabled generally for Lobs in Oracle. That is, even if run with metadata_access set to true, which does not produce this exception, internally the stream is first read to a byte array, and then wrapped in an Oracle blob proxy (See org.hibernate.type.descriptor.java.BlobJavaType#getOrCreateBlob).
So, before I open a bug ticket, I’d like to discuss if this is generally discouraged now, and if there’s an alternative way of streaming data into blobs in Hibernate, which does not load everything into memory.
I find the current API (using BlobProxy.generateProxy(InputStream stream, long length)) quite misleading, because it looks like its able to handle an input stream, and not load the whole file into memory internally.
This is unfortunate. From what I can see, we would need to add a configuration option to org.hibernate.cfg.JdbcSettings to allow controlling this behavior when JDBC metadata access is disabled. Please create an improvement request for that on our issue tracker.
You have a point. I agree that the JdbcType implementations should avoid materializing the full data when the passed value is based on a stream. It seems there even was some experimentation done on that matter, but wasn’t finished. Please also create an improvement request on our issue tracker for this.
Sorry, I did not follow up on this after having some troubles creating an account for the issue tracker and eventually choosing another route (moving away from blobs in the DB entirely) due to time pressure.
So, afaik there is currently noone working on this. But I can give it another shot and finally create some tickets for it. Or if you have resources available, you’re welcome to use my test case for the ticket.
Essentially there are two issues here:
Make the blob binding configurable, so hibernate does not rely on the metadata access to be enabled
Fix the blob (un-) wrapping to not read the whole stream into memory
I don’t know enough about the internal workings to give the proper information, certainly not to the extent you did. Would be great if you could create a support ticket.
We tried to switch to byte instead of SerialBlob, but then all data in the database already saved as SerialBlob failed to load as byte. I hoped they were compatible, but apparently not.
HHH-19200 for making the lob streaming configurable
HHH-19201 for adapting the blob wrapping to not read the whole input stream into a byte array
However, I’m also far from a hibernate expert, so the descriptions in the tickets might not consider the whole picture. (E.g. I don’t quite understand the reasoning behind disabling the streaming support for oracle in the first place)
@swd_eagle
I’m not sure if this is exactly your issue then, because I would have assumed that de-serializing to a byte array should work just fine. You’ll loose the streaming behavior, but other than that I don’t see why it shouldn’t work. Maybe there’s an encoding issue with your data?
There is no update as you can see on the issues. When we find the time, we will look into the reports.
If you have pressing needs, please consider requesting a fix through Red Hat or IBM support. As you can imagine, the Hibernate team treats such requests with the highest priority, since the money for such support contracts is essentially what pays our salaries.
When paying for support is no option, you will have to wait until we find the time to look into this or pay someone else to work on this if you can’t or don’t want to contribute yourself.