Can't find a codec for class org.hibernate.ogm.datastore.mongodb.type.GridFS

Hibernate OGM
1

I need to store big (250k+ pojo objects) lists inside a single mongodb document. The 16MB limit in this case is a huge problem: you will reach the limit with just 30k-40k elements.

So I’m trying to experiment with GridFS following the logic here

Here the code

MyEntity event= new MyEntity();
event.setName("name");
ArrayList<RowElement> elements = new ArrayList<RowElement>();
elements.add(new RowElement("name1",1L,"itemCode","itemDescription"));
elements.add(new RowElement("name2",1L,"itemCode","itemDescription"));
event.setList(new GridFS(SerializationUtils.serialize(elements)));

EntityManager e = Persistence.createEntityManagerFactory("hibernatedebug").createEntityManager();
EntityTransaction tx = e.getTransaction();
tx.begin();
e.persist(event);
tx.commit();

Note SerializationUtils comes from Apache commons.

When I try to commit the transaction I get the error Caused by: org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class org.hibernate.ogm.datastore.mongodb.type.GridFS.

So GridFS is not recognized as valid type…how can I fix?

Here the stacktrace

javax.persistence.RollbackException: Error while committing the transaction
	at org.hibernate.internal.ExceptionConverterImpl.convertCommitException(ExceptionConverterImpl.java:81)
	at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:101)
	at it.anto.hibernate.Test.main(Test.java:45)
Caused by: org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for class org.hibernate.ogm.datastore.mongodb.type.GridFS.
	at org.bson.codecs.configuration.CodecCache.getOrThrow(CodecCache.java:46)
	at org.bson.codecs.configuration.ProvidersCodecRegistry.get(ProvidersCodecRegistry.java:63)
	at org.bson.codecs.configuration.ChildCodecRegistry.get(ChildCodecRegistry.java:51)
	at org.bson.codecs.DocumentCodec.writeValue(DocumentCodec.java:184)
	at org.bson.codecs.DocumentCodec.writeMap(DocumentCodec.java:199)
	at org.bson.codecs.DocumentCodec.writeValue(DocumentCodec.java:182)
	at org.bson.codecs.DocumentCodec.writeMap(DocumentCodec.java:199)
	at org.bson.codecs.DocumentCodec.encode(DocumentCodec.java:141)
	at org.bson.codecs.DocumentCodec.encode(DocumentCodec.java:45)
	at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:63)
	at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:29)
	at com.mongodb.operation.BulkWriteBatch$WriteRequestEncoder.encode(BulkWriteBatch.java:387)
	at com.mongodb.operation.BulkWriteBatch$WriteRequestEncoder.encode(BulkWriteBatch.java:377)
	at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:63)
	at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:29)
	at com.mongodb.internal.connection.BsonWriterHelper.writeDocument(BsonWriterHelper.java:75)
	at com.mongodb.internal.connection.BsonWriterHelper.writePayload(BsonWriterHelper.java:59)
	at com.mongodb.internal.connection.CommandMessage.encodeMessageBodyWithMetadata(CommandMessage.java:143)
	at com.mongodb.internal.connection.RequestMessage.encode(RequestMessage.java:138)
	at com.mongodb.internal.connection.CommandMessage.encode(CommandMessage.java:57)
	at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:244)
	at com.mongodb.internal.connection.UsageTrackingInternalConnection.sendAndReceive(UsageTrackingInternalConnection.java:99)
	at com.mongodb.internal.connection.DefaultConnectionPool$PooledConnection.sendAndReceive(DefaultConnectionPool.java:444)
	at com.mongodb.internal.connection.CommandProtocolImpl.execute(CommandProtocolImpl.java:72)
	at com.mongodb.internal.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:200)
	at com.mongodb.internal.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:269)
	at com.mongodb.internal.connection.DefaultServerConnection.command(DefaultServerConnection.java:131)
	at com.mongodb.operation.MixedBulkWriteOperation.executeCommand(MixedBulkWriteOperation.java:419)
	at com.mongodb.operation.MixedBulkWriteOperation.executeBulkWriteBatch(MixedBulkWriteOperation.java:257)
	at com.mongodb.operation.MixedBulkWriteOperation.access$700(MixedBulkWriteOperation.java:68)
	at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:201)
	at com.mongodb.operation.MixedBulkWriteOperation$1.call(MixedBulkWriteOperation.java:192)
	at com.mongodb.operation.OperationHelper.withReleasableConnection(OperationHelper.java:424)
	at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:192)
	at com.mongodb.operation.MixedBulkWriteOperation.execute(MixedBulkWriteOperation.java:67)
	at com.mongodb.client.internal.MongoClientDelegate$DelegateOperationExecutor.execute(MongoClientDelegate.java:193)
	at com.mongodb.client.internal.MongoCollectionImpl.executeInsertMany(MongoCollectionImpl.java:520)
	at com.mongodb.client.internal.MongoCollectionImpl.insertMany(MongoCollectionImpl.java:504)
	at com.mongodb.client.internal.MongoCollectionImpl.insertMany(MongoCollectionImpl.java:499)
	at org.hibernate.ogm.datastore.mongodb.MongoDBDialect.flushInserts(MongoDBDialect.java:1777)
	at org.hibernate.ogm.datastore.mongodb.MongoDBDialect.executeBatch(MongoDBDialect.java:1596)
	at org.hibernate.ogm.dialect.impl.ForwardingGridDialect.executeBatch(ForwardingGridDialect.java:188)
	at org.hibernate.ogm.dialect.impl.BatchOperationsDelegator.executeBatch(BatchOperationsDelegator.java:79)
	at org.hibernate.ogm.dialect.eventstate.impl.EventStateLifecycles$OperationsQueueLifecycle.onFinish(EventStateLifecycles.java:111)
	at org.hibernate.ogm.dialect.eventstate.impl.EventStateLifecycles$OperationsQueueLifecycle.onFinish(EventStateLifecycles.java:88)
	at org.hibernate.ogm.dialect.eventstate.impl.EventContextManager.onFinish(EventContextManager.java:94)
	at org.hibernate.ogm.dialect.eventstate.impl.EventContextManager.onEventFinished(EventContextManager.java:83)
	at org.hibernate.ogm.dialect.eventstate.impl.EventContextManagingFlushEventListener.onFlush(EventContextManagingFlushEventListener.java:42)
	at org.hibernate.internal.SessionImpl.doFlush(SessionImpl.java:1454)
	at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:511)
	at org.hibernate.internal.SessionImpl.flushBeforeTransactionCompletion(SessionImpl.java:3283)
	at org.hibernate.internal.SessionImpl.beforeTransactionCompletion(SessionImpl.java:2479)
	at org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.beforeTransactionCompletion(JdbcCoordinatorImpl.java:473)
	at org.hibernate.ogm.transaction.impl.ForwardingTransactionCoordinatorOwner.beforeTransactionCompletion(ForwardingTransactionCoordinatorOwner.java:37)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.beforeCompletionCallback(JdbcResourceLocalTransactionCoordinatorImpl.java:178)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl.access$300(JdbcResourceLocalTransactionCoordinatorImpl.java:39)
	at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:271)
	at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:98)
	... 1 more
2

I made a repo to show the issue.
I’m sure I’m missing something.

3

I think the problem is that you added the GridFS field in an embeddable.
I think this might be a bug in OGM but
It should work if you use an entity instead.

Here an example:

@Entity
@DiscriminatorValue("OgmBigListEvent")
public class OgmBigListEvent extends BigEvent implements Serializable{

    @Id
    private Long id;
	
    @OneToOne
    private OgmBigListEventBody ogmEventBody;

@Entity
public class OgmBigListEventBody implements Serializable{

	@Id
	@GeneratedValue
	private Long id;

	private String name;
	
        GridFS list;

Or you could avoid using the body object and put everything in the event.

4

Some additional notes:

  • It’s bad practice to use primitives for ids. You should use Long instead of long
  • You forgot the @Id annotation in the OgmBigListEvent entity
  • Remember to close the factory at the end of the application, even in case of exception
  • If you go for the OneToOne solution, remember to persist both the event and the body
5

the class structure should stay as I defined (I do not choose this weird Entity-body structure by myself)
so can’t I use GridFS in @Embedded class? Are you sure?

6

so here the situation: I make it works the OneToOne solution (not sure if in its best way performance-wise).
In order to store huge lists of pojo object, I have 2 options:

  • serialize the list in byte[] and store in a GridFS field
  • use OneToMany relation between Body Entity and List<RowElement>

in your opinion (I haven’t test yet) which is the best option performance-wise (the lists are about 250k java objects of 10-20 fields)

7

It’s not something I can answer. You need to write some performance tests and check if mongodb can handle a list of object of that size for your use cases. If it can, I wouldn’t go for the GridFs solution unless it performs better.

8

why couldn’t MongoDB handle a Collection of 250k elements?

9

Sorry, I misunderstood what you were trying to do.

I don’t have enough information in regard to your use cases and hardware, so it’s kind of hard for me to make assumptions related to performances. You should write some tests and see for yourself.

That said, if you store the collection of elements using GridFS you have to read and write (and serialize) all the entries each time you want to access or modify the collection.

So, I’m incline to say that an association will perform better even if you have to store the elements in different documents.