Jparepository saveAll() drives to bad performance for indexed object

[Tony]

Hi. Using hibernate search 6.1.8 and calling JpaRepository, repository.saveAll(list) where the list contains >20k records i notice that saveAll has too bad performance because it updates the indexes of my object as well.
Is there any way to improve the performance somehow ?

@Getter
@Setter
@RequiredArgsConstructor
@Entity
@Indexed(index = "idx_object")
@DynamicUpdate
@Table(name = "object")
public class Object extends BaseEntity {
...
}

Thank you.

[mbekhta]

Hey Tony,

@Tony

where the list contains >20k records

Persisting 20k records at a time is already problematic, and adding indexing on top of that doesn’t help. Depending on your use case, you could try:

• doing this in smaller batches, see Hibernate Search 7.2.1.Final: Reference Documentation ( Example 159/ Example 160).
• disable the listener-triggered indexing (either for this update or entirely) and use mass indexer to index your records. Hibernate Search 7.2.1.Final: Reference Documentation / Hibernate Search 7.2.1.Final: Reference Documentation
• use outbox-polling to offload indexing to a background process: Hibernate Search 7.2.1.Final: Reference Documentation

But… indexing of 20k records will have to happen somewhere at some point, be it right away, later or in chunks. Hence, I’d suggest looking into how to reduce the number of entities updated/created, and if it really has to be 20k at a time … then the expectations just have to be adjusted it’s going to take time …