Description
We have basically a spring boot project extended with Hibernate Search and a Lucene backend. Because of the large number of entities processed in single transactions we would like to migrate from automatic indexing to outbox-polling coordination strategy. This would increase reactivity of our application, as not all lazy mappings need to update the indexes to be initialized in the one transaction.
However when configuring outbox-polling (through spring-boot mapped configs, bundled in the application.yaml file) the following config lead to fatal exceptions during startup. Note that we did leave automatic_indexing configs present (but disabled) during the migration:
spring:
datasource:
url: jdbc:h2:file:./test
driverClassName: org.h2.Driver
jpa:
hibernate:
ddl-auto: create-drop
properties:
hibernate:
search:
automatic_indexing:
enabled: false
synchronization:
strategy: async
coordination:
strategy: outbox-polling
Producing the following exception:
org.hibernate.search.util.common.SearchException: HSEARCH000501: Invalid value for configuration property 'hibernate.search.automatic_indexing.synchronization.strategy': 'async'. HSEARCH800042: Cannot customize the synchronization strategy: the selected automatic indexing strategy always processes events asynchronously, through a queue.
My understanding is, outbox-polling and automatic-indexing are mutually exclusive, so I would expect this configuration to be ignored (especially as automatic indexing is disabled). However, all works fine if the automatic-indexing configuration field is simply not defined.
spring:
datasource:
url: jdbc:h2:file:./test
driverClassName: org.h2.Driver
jpa:
hibernate:
ddl-auto: create-drop
properties:
hibernate:
search:
coordination:
strategy: outbox-polling
I am asking, would it make sense in case of automatic_indexing.enabled: false to ignore all remaining configuration? It sounds logically to me, as you can also leave coordination.strategy: none in the configs with automatic indexing enabled.
As the hibernate search configuration fields require automatic_indexing to be absent if coordination.strategy: outbox-polling is set, could you note this requirement in the online documentation?
Dependencies
- Hibernate ORM 5.6.5.Final
- Hibernate Search 6.1.8.Final
- Apache Lucene (bundled) 8.11.1
- Spring Boot 2.7.0
Hey Andreas, thanks for sharing the feedback with us.
The term “automatic indexing” was a bit overused in the past, and we’ve tried to address that in the newer versions of Hibernate Search (in 6.2 in particular).
In this case, it is a bit misleading, and users may assume that the synchronization.strategy is only relevant for the “automatic indexing”. We have renamed this property to hibernate.search.indexing.plan.synchronization.strategy. Now as to why – this strategy defines how the indexing plan is synchronized, and indexing plans are used in both coordination strategies. In the case of a “none” coordination strategy, the indexing plan is built from the Hibernate ORM events, and that results in this “automatic indexing”, while in the case of an “outbox-polling” coordination strategy, the plan is built by the background event consumer in its own thread.
Hopefully, this gives you a bit more insight into what’s going on with the indexing plan synchronization and the changes we’ve made in the newer version will eliminate any confusion for other users.
As for adding a tip to the documentation – that sounds like a nice idea, I’ll add that to our backlog.
Yeah, it’s as Marko explained.
automatic_indexing.enabled disables only part of the features related to indexing plans, basically “listener-triggered” indexing. There’s still the possibility of using indexing plans manually, and then we do perform synchronization of the indexing plan with your transaction, so the synchronization strategy is still relevant.
Hopefully the situation will be a tad clearer with the renamings we did in 6.2:
hibernate.search.automatic_indexing.synchronization.strategy is now deprecated in favor of hibernate.search.indexing.plan.synchronization.strategy.hibernate.search.automatic_indexing.enabled is now deprecated in favor of hibernate.search.indexing.listeners.enabled.
And yes, +1 to improve documentation; do you have a link to that issue, @mbekhta ?
Thanks for the reminder (it was still on the todo list 
)! Here’s the ticket: [HSEARCH-4909] - Hibernate JIRA
thx, yes this does make it a lot clearer and the decision to reorganize the propertiesmakes sense from this perspective. For that reason we also consider upgrading to the latest 6.2 version - however the current incubating status of the ‘outbox-polling’ coordination strategy is seen as a risk within our team. A risk of putting the feature in production code, although our application would greatly benefit from its increase in reactivity. Do you think the outbox-polling feature has a chance to become stable in the foreseeable future?
@Medexter_AC As far as we’re concerned the outbox-polling coordination strategy is ready for use in production.
It’s marked as incubating mainly because we’re still waiting for more feedback, some of which may require that we make changes to the DB schema of the corresponding tables. However we always document how to migrate the schema of mainstream database vendors in such case, you can see an example here for 6.2, and on more here for the upcoming 7.0 (though only for PostgreSQL in this case).
As to when we’ll officially drop preview/incubating status… We haven’t reached a decision yet, and as a rule we don’t make promises regarding dates, sorry.