In our specific use case we are using the ES implementation, but this would also be applicable when using a file based index.
I was wondering if it is possible whilst updating/creating entities through hibernate the app is shutdown which causes updates to be written to the DB without updating the index? If so is there a way of protecting against this, as we update our data every 30 mins and do near daily releases, and we don’t want the index returning stale data. Thanks!
I’m sorry, I don’t understand. If your application is shut down, how could it be writing to the database?
Sorry, wasn’t very clear with the question. If we are in the process of updating our data and writing them to the DB and the app is then shutdown mid process, is there a chance it will go to the DB but not the index? Or is the whole process transactional?
If the application breaks down or is shut down during a transaction, you’re safe. If it breaks down or is shut down just after a transaction, while indexing, then yes, there is a chance the index will be out of sync, because 1. Elasticsearch doesn’t support transactions and 2. Even if it did, Hibernate Search would have to take additional measures to ensure that database transactions and Elasticsearch transactions are synchronized, which it doesn’t.
You can take steps to avoid that kind of problem, in particular when you release your application: first stop accepting requests, wait for all ongoing requests to be processed, then shut down the application cleanly. Hibernate Search, when shut down cleanly, will wait for all ongoing indexing works to finish before stopping.
Regarding unforeseen application breakdowns or physical failures, there are no solutions at the moment. You might however be interested in this other thread; another community member is currently exploring a solution where indexing happens asynchronously: indexing events are pushed to a database table in the same transaction as your entity changes, and indexing happens asynchronously in a background process. This would effectively ensure indexes will always be in sync eventually, even if a physical failure happens.
If you’re interested in this solution, you can join the discussion here: https://hibernate.zulipchat.com/#narrow/stream/132092-hibernate-search-dev/topic/Async.20event.20processing.20using.20outbox.20pattern
Thanks for the help, unforeseen breakdowns and failures are rare luckily, so we should be ok making sure that we shutdown the application cleanly. Really appreciate the help and the quick reply 