I’m not able to get this custom failure handler working. In my test, Hibernate Search is configured to work with a ElasticSearch instance and when I shutdown the ElasticSearch instance I expect this failure handle to be called. Or maybe I’m wrong and won’t be called in this scenario. Can anyboy tell me?
This is the implementation which is based on the example found at reference documentation:
public class HibernateSearchFailureHandler implements FailureHandler {
private static final Logger log = LoggerFactory.getLogger(HibernateSearchFailureHandler.class);
@Override
public void handle(FailureContext context) {
String failingOperation = context.failingOperation().toString();
Throwable throwable = context.throwable();
log.error("FailureContext: {}", failingOperation, throwable);
}
@Override
public void handle(EntityIndexingFailureContext context) {
String failingOperation = context.failingOperation().toString();
Throwable throwable = context.throwable();
List<String> entities = new ArrayList<>();
for (Object entityReference : context.entityReferences()) {
entities.add(entityReference.toString());
}
log.error("EntityIndexingFailureContext: {} => {}", failingOperation, entities, throwable);
}
}
BTW, in this documentation prepends the “class:” string and in other parts in the same documentation only put the full class with package. Are both the same thing or which one is the correct. In any caso none of them worked.
I’d have to know which failing operation you are talking about exactly.
But in general, yes, the background failure handler only gets called for failures happening in the background, i.e. for which you wouldn’t normally get an exception in the user thread.
If your goal is to ignore indexing errors, first I wouldn’t recommend that, as you would eventually end up with an out-of-sync index. This might not matter if you reindex every night, though.
Second, there are other options at your disposal:
If you set the automatic indexing synchronization strategy to async, indexing will become de facto a “background” operation and failures will get passed to the background failure handler.
With the outbox-polling coordination strategy, you could push entity change events to a queue in the database, and have them processed in the background, so that your HTTP request processing threads are unaffected by Elasticsearch not being available. However, if it stays unavailable for a long time, events will pile up in the database and this might become a problem.
Alternatively, I’ve seen people check whether Elasticsearch is up on Spring Boot’s startup, and generate some configuration dynamically to disable Hibernate Search if it’s not. This only works if you don’t expect Elasticsearch to become unavailable after startup, though.
As to ignoring search errors, you’ll have to catch exceptions and do it by hand. Using an abstraction layer or aspect-oriented programming might help.
The problem was that I didn’t change the automatic indexing synchronization strategy to async.
BTW, I’m not sure if I understood, but chossing async will just keep the queue of items to be processed in memory and to avoid memory issues I should use the outbox-polling coordinator strategy?
Yes, though the queue is there whatever happens. async mostly means your HTTP request thread will go on without checking whether indexing succeeded, it’ll just let that run in the background.
Ok, but this would only happen with ElasticSearch. In case of Lucene backend I’ve had OutOfMemory problems under heavy load. I suppose Lucene (or the computer disk) were not fast enought to process all these requests.
I’d expect such errors to come from entity processing (i.e. when going through the entity graph to extract all the information to index), and that would happen for both Lucene and Elasticsearch. In any case, yes, outbox-polling coordination could help with this since you would be able to scale HTTP processing independently from indexing, e.g. have 100 HTTP request processing threads and only 1-2 indexing threads. See Hibernate Search 6.2.2.Final: Reference Documentation
I could be more useful with a reproducer, but I understand that’s hard.