Need to limit auto reindexing when only excluded properties change

[yrodiere]

@busitech

In the case where B changes, A, C and D are all reindexed into A even though neither C nor D changed at all

Right. Don’t get me wrong, in Search 6 we’ll still need to load C and D when we actually need to reindex A. But at least we will only reindex A when we actually need to.

@busitech

It gets even more unruly when B changes and the changed property isn’t included in any index, yet reindexing is triggerred anyway…

That should only happen when optimization is disabled, which (IIRC) only happens if you rely on class bridges. In Search 6, the new TypeBinder allows you to specify which properties you depend on.

@busitech

It would be nice if lucene could be instructed to replace the value of only changed properties during reindexing while leaving the rest alone. Then the attributes representing properties from C and D could remain the same, to avoid reading those records from the database, while the changed properties of B are refreshed in A.

It would be interesting indeed, but rather complex, and I’m not even sure it’s entirely possible. As far as I know, Lucene only allows such “per-field updates” for doc-values, but not for indexes or storage. That means it would only work for fields designed to be sorted/aggregated on, but not searched on nor projected on. Related: HSEARCH-1236

@busitech

Batch size does not seem to be considered in Hibernate Search 5, as we’re seeing records being read from the database one at a time, even though thousands of records of the same type need to be retrieved

That’s interesting, never heard of that. May be worth investigating, and if you can create a reproducer, opening a ticket?

@busitech

It would be terrific to be able to specify which indexes have a higher priority than others

Even without fully asynchronous indexing, in a closer future, you might be interested in HSEARCH-168. It’s about disabling automatic indexing for specific indexes. Then you can plan mass indexing periodically (e.g. overnight) for these indexes. It’s currently planned for 6.0.0.Final, though I’m not very optimistic about it; we might have to delay it to 6.1.