Elasticsearch index alias opt-out?

Hibernate Search
#1

I have been upgrading from Beta2 to CR1 and I ran into this issue. Beta2 wrote to, and read from, the index by name directly. There was no support for aliases. Now that aliases are supported, there seems to be no way to not use aliases. In an attempt to get my application using CR1 the same as it was on Beta2, I created a custom IndexLayoutStrategy that just always returned the index name. However, startup fails because of the check in ElasticsearchBackendImpl#createIndexNames that doesn’t allow the write index to be the same as the read index.

I know the use of aliases is good, and I was part of the request for their support (Hibernate Search 6 Index Aliases), but there is no requirement that I am aware of in Elasticsearch that forces the use of aliases.

Is there a way to avoid the use of aliases and write to the index directly similar to what was done in Beta2? Or will I need to go through my existing indexes and add aliases for them to get my application to work again?

From a option side of things, I don’t think Hibernate Search for Elasticsearch should force a usage pattern with Elasticsearch. I think it should provide reasonable defaults but allow the developer to choose how they want Elasticsearch to be configured, written to, and queried. Having data in Elasticsearch provides much more use (e.g. analytics, data visualizations, etc.) other than just being able to perform search as part of application features. Being able to disable schema management is just one part of this.

#2

While not required, they are necessary for several important maintenance features. As explained here, aliases are necessary if you want to execute zero-downtime maintenance operations later on. As this is a rather obscure configuration option for most users, it was decided to make aliases the default, to make sure that everyone starts with a future-proof setup. I’m sure you are painfully aware how starting out without aliases can be once you end up having to add these aliases…

Yes, in Hibernate Search 6.0.0.CR1 you must have aliases for all your indexes. Upgrading from Beta2 to the final release will unfortunately require some changes to your application.
That’s definitely not the kind of change we will introduce now after 6.0 becomes stable, but during the beta, it was deemed a necessary and acceptable change.

Several solutions:

  1. Drop your indexes completely, have Hibernate Search re-create them, and re-populate them using mass indexing. While it’s a bit extreme, it’s also the easiest solution.
  2. OR start your application with the property hibernate.search.schema_management.strategy set to create-or-update. Hibernate Search will add the aliases itself. Be careful though, there are risks, so you should try it on a copy of your production environment first. Don’t forget to set the property hibernate.search.schema_management.strategy back to a safer value afterwards.
  3. OR use the createOrUpdate operation on the schema management API. Same risks as above.
  4. OR run a script manually. Assuming you don’t have any of the aliases needed by Hibernate Search right now, this sh script should do:
    ES=http://localhost:9200
curl -XPOST "$ES/_aliases" -H "Content-Type: application/json" -d"{\"actions\": [$(curl "$ES/_cat/indices?h=index" | sed -E 's/.*/{"add":{"index":"\0", "alias":"\0-read", "is_write_index": false}},{"add":{"index":"\0", "alias":"\0-write", "is_write_index": true}}/;2,$i ,')]}"
curl -XGET http://localhost:9200/_cat/aliases\?pretty

Ideally, yes it should allow deep customization. Practically, there are limits to the amount of different configurations we can maintain properly and test regularly, especially when some of them (this one in particular) have effects on every single request sent by Hibernate Search. So we start small, and expand with customization options as necessary. That’s what we did in this case.

#3

Thank you for your response.

I’m sure you are painfully aware how starting out without aliases can be once you end up having to add these aliases…

I personally haven’t experienced any pain from this. I’ve used and managed multiple high performance Elasticsearch clusters for the last 5 years and we’ve rarely used aliases. However, that is not how i would do it today. Most cluster operations were fairly easy to perform with zero downtime. On the rare occasion of needing to perform major changes, it wasn’t an issue for us to take the system down for a bit. But I understand our use cases are a subset of all use cases.

Several solutions: …

For me, option 4 is probably the easiest as I will not need to delete and recreate any of my data. I don’t use any of the mass indexing or index management features provided by Hibernate Search. We prefer to manage our clusters outside of our application runtime. With this, my easiest option is to create a migration script to add aliases to each index and update my code accordingly. I actually disable schema_management completely (set to none) and just rely on the read/write capabilities of HS.

Ideally, yes it should allow deep customization

I’ll create a feature request for this.

#4

Issue created https://hibernate.atlassian.net/browse/HSEARCH-4109