Hibernate search migration - elastic search backend error on startup

harpreetKaur · July 29, 2021, 6:12pm

Hi,
I am trying to migrate my project from hibernate search 5.11.5 to hibernate search 6.0.5. I am just using the elastic search backend but I get below error due to presence of multiple backend types in classpath.

classpath:

<?xml version="1.0" encoding="UTF-8"?>

Hibernate.cfg.xml :
my hibernate properties

<!-- Uncomment this if Elastic Server is OFF -->

	<property name="hibernate.search.backend.type">elasticsearch</property> 
	<property name="hibernate.search.backend.hosts">127.0.0.1:9400</property>
	<property name="hibernate.search.backend.protocol">https</property> 
	<property name="hibernate.search.schema_management.strategy">none</property>
	<property name="hibernate.search.backend.schema_management.minimal_required_status">yellow</property>
	<property name="hibernate.search.backend.log.json_pretty_printing">true</property>

Exception on tomcat startup:
org.hibernate.search.util.common.SearchException: HSEARCH000520: Hibernate Search encountered failures during bootstrap. Failures:

default backend: 
    failures: 
      - HSEARCH000582: Ambiguous backend type: configuration property 'hibernate.search.backend.type' is not set, and multiple backend types are present in the classpath. Set property 'hibernate.search.backend.type' to one of the following to select the backend type: [elasticsearch, lucene]
at org.hibernate.search.engine.reporting.spi.RootFailureCollector.checkNoFailure(RootFailureCollector.java:50)
at org.hibernate.search.engine.common.impl.SearchIntegrationBuilderImpl.prepareBuild(SearchIntegrationBuilderImpl.java:243)
at org.hibernate.search.mapper.orm.bootstrap.impl.HibernateOrmIntegrationBooterImpl.doBootFirstPhase(HibernateOrmIntegrationBooterImpl.java:259)
at org.hibernate.search.mapper.orm.bootstrap.spi.HibernateOrmIntegrationBooterBehavior.bootFirstPhase(HibernateOrmIntegrationBooterBehavior.java:17)
at org.hibernate.search.mapper.orm.bootstrap.impl.HibernateOrmIntegrationBooterImpl.lambda$bootNow$7(HibernateOrmIntegrationBooterImpl.java:218)
at java.util.Optional.orElseGet(Unknown Source)
at org.hibernate.search.mapper.orm.bootstrap.impl.HibernateOrmIntegrationBooterImpl.bootNow(HibernateOrmIntegrationBooterImpl.java:218)
at java.util.concurrent.CompletableFuture.uniApply(Unknown Source)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(Unknown Source)
at java.util.concurrent.CompletableFuture.postComplete(Unknown Source)
at java.util.concurrent.CompletableFuture.complete(Unknown Source)
at org.hibernate.search.mapper.orm.bootstrap.impl.HibernateSearchSessionFactoryObserver.sessionFactoryCreated(HibernateSearchSessionFactoryObserver.java:41)
at org.hibernate.internal.SessionFactoryObserverChain.sessionFactoryCreated(SessionFactoryObserverChain.java:35)
at org.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:385)
at org.hibernate.boot.internal.SessionFactoryBuilderImpl.build(SessionFactoryBuilderImpl.java:468)
at org.hibernate.boot.internal.MetadataImpl.buildSessionFactory(MetadataImpl.java:195)
at bphx.c2ab.hibernate.HibernateSessionFactory.createCachedSessionFactory(HibernateSessionFactory.java:310)
at bphx.c2ab.hibernate.HibernateSessionFactory.<init>(HibernateSessionFactory.java:201)
at bphx.c2ab.hibernate.HibernateSessionFactory.current(HibernateSessionFactory.java:172)
at com.csc.pt.servlet.common.PTBusinessServiceServlet.init(PTBusinessServiceServlet.java:130)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1134)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1089)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:983)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4876)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5185)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:717)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:690)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:705)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:631)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1831)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:526)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:425)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1576)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:309)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)
at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:423)
at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:366)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:936)
at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:841)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1384)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1374)
at java.util.concurrent.FutureTask.run(Unknown Source)
at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:909)
at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:262)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.StandardService.startInternal(StandardService.java:421)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:930)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.startup.Catalina.start(Catalina.java:633)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:343)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:474)

Hibernate:

yrodiere · August 2, 2021, 7:14am

I deleted my previous reply since you’re already using the hibernate.search.backend.type property that should solve the problem. I didn’t see that, sorry, it’s Monday

Are you sure the properties you’re setting are actually taken into account? Try setting hibernate.search.enabled to false to check that: if you’re still getting the error, then your configuration is definitely being ignored. If you’re no longer getting the error, there’s something else; please provide the full content of your hibernate.cfg.xml/persistence.xml/hibernate.properties.

Also, a reproducer could help, because our own integration tests don’t show this problem. There must be something specific in your setup that triggers that issue. Can you please provide a reproducer based on the test case templates?

As a temporary workaround, you can try removing any dependency to the Lucene backend. Know that the migration helper only works with the Lucene backend, as explained in the migration guide; it pulls the Lucene backend as a transitive dependency.

harpreetKaur · August 2, 2021, 6:23pm

@yrodiere Yes, found the root cause , there was a another cached hibernate cfg xml file, where it was expecting the backend type to be set. We didnt require the elastic setting for this file with hibernate search 5.11.5.

We are migrating our project to hibernate search 6.0.5 final version , and after resolving above error , I am getting below exception on tomcat startup due to dependency on elasticsearch-rest-client but none of the versions I have tried to resolve below error (I have tried rest client and sniffer 7.10.0 , 7.13.4 ,6.8.13, 6.8.17)

java.lang.NoSuchMethodError: org.elasticsearch.client.RestClient.performRequestAsync(Lorg/elasticsearch/client/Request;Lorg/elasticsearch/client/ResponseListener;)Lorg/elasticsearch/client/Cancellable;
at org.hibernate.search.backend.elasticsearch.client.impl.ElasticsearchClientImpl.send(ElasticsearchClientImpl.java:112)
at org.hibernate.search.backend.elasticsearch.client.impl.ElasticsearchClientImpl.lambda$submit$0(ElasticsearchClientImpl.java:82)
at org.hibernate.search.util.common.impl.Futures.lambda$create$0(Futures.java:44)
at java.util.concurrent.CompletableFuture.uniComposeStage(Unknown Source)
at java.util.concurrent.CompletableFuture.thenCompose(Unknown Source)
at org.hibernate.search.util.common.impl.Futures.create(Futures.java:44)
at org.hibernate.search.backend.elasticsearch.client.impl.ElasticsearchClientImpl.submit(ElasticsearchClientImpl.java:82)
at org.hibernate.search.backend.elasticsearch.client.impl.ElasticsearchClientUtils.tryGetElasticsearchVersion(ElasticsearchClientUtils.java:64)
at org.hibernate.search.backend.elasticsearch.client.impl.ElasticsearchClientUtils.getElasticsearchVersion(ElasticsearchClientUtils.java:53)
at org.hibernate.search.backend.elasticsearch.impl.ElasticsearchLinkImpl.initVersion(ElasticsearchLinkImpl.java:200)
at org.hibernate.search.backend.elasticsearch.impl.ElasticsearchLinkImpl.onStart(ElasticsearchLinkImpl.java:142)
at org.hibernate.search.backend.elasticsearch.impl.ElasticsearchBackendFactory.create(ElasticsearchBackendFactory.java:115)
at org.hibernate.search.engine.common.impl.IndexManagerBuildingStateHolder.createBackend(IndexManagerBuildingStateHolder.java:128)
at org.hibernate.search.engine.common.impl.IndexManagerBuildingStateHolder.createBackends(IndexManagerBuildingStateHolder.java:69)
at org.hibernate.search.engine.common.impl.SearchIntegrationBuilderImpl.prepareBuild(SearchIntegrationBuilderImpl.java:241)
at org.hibernate.search.mapper.orm.bootstrap.impl.HibernateOrmIntegrationBooterImpl.doBootFirstPhase(HibernateOrmIntegrationBooterImpl.java:259)
at org.hibernate.search.mapper.orm.bootstrap.spi.HibernateOrmIntegrationBooterBehavior.bootFirstPhase(HibernateOrmIntegrationBooterBehavior.java:17)
at org.hibernate.search.mapper.orm.bootstrap.impl.HibernateOrmIntegrationBooterImpl.lambda$bootNow$7(HibernateOrmIntegrationBooterImpl.java:218)
at java.util.Optional.orElseGet(Unknown Source)
at org.hibernate.search.mapper.orm.bootstrap.impl.HibernateOrmIntegrationBooterImpl.bootNow(HibernateOrmIntegrationBooterImpl.java:218)
at java.util.concurrent.CompletableFuture.uniApply(Unknown Source)
at java.util.concurrent.CompletableFuture$UniApply.tryFire(Unknown Source)
at java.util.concurrent.CompletableFuture.postComplete(Unknown Source)
at java.util.concurrent.CompletableFuture.complete(Unknown Source)
at org.hibernate.search.mapper.orm.bootstrap.impl.HibernateSearchSessionFactoryObserver.sessionFactoryCreated(HibernateSearchSessionFactoryObserver.java:41)
at org.hibernate.internal.SessionFactoryObserverChain.sessionFactoryCreated(SessionFactoryObserverChain.java:35)
at org.hibernate.internal.SessionFactoryImpl.(SessionFactoryImpl.java:385)
at org.hibernate.boot.internal.SessionFactoryBuilderImpl.build(SessionFactoryBuilderImpl.java:468)
at org.hibernate.boot.internal.MetadataImpl.buildSessionFactory(MetadataImpl.java:195)
at bphx.c2ab.hibernate.HibernateSessionFactory.createCachedSessionFactory(HibernateSessionFactory.java:310)
at bphx.c2ab.hibernate.HibernateSessionFactory.(HibernateSessionFactory.java:201)
at bphx.c2ab.hibernate.HibernateSessionFactory.current(HibernateSessionFactory.java:172)
at com.csc.pt.servlet.common.PTBusinessServiceServlet.init(PTBusinessServiceServlet.java:130)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1134)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1089)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:983)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4876)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5185)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:717)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:690)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:705)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:631)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1831)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:526)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:425)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1576)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:309)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)
at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:423)
at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:366)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:936)
at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:841)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1384)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1374)
at java.util.concurrent.FutureTask.run(Unknown Source)
at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:909)
at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:262)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.StandardService.startInternal(StandardService.java:421)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:930)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.startup.Catalina.start(Catalina.java:633)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:343)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:474)

Hibernate search elasticsearch backend cant seem to find this function in any version of elasticsearch rest client jar

yrodiere · August 3, 2021, 6:47am

Well, the method is there: elasticsearch/client/rest/src/main/java/org/elasticsearch/client/RestClient.java at 7.13 · elastic/elasticsearch · GitHub , and it’s been there since 7.5: elasticsearch/client/rest/src/main/java/org/elasticsearch/client/RestClient.java at 7.5 · elastic/elasticsearch · GitHub

Generally I’d recommend letting Maven do the dependency resolution for you, and aligning on the dependencies declared by the Hibernate Search POM.

But if you really want to manage your JARs manually, then I’d say you have a “stray” JAR somewhere; probably another, older version of the Elasticsearch Client JAR is hiding somewhere and being scanned for classes before the ones you just added.

harpreetKaur · August 3, 2021, 10:49am

@yrodiere Yes, we are maintaining manual dependencies in our project. And I am trying to migrate to elastic search and hibernate search 6 from 5 series.

I have migrated to hibernate search 6.0.5 final version and for that I am trying to use the elastic search rest client and sniffer version 6.8.17 .
On tomcat startup it tries to find the elastic search version and tried to hit below code
restClient.performRequestAsync(
toRequest( elasticsearchRequest, entity ),
new ResponseListener() {
@Override
public void onSuccess(Response response) {
completableFuture.complete( response );
}
@Override
public void onFailure(Exception exception) {
if ( exception instanceof ResponseException ) {
/*
* The client tries to guess what’s an error and what’s not, but it’s too naive.
* A 404 on DELETE is not always important to us, for instance.
* Thus we ignore the exception and do our own checks afterwards.
*/
completableFuture.complete( ( (ResponseException) exception ).getResponse() );
}
else {
completableFuture.completeExceptionally( exception );
}
}
}
);

But I end up getting the exception : java.lang.NoSuchMethodError: org.elasticsearch.client.RestClient.performRequestAsync(Lorg/elasticsearch/client/Request;Lorg/elasticsearch/client/ResponseListener;)Lorg/elasticsearch/client/Cancellable;
at org.hibernate.search.backend.elasticsearch.client.impl.ElasticsearchClientImpl.send(ElasticsearchClientImpl.java:112)

And on github, I can only see the void type definition for performRequestAsync, not the cancellable one ( as mentioned it was introduced from 7.5 version onwards)

github.com

elastic/elasticsearch/blob/6.8/client/rest/src/main/java/org/elasticsearch/client/RestClient.java#L310

    
      
              return performRequest(request);
          }
          
          
/**
           * Sends a request to the Elasticsearch cluster that the client points to and waits for the corresponding response
           * to be returned. Shortcut to {@link #performRequest(String, String, Map, HttpEntity, HttpAsyncResponseConsumerFactory, Header...)}
           * which doesn't require specifying an {@link HttpAsyncResponseConsumerFactory} instance,
           * {@link HttpAsyncResponseConsumerFactory} will be used to create the needed instances of {@link HttpAsyncResponseConsumer}.
           *
           * @param method the http method
           * @param endpoint the path of the request (without host and port)
           * @param params the query_string parameters
           * @param entity the body of the request, null if not applicable
           * @param headers the optional request headers
           * @return the response returned by Elasticsearch
           * @throws IOException in case of a problem or the connection was aborted
           * @throws ClientProtocolException in case of an http protocol error
           * @throws ResponseException in case Elasticsearch responded with a status code that indicated an error
           * @deprecated prefer {@link #performRequest(Request)}
           */
          @Deprecated

This confuses me since hibernate search 6 is compatible with 6.8 elastic search , so I figured the two versions I am using should work .Yet I still see it cant find the method.

yrodiere · August 3, 2021, 11:03am

Hibernate Search is compatible with Elasticsearch server 6.8.

The Elasticsearch client dependency is fixed and you should not deviate from the version Hibernate Search depends on. The Elasticsearch client in version 7.10 will still work with an Elasticsearch server in version 6.8.

The Elasticsearch server (cluster), on the other hand can be in any supported version.

EDIT: Note that I’m referring to the Elasticsearch client that Hibernate Search uses, i.e. the low-level REST client. If you have an additional dependency to the high-level REST client, things get much more complicated, since that client (which Hibernate Search doesn’t use) is very, very coupled to the Elasticsearch server version, so the high-level REST client version 7.10 is unlikely to work with an Elasticsearch server in version 6.8.

harpreetKaur · August 3, 2021, 12:02pm

yrodiere:

Hibernate Search is compatible with Elasticsearch server 6.8.

The Elasticsearch client dependency is fixed and you should not deviate from the version Hibernate Search depends on. The Elasticsearch client in version 7.10 will still work with an Elasticsearch server in version 6.8.

The Elasticsearch server (cluster), on the other hand can be in any supported version.

EDIT : Note that I’m referring to the Elasticsearch client that Hibernate Search uses, i.e. the low-level REST client . If you have an additional dependency to the high-level REST client, things get much more complicated, since that client (which Hibernate Search doesn’t use) is very, very coupled to the Elasticsearch server version, so the high-level REST client version 7.10 is unlikely to work with an Elasticsearch server in version 6.8.

Thanks for pointing out @yrodiere . i was using elasticsearch server 6.8 / hibernate search 6.0.5 and elastic search client 6.8.17 . Which led to this error , on upgrading the client dependency to 7.10.0, this issue is resolved for the application that doesnt use rest high level client. But one of our project makes use of RestHighLevelClient , so I think elasticsearch server 6.8 and elastic search client 7.10.0 might not work for that.

yrodiere · August 3, 2021, 12:06pm

Indeed, it’s unlikely that the high-level REST client version 6.8 works with the low-level REST client version 7.10. I’d advise to upgrade to at least Elasticsearch server version 7.5 (but ideally 7.10).

harpreetKaur · August 5, 2021, 2:18pm

We did decide to upgrade to elastic search server 7.10.0. And were able to drop and create indexes on startup and later used massindexer .
But the index name have a number appended to them.
@Entity
@Indexed(index=“ASBACPP”)// index name provided
public class AsbacppTO

but after startup below is the index I get :
yellow open asbacpp-000001 oIbym6OyRReyrBFAnSNuKQ 1 1 3382 0 226.4kb 226.4kb

Earlier we didnt have this -000001 appended to our index names.
We dont have any custom indexing property set, but is there any configuration property in hibernate search 6 that could help me only have the class name as the index name.

yrodiere · August 5, 2021, 2:20pm

It’s because of the new index layout strategies in Hibernate Search 6.
See Hibernate Search 6.0.6.Final: Reference Documentation

harpreetKaur · August 5, 2021, 2:59pm

thanks for the quick reply. I missed this section of the document

Topic		Replies	Views
HSEARCH000582: Ambiguous backend type: configuration property 'hibernate.search.backend.type' is not set Hibernate Search	3	822	January 2, 2023
Hibernate Search + Elastic Search Hibernate Search	12	1869	September 15, 2020
Robust Hibernate Configuration Hibernate Search	11	646	March 5, 2020
HibernateSearch Error while trying to fetch data Hibernate Search	4	2231	November 24, 2020
Migration hibernate search from 5.11 to 6.0.10 final Hibernate Search	3	639	January 13, 2023

Hibernate search migration - elastic search backend error on startup

Related topics