Hibernate search and elastics search are on a boat


#1

But this boat don’t navigate together very well…

Hi all, i have see this video https://www.youtube.com/watch?v=pQJY-Mu-yGA

In an old application we have a lot of function on postgresql to make some statistics and this take very long time.
With this video we can take the powerfull of hibernate search (spring data + hibernate + jpa repository) to index directly some data in elastics search -> that’s works very well ! thank a lot !

But we need to retrieve some statistics (search, agregation, filter) to display some graph, i will use the ElasticsSearch REST HighLevel Http client for that

https://mvnrepository.com/artifact/org.elasticsearch.client/elasticsearch-rest-high-level-client

But i have a conflict in maven dependencies. A old version (ES HTTP client: 5.6.7 include elastic search 2.4.5, Lucenne 5.5.4 -> java.lang.NoClassDefFoundError: org/apache/lucene/document/Field$TermVector) of ElasticSearch Http Client is used for hibernate Search with an old lucene version. Can you update the dependencies in the next version for be able to use the power of hibernate search and elastics search (query to retrieve entity with your’s project and be able to extract data statistics with elastics search REST api).

Thank’s,

I don’t have writte a message in english since a long time ago, sorry if my english is not perfect.


#2

Hi @alexis-puska ,

So, the big issue is that the Elasticsearch high level rest client depends on Elasticsearch itself. This is unfortunate.

The other unfortunate thing is that we won’t be able to upgrade the Lucene dependency of Search for now. We are working on Search 6 but it’s still in very early stages.

That being said, I wonder if you could make it work by excluding the Lucene dependency brought by the Elasticsearch client (not the one coming from Hibernate Search, of course). From what I see, the client is just using the Elasticsearch dependency to easily deserialize the JSON in Java objects so it might not use Lucene at all.

Try that and come back to us, I hope it will help.


#3

I will try this on the next week, i keep you in touch.


#4

I have had some time to make this test. I have exclude of the elastic package the dependencies to Lucene. But not works.

org.hibernate hibernate-search-orm 5.8.1.Final org.hibernate hibernate-search-elasticsearch 5.8.1.Final
	<dependency>
	    <groupId>org.elasticsearch</groupId>
	    <artifactId>elasticsearch</artifactId>
	    <version>5.6.7</version>
	    <exclusions>
			<exclusion>
			    <groupId>org.apache.lucene</groupId>
		    	<artifactId>lucene-core</artifactId>
		    </exclusion>
		    <exclusion>
				<groupId>org.apache.lucene</groupId>
			    <artifactId>lucene-analyzers-common</artifactId>
			</exclusion>
		    <exclusion>
			    <groupId>org.apache.lucene</groupId>
			    <artifactId>lucene-backward-codecs</artifactId>
		    </exclusion>
		    <exclusion>
			    <groupId>org.apache.lucene</groupId>
			    <artifactId>lucene-grouping</artifactId>
		    </exclusion>
		    <exclusion>
			    <groupId>org.apache.lucene</groupId>
			    <artifactId>lucene-highlighter</artifactId>
		    </exclusion>
		    <exclusion>
			    <groupId>org.apache.lucene</groupId>
			    <artifactId>lucene-join</artifactId>
		    </exclusion>
		    <exclusion>
			    <groupId>org.apache.lucene</groupId>
			    <artifactId>lucene-memory</artifactId>
		    </exclusion>
		    <exclusion>
			    <groupId>org.apache.lucene</groupId>
			    <artifactId>lucene-misc</artifactId>
		    </exclusion>
		    <exclusion>
			    <groupId>org.apache.lucene</groupId>
			    <artifactId>lucene-queries</artifactId>
		    </exclusion>
		    <exclusion>
			    <groupId>org.apache.lucene</groupId>
			    <artifactId>lucene-queryparser</artifactId>
		    </exclusion>
		    <exclusion>
			    <groupId>org.apache.lucene</groupId>
			    <artifactId>lucene-sandbox</artifactId>
		    </exclusion>
		    <exclusion>
			    <groupId>org.apache.lucene</groupId>
			    <artifactId>lucene-spatial</artifactId>
		    </exclusion>
		    <exclusion>
			    <groupId>org.apache.lucene</groupId>
			    <artifactId>lucene-spatial-extras</artifactId>
		    </exclusion>
		    <exclusion>
			    <groupId>org.apache.lucene</groupId>
			    <artifactId>lucene-spatial3d</artifactId>
		    </exclusion>
		    <exclusion>
			    <groupId>org.apache.lucene</groupId>
				<artifactId>lucene-suggest</artifactId>
		    </exclusion>
		</exclusions>
	</dependency>
	<dependency>
	    <groupId>org.elasticsearch.client</groupId>
	    <artifactId>elasticsearch-rest-high-level-client</artifactId>
	    <version>5.6.7</version>
	</dependency>

Caused by: java.lang.NoSuchFieldError: LUCENE_6_0_0

The only solution for me that to reimplement all your work make in the dependencies hibernate search elastic search. (rebuild index, document indexation on JPA event (insert/delete/update), create index on startup if don’t exists, get id for full text search… or wait for the update of Lucene inside hibernateSearch.

One way hibernate search with elastic search, but the only way to use aggregation is the low level reste api
The second way, implement your works and use high rest client api to request for aggregation.

Or see with the team of elastic search for an Rest api without any Lucene dependencies.


#5

I have found a very ugly workaround, exclude all dependencies of Lucene is not enough. I have found an old version of the class Version or.apache.lucene.Version

I had a function missing and some static version number in this class definition, i have had this and finally the webApp works, finally the webApp works. HighRestClient object can be instantiate!

In waiting for other solution i think this can be able to continue to develop quickly.

I have send a message to the elasticsearch team to fix this high relation with lucene in this class. This library doesn’t need Lucene to communicate with elasticsearch for me…

Thank’s for the start of solution :slight_smile:


#6

@alexis-puska could you try to share a project based on https://github.com/hibernate/hibernate-test-case-templates/tree/master/search/hibernate-search-elasticsearch/hibernate-search-elasticsearch-5 where we can see the issue?

I think your use case is quite common and it would be nice if we could find a proper solution.

Thanks for sharing.


#7

I couldn’t find @alexis-puska 's message to the Elasticsearch team regarding the dependency problem, so reported it on their GitHub issue tracker: https://github.com/elastic/elasticsearch/issues/29184
Let’s see how it goes.
cc @gsmet