Searching for Long numbers in @GenericFiled

Hibernate Search
1

I migrated from older releases to Hibernate Search 6.0.5.Final and Lucene 8.7.0 (from few reasons I cannot use later versions just now).
I need to have indexed numeric fields for which I was using NumericField in HS 5.x which I migrated to @GenericField in HS 6.x. The problem is I don’t know how get hits queried by generic fields mapped for Long field in the entity.

These fields are successfully created in the index, if I configure them as projectable their values are stored.
In Luke they are shown like this:
num -------S-#i64--------T8/1 0 1

Not only from Java but even from Luke I’m not able to create a search query which returns hits back. The list is always empty.

Am I missing or misunderstanding anything important?

2

You’re running into trouble. Hibernate Search is not in any way designed to work with another version of Lucene than the one it depends on. You’ve been warned. => That’s wrong, I misunderstood this sentence. See below.

Which mapping, which query? Please show code samples of both your entity code with @GenericField annotations, and your search code that builds the query.

3

From here
6.0 series - Hibernate Search
I understand my versions should be ok. Or not?

4

Right, sorry about that.

I read your message as “I’m using Lucene 8.7 because I can’t use the later one that Hibernate Search 6.0 depends on”, and I didn’t check what was the version that Hibernate Search 6.0 depends on exactly.

If you’re using an older version of Hibernate Search (6.0) but you make sure to use the right Lucene version (8.7), then you’ll be fine.

5

Mapping (Entity class is is annotatedas @Indexed):

@GenericField(searchable = Searchable.YES, projectable = Projectable.YES)
private Long latestSnapshotId;

+ public setter and getter

After index files are created from scratch and fields are indexed, latestSnapshotId is present in the index and it has even values saved (values are 1 and 2, I can see them in DB). From Luke there are no tokens indexed for this field, but I don’t know if it is relevant for numeric fields.
In any case, a simple query
latestSnapshot:[1 TO 2] (or [1 TO 1])
returns no hits.
It looks to me like the field was correctly registered in the index, but no terms/tokens were indexed for it.
See the screenshots (I have to upload them one by one)
The error message from the bottom of the third screenshot says:
java.lang.IllegalArgumentException: field="latestSnapshotId" was indexed with bytesPerDim=8 but this query has bytesPerDim=4
Maybe it is only because there is not term available for the field? Don’t know, haven’t found anything useful about it.

image
image862×640 32.4 KB

6

Screenshot 2

Luke-2
Luke-2954×687 31.5 KB

7

Screenshot 3

Luke-3
Luke-31020×687 36.6 KB

8

Luke very likely creates string queries in this case, which definitely won’t work on a numeric field.

Please provide the code of your mapping (entity with annotations) and search (query creation).

9

Sorry, I’m not familiar with tool and it seems after re-editing previous message it disappeared. Will post it again.

10

Ah, you did, but your comment was detected as spam for some reason. Solving that…

11

Ok, so you only provided info on how you query from Luke. My previous answer covers Luke queries:

As to why queries you create in Hibernate Search don’t work… I’ll need you to show me how you create them.

12

The real query is a bit more complex, but as a filter I’m using sort of this

        BooleanQuery.Builder booleanQueryBuilder = new BooleanQuery.Builder();
        booleanQueryBuilder.add(LongPoint.newExactQuery("latestSnapshotId", 1L, BooleanClause.Occur.SHOULD);
        }
        BooleanQuery query = booleanQueryBuilder.build();

Which is then used as one of clauses in the final query as a filter.
If I disable this “filter” query I get all hits which I’d expect (i.e., not filtered out).

Maybe you can see something wrong in it. If not I can try to simplify it and send an exact example.

13

BTW: I used Luke only to verify things, because I was not sure if I did not do anything wrong in the configuration. I’m useing a programatic configuration, but for the field described above only a simple annotation @GenericField is used.

14

The snippet you gave seems fine, though I would recommend that you use the Hibernate Search DSL instead: it’s easier to get right than the very powerful, but complex Lucene APIs.

Ok, it seems there’s much more complexity than what you’re showing here. I think if I try to poke around randomly to find where the problem is, we’ll both be very frustrated.

Maybe it’s best if you can create a small reproducer (for example using this test case template), publish it somewhere and then I’ll have a look?

15

Ok, I think a bit more faster might be if I simulate the issue in my current production code by proceeding a simple query for the single field and show you how the query looks like in the debugger before it is sent to Lucene.
But I look at the test case template as well.
It will take some time.
.

16

During an attempt for a simplification of the code for demonstration I found where the core problem was.
In an another filter subquery a TermQuery was used for searching numbers which apparently could not succeed (at least with HS 6 and Lucene 8). After replacement of the TermQuery by LongPoint.newExactQuery I got the expected result.

Anyway, thanks for your help. Your suspicion that Luke might not do what one could expect and confirmation that ExactQuery is a compatible query for @GenericField saved me a lot of time, because being confused by Luke I thought the problem should have been more serious.

BTW: Yes, I agree using DSL would be more effective and straightforward, but a usage of Lucene queries is a part of the more complex code structures and we don’t have a time for their general replacement just now, maybe later.

1 Like