Conditional Aggregations

Hibernate Search
1

I am using a json object to craft a searchSession query on the backend it looks like this:

{
   "searchFilters":{
      "diseaseRelationFilter":{"diseaseRelation": "6"},
      "negatedFilter": {"negated":"false"}
   },
   "sortOrders": []
}

What I would like to be able to do is add aggregation fields to this object and have the backend have them in the results

   "sortOrders": [],
   "agg_field_list": ["agg_field1", "agg_field2"]

My code on the back looks like this:

SearchQuery<E> query = searchSession.search(myClass)
    .where(w -> { /* ... Stuff to handle the searchFilters ... */ })
    .sort(s -> { /* ... Stuff to handle the sortOrders ... */ })
    .toQuery();

However I can not conditionally add aggregations… I would like to be able to do something like the following:

if(agg_field_list.size() > 0) {
   for(String agg_field: agg_field_list) {
      AggregationKey<Map<String, Long>> aggKey = AggregationKey.of(agg_field);
      someObject.aggregation(aggKey, p -> p.terms().field(agg_field, String.class).maxTermCount(10);
   }
}

This should generate the following ES query:

"aggregations": {
    "agg_field1": {
      "terms": {
        "field": "agg_field1", "size": 10
      }
    },
    "agg_field2": {
      "terms": {
        "field": "agg_field2", "size": 10
      }
    }
  },

I have tried the following:

DefaultSearchQueryOptionsStep step = searchSession.search(myClass)
    .where(w -> { /* ... Stuff to handle the searchFilters ... */ })
    .sort(s -> { /* ... Stuff to handle the sortOrders ... */ });

step.aggregation(aggkey, a -> { /* ... Stuff to handle the agg_field_list */ });

However DefaultSearchQueryOptionsStep is not visible therefore can’t be used like this. Also the three methods .where() .sort() and .aggregation() are defined in the AbstractSearchQueryOptionsStep class and these, do not seem to have a way to split them up. You have to chain them together.

So the documentation offers a direct json object, for options for creating these manual:

Search DSL From Json

Example:

       .aggregation(countsByPriceHistogramKey, f -> f.fromJson( "{"
                        + "\"histogram\": {"
                                + "\"field\": \"price\","
                                + "\"interval\": 10"
                        + "}"
                + "}" ) )

However this does not work as f does not have the method fromJson so the doc’s need to be updated in this case to remove all the examples showing the fromJson:

Screen Shot 2022-02-09 at 7.21.53 AM

Has someone run into this issue… is this a feature request? Bug? Just not sure where to turn. Thanks for your help.

The full example of how I am trying to use this can be found here:

Code Example

2

The doc is right. Your code is missing this critical part:

        .extension( ElasticsearchExtension.get() )

That’s what allows you to introduce Elasticsearch-specific code (like .fromJson) to your query. Without that, adding JSON in your query just does not make sense, since your query could be targeting a non-Elasticsearch backend (e.g. embedded Lucene).

You’re not using the right type nor the right generic type arguments for the step variable.
If you’re on Java 11+, I’d recommend the var keyword in this particular case; that’ll save you some headaches:

var step = searchSession.search(myClass)
    .extension( ElasticsearchExtension.get() )
    .where(w -> { /* ... Stuff to handle the searchFilters ... */ })
    .sort(s -> { /* ... Stuff to handle the sortOrders ... */ });

List<AggregationKey<Map<String, Long>>> aggKeys = new ArrayList<>();
for(String agg_field: agg_field_list) {
    AggregationKey<Map<String, Long>> aggKey = AggregationKey.of(agg_field);
    aggKeys.add(aggKey);
    step = step.aggregation(aggKey, p -> p.terms().field(agg_field, String.class).maxTermCount(10));
}

SearchResult<MyClass> result = step.fetch(20);
// I presume you'll want to retrieve the aggregations? You can do it like this.
Map<String, Map<String, Long>> aggByField = aggKeys.stream()
        .collect(Collectors.toMap(AggregationKey::name, result::aggregation));

If you’re still on Java 8, things will get a bit more complex, but that’s still possible:

ElasticsearchSearchQueryOptionsStep<MyClass, SearchLoadingOptionsStep> step = searchSession.search(myClass)
    .extension( ElasticsearchExtension.get() )
    .where(w -> { /* ... Stuff to handle the searchFilters ... */ })
    .sort(s -> { /* ... Stuff to handle the sortOrders ... */ });

List<AggregationKey<Map<String, Long>>> aggKeys = new ArrayList<>();
for(String agg_field: agg_field_list) {
    AggregationKey<Map<String, Long>> aggKey = AggregationKey.of(agg_field);
    aggKeys.add(aggKey);
    step = step.aggregation(aggKey, p -> p.terms().field(agg_field, String.class).maxTermCount(10));
}

SearchResult<MyClass> result = step.fetch(20);
// I presume you'll want to retrieve the aggregations? You can do it like this.
Map<String, Map<String, Long>> aggByField = aggKeys.stream()
        .collect(Collectors.toMap(AggregationKey::name, result::aggregation));

Had you not been using the Elasticsearch extension, the type of “step” would have been a tad more complex:

SearchQueryOptionsStep<?, MyClass, SearchLoadingOptionsStep, ?, ?> step = searchSession.search(myClass)
    .where(w -> { /* ... Stuff to handle the searchFilters ... */ })
    .sort(s -> { /* ... Stuff to handle the sortOrders ... */ });

List<AggregationKey<Map<String, Long>>> aggKeys = new ArrayList<>();
for(String agg_field: agg_field_list) {
    AggregationKey<Map<String, Long>> aggKey = AggregationKey.of(agg_field);
    aggKeys.add(aggKey);
    step = step.aggregation(aggKey, p -> p.terms().field(agg_field, String.class).maxTermCount(10));
}

SearchResult<MyClass> result = step.fetch(20);
// I presume you'll want to retrieve the aggregations? You can do it like this.
Map<String, Map<String, Long>> aggByField = aggKeys.stream()
        .collect(Collectors.toMap(AggregationKey::name, result::aggregation));
1 Like
3

@yrodiere now I just feel silly… sorry, I guess I am an old java fart when it comes to things. I was thinking I needed to do this in a the java 8 sorta way without the extension, even though I am using java 11.

So yes the var step worked perfectly. I had almost got to the point of:

SearchQueryOptionsStep<?, E, SearchLoadingOptionsStep, ?, ?> step = ...

I was missing the SearchLoadingOptionsStep class in the definition. The var step works great… it also allows me to make the sort conditional as well.

I had not got to the part about pulling the aggs out of the result, thank you for the example.

1 Like