Mass indexer : querying on a @FullTextField causes elastic search response failure

[fraf]

Hello everyone,

I did some changes on my hsearch mapping, particularly adding a lot of @IndexingDependency that were missing. Then I reindex all from zero. The mass indexer finished properly without any error.
Then I directly executed a search query using a rest client, requesting my elastic search server. And so, elastic search server answered with a failure :

{
    "took": 39,
    "timed_out": false,
    "_shards": {
        "total": 5,
        "successful": 2,
        "skipped": 0,
        "failed": 3,
        "failures": [
            {
                "shard": 0,
                "index": "depot-000001",
                "node": "pAQd-uETRCq__wKrUsT7eQ",
                "reason": {
                    "type": "illegal_argument_exception",
                    "reason": "boost must be a positive float, got -1.0"
                }
            },
            {
                "shard": 0,
                "index": "acteprive-000001",
                "node": "pAQd-uETRCq__wKrUsT7eQ",
                "reason": {
                    "type": "illegal_argument_exception",
                    "reason": "boost must be a positive float, got -1.0"
                }
            },
            {
                "shard": 0,
                "index": "mentionmicen-000001",
                "node": "pAQd-uETRCq__wKrUsT7eQ",
                "reason": {
                    "type": "illegal_argument_exception",
                    "reason": "boost must be a positive float, got -1.0"
                }
            }
        ]
    },
    "hits": {
        "total": {
            "value": 10000,
            "relation": "gte"
        },
        "max_score": 9.077475,
        "hits": []
    }
}

To cause this response failure, I have to request with one the two following indexed fields (below the whole request) :

{
	"query": {
		"bool": {
			"should": [
				{
					"bool": {
						"must": {
							"match": {
								"searchLabel": {
									"fuzziness": 2,
									"prefix_length": 0,
									"query": "FFO test 5"
								}
							}
						},
						"minimum_should_match": "0"
					}
				},
				{
					"bool": {
						"must": {
							"match": {
								"searchExtendedLabel": {
									"fuzziness": 2,
									"prefix_length": 0,
									"query": "FFO test 5"
								}
							}
						},
						"minimum_should_match": "0"
					}
				}
			]
		}
	},
	"sort": [
		{
			"dateModification": {
				"order": "desc"
			}
		}
	],
	"_source": [
		"*"
	],
	"track_scores": true,
	"highlight": {
		"require_field_match": "true",
		"fields": {
			"*": {
				"pre_tags": [
					"<mark>"
				],
				"post_tags": [
					"</mark>"
				],
				"type": "plain",
				"fragment_size": "300"
			}
		}
	}
}

As you can see, the two fields are : searchLabel and searchExtendedLabel.
If I don’t request on this two fields, the response is correct returning matching documents (if any).
So here the code for indexing these two fields :

@Override
	@Transient
	@FullTextField(name = SEARCH_EXTENDED_LABEL, analyzer = EDGE_NGRAM_2_15, searchAnalyzer = CUSTOM, searchable = Searchable.YES)
	@KeywordField(name = SEARCH_EXTENDED_LABEL_SORT, sortable = Sortable.YES, aggregable = Aggregable.YES)
	@IndexingDependency(
			derivedFrom = {
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DEMANDES_COMPTABLES),
							@PropertyValue(propertyName = DemandeComptable_.IGNORER)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.ETAT_DEPOT)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.VALIDE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.CODE_WORKFLOW)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DEMANDES_COMPTABLES),
							@PropertyValue(propertyName = DemandeComptable_.ETAT_DEMANDE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DEPOTS_COMPLEMENTAIRES),
							@PropertyValue(propertyName = Depot_.DEPOT_INITIAL)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DEPOTS_COMPLEMENTAIRES),
							@PropertyValue(propertyName = Depot_.REGULARISE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DEPOTS_COMPLEMENTAIRES),
							@PropertyValue(propertyName = Depot_.ETAT_DEPOT)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DEPOTS_COMPLEMENTAIRES),
							@PropertyValue(propertyName = Depot_.CODE_WORKFLOW)
					})
			}
	)
	public String getSearchExtendedLabel() {
//some computation
}

@Override
	@Transient
	@FullTextField(name = SEARCH_LABEL, analyzer = EDGE_NGRAM_2_15, searchAnalyzer = CUSTOM, searchable = Searchable.YES, projectable = Projectable.YES)
	@KeywordField(name = SEARCH_LABEL_SORT, sortable = Sortable.YES)
	@IndexingDependency(
			derivedFrom = {
					/* Teledepot, Inconnu */
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.ACTE),
							@PropertyValue(propertyName = Acte_.TYPE_ACTE),
							@PropertyValue(propertyName = TypeActe_.LIBELLE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.ACTE),
							@PropertyValue(propertyName = Acte_.LIBELLE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.ACTE),
							@PropertyValue(propertyName = Acte_.DOSSIER),
							@PropertyValue(propertyName = Dossier_.NUMERO)
					}),
					/* Telerequisition */
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.REQUISITIONS),
							@PropertyValue(propertyName = Requisition_.REQUISITION_PROROGEE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.REQUISITIONS),
							@PropertyValue(propertyName = Requisition_.ANF)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.REQUISITIONS),
							@PropertyValue(propertyName = Requisition_.HORS_DEMANDE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.REQUISITIONS),
							@PropertyValue(propertyName = Requisition_.HORS_FORMALITE)
					}),
					/* Demande de copie */
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DEMANDES_COPIES),
							@PropertyValue(propertyName = DemandeCopie_.NATURE_DEMANDE_COPIE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DEMANDES_COPIES),
							@PropertyValue(propertyName = DemandeCopie_.AFFAIRE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DEMANDES_COPIES),
							@PropertyValue(propertyName = DemandeCopie_.DOSSIER),
							@PropertyValue(propertyName = Dossier_.NUMERO)
					}),
					/* Biv */
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.VENTE),
							@PropertyValue(propertyName = Vente_.ACTE),
							@PropertyValue(propertyName = Acte_.TYPE_ACTE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.VENTE),
							@PropertyValue(propertyName = Vente_.ACTE),
							@PropertyValue(propertyName = Acte_.LIBELLE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.VENTE),
							@PropertyValue(propertyName = Vente_.ACTE),
							@PropertyValue(propertyName = Acte_.DOSSIER),
							@PropertyValue(propertyName = Dossier_.NUMERO)
					}),
					/* Biac */
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DAC),
							@PropertyValue(propertyName = Dac_.ACTE),
							@PropertyValue(propertyName = Acte_.TYPE_ACTE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DAC),
							@PropertyValue(propertyName = Dac_.ACTE),
							@PropertyValue(propertyName = Acte_.LIBELLE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DAC),
							@PropertyValue(propertyName = Dac_.DAC_PIECE),
							@PropertyValue(propertyName = DacPiece_.DOSSIER),
							@PropertyValue(propertyName = Dossier_.NUMERO)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DAC),
							@PropertyValue(propertyName = Dac_.DAC_SOUS_PRODUIT),
							@PropertyValue(propertyName = DacSousProduit_.ACTE),
							@PropertyValue(propertyName = Acte_.DOSSIER),
							@PropertyValue(propertyName = Dossier_.NUMERO)
					}),
					/* Cjn */
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DEMANDE_CJN),
							@PropertyValue(propertyName = DemandeCjn_.DOSSIER),
							@PropertyValue(propertyName = Dossier_.TYPE),
							@PropertyValue(propertyName = TypeDossier_.GENRE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DEMANDE_CJN),
							@PropertyValue(propertyName = DemandeCjn_.DOSSIER),
							@PropertyValue(propertyName = Dossier_.LIBELLE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DEMANDE_CJN),
							@PropertyValue(propertyName = DemandeCjn_.DOSSIER),
							@PropertyValue(propertyName = Dossier_.NUMERO)
					}),
					/* COMEDEC */
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.COURRIER),
							@PropertyValue(propertyName = Courrier_.DOSSIER),
							@PropertyValue(propertyName = Dossier_.TYPE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.COURRIER),
							@PropertyValue(propertyName = Courrier_.DOSSIER),
							@PropertyValue(propertyName = Dossier_.LIBELLE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.COURRIER),
							@PropertyValue(propertyName = Courrier_.DOSSIER),
							@PropertyValue(propertyName = Dossier_.NUMERO)
					}),
					/* Demande Fcddv */
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.ETAT_DEPOT)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.LIBELLE_DISPOSANT)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DEMANDE_FCDDV),
							@PropertyValue(propertyName = DemandeFcddv_.DOSSIER),
							@PropertyValue(propertyName = Dossier_.LIBELLE)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DEMANDE_FCDDV),
							@PropertyValue(propertyName = DemandeFcddv_.DOSSIER),
							@PropertyValue(propertyName = Dossier_.NUMERO)
					}),
					@ObjectPath({
							@PropertyValue(propertyName = Depot_.DEMANDE_FCDDV),
							@PropertyValue(propertyName = DemandeFcddv_.DOSSIER),
							@PropertyValue(propertyName = Dossier_.TYPE),
							@PropertyValue(propertyName = TypeDossier_.GENRE)
					})

			}
	)
	public String getSearchLabel() {
//some computation
}

These two methods implement an interface. These mappings where always correct, I lastly just added @ObjectPath on @IndexingDependency. And now, when requesting on one of these fields I always have no result because of this search failure.
Why boost is negative then ? (I never specify this).

Thanks.

[fraf]

Ok I found the error, it is a side effect coming from the query :

	"pre_tags": [
					"<mark>"
				],
				"post_tags": [
					"</mark>"
				],
				"type": "plain",
				"fragment_size": "300"

Problem with syntax surely.

Ticket can be closed.

Thanks.

[yrodiere]

This is definitely very odd.

First, let’s make something clear: @IndexingDependency is purely about metadata for Hibernate Search itself, and only during automatic indexing. It doesn’t impact the Elasticsearch schema, it doesn’t impact the document content, it doesn’t impact search, and it doesn’t impact mass indexing.

So I doubt this new error is related to your changes to your @IndexingDependency.

I think it’s more likely related to highlighting or the fuzziness; did you add that recently too, by any chance? Or it could be related to the content of your index, which changed when you reindexed; the exact behavior of a search query can change depending on the content of your index, especially with highlighting.

I’m accusing highligting and fuziness because a quick web search for your error message gave me several hits related to highlighting and fuzzy:

• java - How to highlight Boolean FuzzyQueries in Lucene - boost must be a positive float? - Stack Overflow
• [LUCENE-9568] FuzzyTermEnums sets negative boost for fuzzy search & highlight - ASF JIRA

See "boost must be a positive float, got -1.0" at DuckDuckGo for all hits.

This really looks like a bug in Lucene (which Elasticsearch uses under the hood). I’d suggest going through the links above to find if there’s a workaround. You could also try reporting the problem to the Elasticsearch team; if you provide a reproducer and a link to the Lucene bug, they may be able to debug and maybe even give you a workaround.

I think giving up on the “fuzzy” feature could be a solution. You can get results similar to fuzzy queries by using dedicated analyzers, for example with the ngram token filter. Maybe that will work better when highlighting?

---

Also, not related to your question, but given the size of your @IndexingDependency, I wonder if maybe you’re not making your life more difficult than it needs to be by using a single searchLabel that (I suppose) concatenates multiple strings from multiple places. Did you know you can target multiple fields in a single match predicate? You don’t have to concatenate everything at indexing time: you can just populate multiple fields, and query all these fields at once when searching.

[fraf]

@fraf

"highlight": {
		"require_field_match": "true",
		"fields": {
			"*": {
				"pre_tags": [
					"<mark>"
				],
				"post_tags": [
					"</mark>"
				],
				"type": "plain",
				"fragment_size": "300"
			}
		}
	}

Thanks for your answers, as I said, it was a side effect, a really whatever failure message was returned. The highlight part should be :

"highlight": {
		"require_field_match": "true",
		"fields": {
			"_all": {
				"pre_tags": [
					"<mark>"
				],
				"post_tags": [
					"</mark>"
				],
				"type": "plain",
				"fragment_size": "300"
			}
		}
	}

The difference is requesting on “_all” field instead of “*”. A pure Elastic search syntax problem.

Thanks anyway.