# Index validation failure directly after MassIndexing

**URL:** <https://discourse.hibernate.org/t/index-validation-failure-directly-after-massindexing/6518>\
**Category:** Hibernate Search\
**Created:** [July 25, 2022, 11:52am UTC](https://discourse.hibernate.org/t/index-validation-failure-directly-after-massindexing/6518 "2022-07-25T11:52:31Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![fraf](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/fraf/32/1614_2.png) [@fraf](https://discourse.hibernate.org/u/fraf)\
**Post date:** [July 25, 2022, 11:52am UTC](https://discourse.hibernate.org/t/index-validation-failure-directly-after-massindexing/6518/1 "2022-07-25T11:52:32Z")

</div>

Following a successful mass indexing process, when starting the application, I have a validation failure on the only one index :

```auto
Caused by: org.hibernate.search.util.common.SearchException: HSEARCH000520: Hibernate Search encountered failures during bootstrap. Failures:

    Hibernate ORM mapping: 
        type 'com.allegoria.notariat.business.ActeParticipation': 
            failures: 
              - Validation of the existing index in the Elasticsearch cluster failed. See below for details.
            field 'references': 
                attribute 'search_analyzer': 
                    failures: 
                      - Invalid value. Expected 'standard_min3_search_analyzer', actual is 'null'
            field 'spf': 
                attribute 'search_analyzer': 
                    failures: 
                      - Invalid value. Expected 'standard_min3_search_analyzer', actual is 'null'
	at org.hibernate.search.engine.reporting.spi.RootFailureCollector.checkNoFailure(RootFailureCollector.java:50)
	at org.hibernate.search.engine.common.impl.SearchIntegrationPartialBuildStateImpl$SearchIntegrationFinalizerImpl.finalizeIntegration(SearchIntegrationPartialBuildStateImpl.java:187)
	at org.hibernate.search.mapper.orm.bootstrap.impl.HibernateOrmIntegrationPartialBuildState.doBootSecondPhase(HibernateOrmIntegrationPartialBuildState.java:75)
	at org.hibernate.search.mapper.orm.bootstrap.impl.HibernateOrmIntegrationBooterImpl.bootNow(HibernateOrmIntegrationBooterImpl.java:176)
	at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:642)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
	at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
	at org.hibernate.search.mapper.orm.bootstrap.impl.HibernateSearchSessionFactoryObserver.sessionFactoryCreated(HibernateSearchSessionFactoryObserver.java:41)
	at org.hibernate.internal.SessionFactoryObserverChain.sessionFactoryCreated(SessionFactoryObserverChain.java:35)
	at org.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:394)
	at org.hibernate.boot.internal.SessionFactoryBuilderImpl.build(SessionFactoryBuilderImpl.java:471)
	at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:728)
	at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:746)
	at org.springframework.orm.hibernate5.LocalSessionFactoryBean.buildSessionFactory(LocalSessionFactoryBean.java:615)

```

And the corresponding mapping of ActeParticipation :

```auto
public void configureReferences(TypeMappingStep typeMapping, PojoModelPathValueNode... derivedFrom) {
        PropertyMappingStep propertyMappingStep = typeMapping.property(PROPERTY_PARAPHEUR_REFERENCES);

        fullTextFieldSTANDARD(propertyMappingStep, SearchableParapheurIndexed.REFERENCES);
        keywordField(propertyMappingStep, SearchableParapheurIndexed.REFERENCES_SORT, Aggregable.NO);

        propertyMappingStep
                .indexingDependency()
                .reindexOnUpdate(ReindexOnUpdate.NO);
    }

```

```auto
public static void fullTextFieldSTANDARD(PropertyMappingStep propertyMappingStep, String fieldName) {
        propertyMappingStep
                .fullTextField(fieldName)
                .analyzer(STANDARD_MIN_3)
                .searchAnalyzer(STANDARD_MIN_3)
                .searchable(Searchable.YES)
                .projectable(Projectable.YES);
    }

```

According to the documentation :

> searchAnalyzer  
> An optional different analyzer, overriding the one defined with the analyzer attribute, to use only  
> when analyzing searched terms.  
> If not defined, the analyzer assigned to analyzer will be used.

So I removed the mapping :

`.searchAnalyzer(STANDARD_MIN_3) //turn into comment`

And it starts, the validation was successful. I think, there is a little incoherence between the massindexer and the process of the validation.

The massindexer didn’t push the searchAnalyser because it is the same (contrary to other fields) :

```auto
GET /acteparticipation-read/_mapping
          "references": {
                    "type": "text",
                    "analyzer": "standard_min3_search_analyzer"
                },
              "referencesSort": {
                  "type": "keyword"
              },
              "searchExtendedLabel": {
                  "type": "text",
                  "analyzer": "parapheur_shingle",
                  "search_analyzer": "exact_match_search_analyzer"
              },

```

---

<div class="post-metadata">

**Author:** ![yrodiere](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/yrodiere/32/1760_2.png) [@yrodiere](https://discourse.hibernate.org/u/yrodiere)\
**Post date:** [July 25, 2022, 1:41pm UTC](https://discourse.hibernate.org/t/index-validation-failure-directly-after-massindexing/6518/2 "2022-07-25T13:41:47Z")

</div>

Thanks for reporting. I reproduced this locally, it is a bug: [[HSEARCH-4652] - Hibernate JIRA](https://hibernate.atlassian.net/browse/HSEARCH-4652)

---

<div class="post-metadata">

**Author:** ![fraf](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/fraf/32/1614_2.png) [@fraf](https://discourse.hibernate.org/u/fraf)\
**Post date:** [July 25, 2022, 2:08pm UTC](https://discourse.hibernate.org/t/index-validation-failure-directly-after-massindexing/6518/3 "2022-07-25T14:08:17Z")

</div>

Good. Same bug, but another test case :

```auto
Caused by: org.hibernate.search.util.common.SearchException: HSEARCH000520: Hibernate Search encountered failures during bootstrap. Failures:

    Hibernate ORM mapping: 
        type 'com.allegoria.notariat.business.ActeDocument': 
            failures: 
              - Validation of the existing index in the Elasticsearch cluster failed. See below for details.
            field 'depots.statutSuggest': 
                attribute 'analyzer': 
                    failures: 
                      - Invalid value. Expected 'default', actual is 'simple'
        type 'com.allegoria.notariat.business.ActePrive': 
            failures: 
              - Validation of the existing index in the Elasticsearch cluster failed. See below for details.
            field 'depots.statutSuggest': 
                attribute 'analyzer': 
                    failures: 
                      - Invalid value. Expected 'default', actual is 'simple'
        type 'com.allegoria.notariat.business.Depot': 
            failures: 
              - Validation of the existing index in the Elasticsearch cluster failed. See below for details.
            field 'statutSuggest': 
                attribute 'analyzer': 
                    failures: 
                      - Invalid value. Expected 'default', actual is 'simple'
	at org.hibernate.search.engine.reporting.spi.RootFailureCollector.checkNoFailure(RootFailureCollector.java:50)
	at org.hibernate.search.engine.common.impl.SearchIntegrationPartialBuildStateImpl$SearchIntegrationFinalizerImpl.finalizeIntegration(SearchIntegrationPartialBuildStateImpl.java:187)
	at org.hibernate.search.mapper.orm.bootstrap.impl.HibernateOrmIntegrationPartialBuildState.doBootSecondPhase(HibernateOrmIntegrationPartialBuildState.java:75)
	at org.hibernate.search.mapper.orm.bootstrap.impl.HibernateOrmIntegrationBooterImpl.bootNow(HibernateOrmIntegrationBooterImpl.java:176)
	at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:642)
	at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506)
	at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073)
	at org.hibernate.search.mapper.orm.bootstrap.impl.HibernateSearchSessionFactoryObserver.sessionFactoryCreated(HibernateSearchSessionFactoryObserver.java:41)
	at org.hibernate.internal.SessionFactoryObserverChain.sessionFactoryCreated(SessionFactoryObserverChain.java:35)
	at org.hibernate.internal.SessionFactoryImpl.<init>(SessionFactoryImpl.java:394)
	at org.hibernate.boot.internal.SessionFactoryBuilderImpl.build(SessionFactoryBuilderImpl.java:471)
	at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:728)
	at org.hibernate.cfg.Configuration.buildSessionFactory(Configuration.java:746)
	at org.springframework.orm.hibernate5.LocalSessionFactoryBean.buildSessionFactory(LocalSessionFactoryBean.java:615)
	at org.springframework.orm.hibernate5.LocalSessionFactoryBean.afterPropertiesSet(LocalSessionFactoryBean.java:599)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1821)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1758)
	... 112 common frames omitted

```

And the corresponding mapping of Depot :

```auto
private void configureStatut(TypeMappingStep depotMapping) {
        PropertyMappingStep statut = depotMapping
                .property(PROPERTY_DEPOT_STATUT);
        statut
                .genericField(STATUT)
                .sortable(Sortable.YES)
                .aggregable(Aggregable.YES);
        statut
                .nonStandardField(STATUT_SUGGEST)
                .valueBinder(completionBinder);
        indexingDependency(statut, .....);
}

```

The problem is on the `completionBinder` :

```auto
@Component
public class CompletionBinder implements ValueBinder {

	@Override
	public void bind(ValueBindingContext<?> context) {
		context.bridge(
				String.class,
				new CompletionBridge(),
				context.typeFactory()
						.extension(ElasticsearchExtension.get())
						.asNative()
						.mapping("{\"type\": \"completion\"}"));
	}

	private static class CompletionBridge implements ValueBridge<String, JsonElement> {
		@Override
		public JsonElement toIndexedValue(String value, ValueBridgeToIndexedValueContext context) {
			return value == null ? null : new JsonPrimitive(value);
		}

		@Override
		public String fromIndexedValue(JsonElement value, ValueBridgeFromIndexedValueContext context) {
			return value == null ? null : value.getAsString();
		}
	}
}

```

I have to mention explicity an analyser to pass the validation error at startup :

`.mapping("{\"type\": \"completion\", \"analyzer\": \"" + EDGE_NGRAM_1_15 + "\"}"));`

Then massIndexing followed by validation is working.

---

<div class="post-metadata">

**Author:** ![yrodiere](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/yrodiere/32/1760_2.png) [@yrodiere](https://discourse.hibernate.org/u/yrodiere)\
**Post date:** [July 25, 2022, 2:41pm UTC](https://discourse.hibernate.org/t/index-validation-failure-directly-after-massindexing/6518/4 "2022-07-25T14:41:44Z")

</div>

This is a quirk of Elasticsearch that is specific to the `completion` type. Its default analyzer is `simple`, while elsewhere it’s `default`: [Suggesters | Elasticsearch Guide [8.3] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/8.3/search-suggesters.html#completion-suggester-mapping), [Text type family | Elasticsearch Guide [8.3] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/8.3/text.html#text-params)

Since the `completion` type is not officially supported (you had to go through native APIs after all), and you have a workaround, I don’t think it would make sense to spend time fixing that.

---

<div class="post-metadata">

**Author:** ![fraf](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/fraf/32/1614_2.png) [@fraf](https://discourse.hibernate.org/u/fraf)\
**Post date:** [July 25, 2022, 3:00pm UTC](https://discourse.hibernate.org/t/index-validation-failure-directly-after-massindexing/6518/5 "2022-07-25T15:00:18Z")

</div>

That’s ok. I read here : [[Specify an analyzer | Elasticsearch Guide [8.3] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/specify-analyzer.html#:~:text=analyzer.-,default%20setting.,my%2Dindex%2D000001%20index.)]  
That we can specify a default analyser if mapping don’t provide it explicitly :

```auto
PUT my-index-000001
{
  "settings": {
    "analysis": {
      "analyzer": {
        "default": {
          "type": "simple"
        }
      }
    }
  }
}

```

Then during validation, we should retrieved the default one (ie “simple”). Should it be configurable through hibernate search ? (I don’t know).

---

<div class="post-metadata">

**Author:** ![yrodiere](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/yrodiere/32/1760_2.png) [@yrodiere](https://discourse.hibernate.org/u/yrodiere)\
**Post date:** [July 26, 2022, 6:43am UTC](https://discourse.hibernate.org/t/index-validation-failure-directly-after-massindexing/6518/6 "2022-07-26T06:43:25Z")

</div>

> [@fraf](#):
>
> Then during validation, we should retrieved the default one (ie “simple”).

I don’t think so. Especially since the `completion` type doesn’t use the `default` analyzer, but the `simple` analyzer. You can try, though.

> [@fraf](#):
>
> Should it be configurable through hibernate search ? (I don’t know).

It is configurable: you can just use an [analysis configurer](https://docs.jboss.org/hibernate/stable/search/reference/en-US/html_single/#backend-elasticsearch-analysis-analyzers) to configure the analyzer named `default`, or put the JSON you gave as an example in the [custom index settings](https://docs.jboss.org/hibernate/stable/search/reference/en-US/html_single/#_custom_index_settings)

But as I mentioned above, I don’t think this will solve your problem with the `completion` type. Only setting the analyzer explicitly (even if you just set it to `simple`) will solve your problem.

---

<div class="post-metadata">

**Author:** ![yrodiere](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/yrodiere/32/1760_2.png) [@yrodiere](https://discourse.hibernate.org/u/yrodiere)\
**Post date:** [August 4, 2022, 1:21pm UTC](https://discourse.hibernate.org/t/index-validation-failure-directly-after-massindexing/6518/7 "2022-08-04T13:21:49Z")

</div>

> [@fraf](#):
>
> Following a successful mass indexing process, when starting the application, I have a validation failure on the only one index :
> 
> ```auto
> Caused by: org.hibernate.search.util.common.SearchException: HSEARCH000520: Hibernate Search encountered failures during bootstrap. Failures:
> 
> Hibernate ORM mapping: 
> type 'com.allegoria.notariat.business.ActeParticipation': 
> failures: 
> - Validation of the existing index in the Elasticsearch cluster failed. See below for details.
> field 'references': 
> attribute 'search_analyzer': 
> failures: 
> - Invalid value. Expected 'standard_min3_search_analyzer', actual is 'null'
> field 'spf': 
> attribute 'search_analyzer': 
> failures: 
> - Invalid value. Expected 'standard_min3_search_analyzer', actual is 'null'
> 
> ```

> [@yrodiere](#):
>
> Thanks for reporting. I reproduced this locally, it is a bug: [[HSEARCH-4652] - Hibernate JIRA ](https://hibernate.atlassian.net/browse/HSEARCH-4652)

The bug was fixed in Hibernate Search 6.1.6.Final, released moments ago: [Hibernate Search 6.1.6.Final released - In Relation To](https://in.relation.to/2022/08/04/hibernate-search-6-1-6-Final/)