# Hibernate Search with special characters

**URL:** https://discourse.hibernate.org/t/hibernate-search-with-special-characters/4983
**Category:** Hibernate Search
**Created:** [January 27, 2021, 11:53am UTC](https://discourse.hibernate.org/t/hibernate-search-with-special-characters/4983 "2021-01-27T11:53:30Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![Humoyun\_Norboboev](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/humoyun_norboboev/32/1363_2.png) [@Humoyun\_Norboboev](https://discourse.hibernate.org/u/Humoyun_Norboboev)
#### Post date: [January 27, 2021, 11:53am UTC](https://discourse.hibernate.org/t/hibernate-search-with-special-characters/4983/1 "2021-01-27T11:53:30Z")

</div>

Hi. I’m using the hibernate full-text search on my application. My problem is that I could not search if the search string has some special characters. For example, the search strings are like that “042974/UzGTL” or “UzGTL-CON-1556/174278”, hibernate search returns empty response. If I look for _“UzGTL”_,  
I can get expected responses, such as

> _UzGTL-CON-1556/174278_  
> _UzGTL-CON-1465/043158_  
> _046562/UzGTL-CON-1501_  
> _2074404/UzGTL-CON-1520_  
> _042978/UzGTL-CON-1462_  
> _042976/UzGTL-CON-1459_  
> _UzGTL-CON-1510 / 2053510_  
> _169053/UzGTL-CON-1492_  
> _042974/UzGTL-CON-1457_

but if I look for _“UzGTL-”_, then it’ll return empty response.

I also tried to solve with `ASCIIFoldingFilterFactory`. But it didn’t worked. Could you help me to find what is wrong with my code? Thanks for any help.

\*\*The field I need to search is “_ **code** _”

```auto
@Indexed
@Getter
@Setter
@Entity
@DynamicInsert
@DynamicUpdate
@Table(name = "contracts")
@Audited(targetAuditMode = RelationTargetAuditMode.NOT_AUDITED, withModifiedFlag = true)
@org.hibernate.annotations.Cache(usage = CacheConcurrencyStrategy.NONSTRICT_READ_WRITE)
public class _Contract extends _Entity {

    @AnalyzerDef(name = "searchTextAnalyzer",
            tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class),
            filters = {@TokenFilterDef(factory = LowerCaseFilterFactory.class),
                       @TokenFilterDef(factory = ASCIIFoldingFilterFactory.class)})
    @Fields(value = {@Field(analyze = Analyze.YES, bridge = @FieldBridge(impl = StringBridge.class)),
            @Field(analyze = Analyze.NO, name = "codeSort", index = org.hibernate.search.annotations.Index.YES)})
    @SortableField(forField = "codeSort")
    @Column(unique = true)
    private String code;

```

**Query execution is below**

```auto
 SearchFilter searchFilter = new SearchFilter();
  if (filter.has("contractCode"))
            searchFilter.and("code", QueryParser.escape(filter.getString("contractCode")).trim() + "*");
        QueryParser parser = new QueryParser("search", searchFactory().getAnalyzer(_Contract.class));
        parser.setLowercaseExpandedTerms(true);

        org.apache.lucene.search.Query luceneQuery = parser.parse(searchFilter.toString());
        FullTextQuery fullTextQuery = fullTextSession().createFullTextQuery(luceneQuery, _Contract.class);
        fullTextQuery.initializeObjectsWith(ObjectLookupMethod.SECOND_LEVEL_CACHE, DatabaseRetrievalMethod.QUERY);
        fullTextQuery.setFirstResult(filter.getStart());
        fullTextQuery.setMaxResults(filter.getSize());

```

I use Lucene.

---

<div class="post-metadata">

### Author: ![yrodiere](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/yrodiere/32/1760_2.png) [@yrodiere](https://discourse.hibernate.org/u/yrodiere)
#### Post date: [January 27, 2021, 12:02pm UTC](https://discourse.hibernate.org/t/hibernate-search-with-special-characters/4983/2 "2021-01-27T12:02:05Z")

</div>

Hello,

> [@Humoyun\_Norboboev](#):
>
> ```auto
> @AnalyzerDef(name = "searchTextAnalyzer",
> tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class),
> filters = {@TokenFilterDef(factory = LowerCaseFilterFactory.class),
> @TokenFilterDef(factory = ASCIIFoldingFilterFactory.class)})
> @Fields(value = {@Field(analyze = Analyze.YES, bridge = @FieldBridge(impl = StringBridge.class)),
> @Field(analyze = Analyze.NO, name = "codeSort", index = org.hibernate.search.annotations.Index.YES)})
> @SortableField(forField = "codeSort")
> @Column(unique = true)
> private String code;
> 
> ```

This is not how analyzer definitions work. The analyzer definition should go on your class. Then you should use `@Field(analyzer = @Analyzer(definition = "searchTextAnalyzer"))` in your field definition to actually use the analyzer. See [this section of the documentation](https://docs.jboss.org/hibernate/search/5.11/reference/en-US/html_single/#_analysis) for more information.

Don’t forget to [reindex your data](https://docs.jboss.org/hibernate/search/5.11/reference/en-US/html_single/#search-batchindex) before you try to search again.

---

<div class="post-metadata">

### Author: ![Humoyun\_Norboboev](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/humoyun_norboboev/32/1363_2.png) [@Humoyun\_Norboboev](https://discourse.hibernate.org/u/Humoyun_Norboboev)
#### Post date: [January 27, 2021, 12:54pm UTC](https://discourse.hibernate.org/t/hibernate-search-with-special-characters/4983/3 "2021-01-27T12:54:05Z")

</div>

I did as you said. I reindexed too. But anyway it returned empty response

```auto
@Indexed
@Getter
@Setter
@Entity
@DynamicInsert
@DynamicUpdate
@Table(name = "contracts")
@Audited(targetAuditMode = RelationTargetAuditMode.NOT_AUDITED, withModifiedFlag = true)
@org.hibernate.annotations.Cache(usage = CacheConcurrencyStrategy.NONSTRICT_READ_WRITE)
@AnalyzerDef(name = "searchTextAnalyzer",
        tokenizer = @TokenizerDef(factory = StandardTokenizerFactory.class),
        filters = {@TokenFilterDef(factory = LowerCaseFilterFactory.class), @TokenFilterDef(factory = ASCIIFoldingFilterFactory.class)})
public class _Contract extends _Entity {

    @Fields(value = {@Field(analyze = Analyze.YES, index = Index.YES, bridge = @FieldBridge(impl = StringBridge.class), analyzer = @Analyzer(definition = "searchTextAnalyzer")),
            @Field(analyze = Analyze.NO, name = "codeSort", index = org.hibernate.search.annotations.Index.NO)})
    @SortableField(forField = "codeSort")
    @Column(unique = true)
    private String code;

```

---

<div class="post-metadata">

### Author: ![yrodiere](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/yrodiere/32/1760_2.png) [@yrodiere](https://discourse.hibernate.org/u/yrodiere)
#### Post date: [January 27, 2021, 1:33pm UTC](https://discourse.hibernate.org/t/hibernate-search-with-special-characters/4983/4 "2021-01-27T13:33:33Z")

</div>

What does `luceneQuery.toString()` return?

---

<div class="post-metadata">

### Author: ![Humoyun\_Norboboev](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/humoyun_norboboev/32/1363_2.png) [@Humoyun\_Norboboev](https://discourse.hibernate.org/u/Humoyun_Norboboev)
#### Post date: [January 27, 2021, 2:10pm UTC](https://discourse.hibernate.org/t/hibernate-search-with-special-characters/4983/5 "2021-01-27T14:10:57Z")

</div>

It does **“code:UzGTL\-”** , if I look for **"UzGTL-"**

---

<div class="post-metadata">

### Author: ![yrodiere](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/yrodiere/32/1760_2.png) [@yrodiere](https://discourse.hibernate.org/u/yrodiere)
#### Post date: [January 27, 2021, 2:54pm UTC](https://discourse.hibernate.org/t/hibernate-search-with-special-characters/4983/6 "2021-01-27T14:54:39Z")

</div>

It looks like your terms were not analyzed properly by your query parser. They should be lowercased, at the very least. What does `searchFilter.toString()` return?

On a side note, you could also use the [Hibernate Search DSL](https://docs.jboss.org/hibernate/search/5.11/reference/en-US/html_single/#search-query-querydsl) to build your queries.

---

<div class="post-metadata">

### Author: ![Humoyun\_Norboboev](https://yyz1.discourse-cdn.com/flex035/user_avatar/discourse.hibernate.org/humoyun_norboboev/32/1363_2.png) [@Humoyun\_Norboboev](https://discourse.hibernate.org/u/Humoyun_Norboboev)
#### Post date: [January 28, 2021, 5:05am UTC](https://discourse.hibernate.org/t/hibernate-search-with-special-characters/4983/7 "2021-01-28T05:05:31Z")

</div>

It worked. Thanks a lot 🙂
