"This iterator can only move forward" issue

Dmytro_Vasyliev · October 17, 2024, 12:28pm

Hello folks!
I’ll try to post one more time. Cause my first post is stuck at pre-moderation bot.
I have a strange situation while sorting by multiple fields.

My model classes structure is following:

public class SourceModel {

    //other fields

    @FullTextField(name = "title", analyzer = "stop")
    @KeywordField(name = "titleSort", sortable = Sortable.YES, normalizer = "lowercase")
    @Column(name = "title")
    private String title;

    @IndexedEmbedded(structure = ObjectStructure.NESTED)
    @IndexingDependency(reindexOnUpdate = ReindexOnUpdate.SHALLOW)
    @OneToMany(cascade = CascadeType.ALL, orphanRemoval = true)
    @JoinColumn(name = "source_id")
    @ToString.Exclude
    private List<CalculationModel> calculations;

    @IndexedEmbedded(structure = ObjectStructure.NESTED)
    @IndexingDependency(reindexOnUpdate = ReindexOnUpdate.SHALLOW)
    @OneToMany(cascade = CascadeType.ALL, orphanRemoval = true)
    @JoinColumn(name = "source_id")
    @ToString.Exclude
    private List<SourceMetricModel> sourceMetrics;

}

public class CalculationModel {

    //other fields

    @GenericField(sortable = Sortable.YES, indexNullAs = "0")
    @Column(name = "citescore")
    private Float citeScore;

    //one more nested level
    @IndexedEmbedded(structure = ObjectStructure.NESTED)
    @IndexingDependency(reindexOnUpdate = ReindexOnUpdate.SHALLOW)
    @OneToMany(cascade = CascadeType.ALL, orphanRemoval = true)
    @JoinColumn(name = "calculation_id")
    @ToString.Exclude
    private List<SubjectCategoryModel> subjectCategories;

}

public class SourceMetricModel {

    //other fields

    @GenericField(name = "sjr", sortable = Sortable.YES, indexNullAs = "0")
    @Column(name = "sjr_value")
    private Float sjrValue;
}

So…
When I do sorting (for any search criteria) like this, everything works fine

sort -> sort.field("calculations.citeScore").asc()
                .filter(f -> f.match().field("calculations.year").matching(year))
                .then().field("titleSort").asc();

But…
When I sort like this…

sort -> sort.field("sourceMetrics.sjr").asc()
                .filter(f -> f.match().field("sourceMetrics.year").matching(year))
                .then().field("titleSort").asc();

… I end up with following exceprion:

This iterator can only move forward (no advancing to the same doc twice, no going backward) -- this may indicate a bug or a missing test in Hibernate Search.

Full stacktrace:

org.hibernate.search.util.common.AssertionFailure: This iterator can only move forward (no advancing to the same doc twice, no going backward) -- this may indicate a bug or a missing test in Hibernate Search. Please report it: https://hibernate.org/community/
	at org.hibernate.search.backend.lucene.lowlevel.join.impl.ChildDocIds.advanceExactParent(ChildDocIds.java:38)
	at org.hibernate.search.backend.lucene.lowlevel.docvalues.impl.DoubleMultiValuesToSingleValuesSource$2.advanceExact(DoubleMultiValuesToSingleValuesSource.java:152)
	at org.hibernate.search.backend.lucene.lowlevel.docvalues.impl.NumericDoubleValues$RawNumericDocValues.advanceExact(NumericDoubleValues.java:97)
	at org.apache.lucene.search.comparators.FloatComparator$FloatLeafComparator.getValueForDoc(FloatComparator.java:70)
	at org.apache.lucene.search.comparators.FloatComparator$FloatLeafComparator.copy(FloatComparator.java:95)
	at org.apache.lucene.search.MultiLeafFieldComparator.copy(MultiLeafFieldComparator.java:84)
	at org.apache.lucene.search.TopFieldCollector$TopFieldLeafCollector.collectCompetitiveHit(TopFieldCollector.java:114)
	at org.apache.lucene.search.TopFieldCollector$SimpleFieldCollector$1.collect(TopFieldCollector.java:206)
	at org.apache.lucene.search.ConjunctionBulkScorer.score(ConjunctionBulkScorer.java:171)
	at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:38)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:751)
	at org.apache.lucene.search.IndexSearcher.lambda$search$2(IndexSearcher.java:703)
	at org.apache.lucene.search.TaskExecutor$TaskGroup.lambda$createTask$0(TaskExecutor.java:117)
	at java.base/java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:317)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java)
	at org.apache.lucene.search.TaskExecutor$TaskGroup.invokeAll(TaskExecutor.java:152)
	at org.apache.lucene.search.TaskExecutor.invokeAll(TaskExecutor.java:76)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:707)
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:674)
	at org.hibernate.search.backend.lucene.search.extraction.impl.LuceneCollectors.collectMatchingDocs(LuceneCollectors.java:91)
	at org.hibernate.search.backend.lucene.search.query.impl.LuceneSearcherImpl.collectMatchingDocs(LuceneSearcherImpl.java:141)
	at org.hibernate.search.backend.lucene.search.query.impl.LuceneSearcherImpl.doSearch(LuceneSearcherImpl.java:96)
	at org.hibernate.search.backend.lucene.search.query.impl.LuceneSearcherImpl.search(LuceneSearcherImpl.java:73)
	at org.hibernate.search.backend.lucene.search.query.impl.LuceneSearcherImpl.search(LuceneSearcherImpl.java:30)
	at org.hibernate.search.backend.lucene.work.impl.SearchWork.execute(SearchWork.java:39)
	at org.hibernate.search.backend.lucene.orchestration.impl.LuceneSyncWorkOrchestratorImpl$WorkExecution.execute(LuceneSyncWorkOrchestratorImpl.java:154)
	at org.hibernate.search.backend.lucene.orchestration.impl.LuceneSyncWorkOrchestratorImpl.doSubmit(LuceneSyncWorkOrchestratorImpl.java:88)
	at org.hibernate.search.backend.lucene.orchestration.impl.LuceneSyncWorkOrchestratorImpl.doSubmit(LuceneSyncWorkOrchestratorImpl.java:31)
	at org.hibernate.search.engine.backend.orchestration.spi.AbstractWorkOrchestrator.submit(AbstractWorkOrchestrator.java:137)
	at org.hibernate.search.backend.lucene.orchestration.impl.LuceneSyncWorkOrchestratorImpl.submit(LuceneSyncWorkOrchestratorImpl.java:57)
	at org.hibernate.search.backend.lucene.orchestration.impl.LuceneSyncWorkOrchestrator.submit(LuceneSyncWorkOrchestrator.java:25)
	at org.hibernate.search.backend.lucene.search.query.impl.LuceneSearchQueryImpl.doSubmit(LuceneSearchQueryImpl.java:200)
	at org.hibernate.search.backend.lucene.search.query.impl.LuceneSearchQueryImpl.doFetch(LuceneSearchQueryImpl.java:175)
	at org.hibernate.search.backend.lucene.search.query.impl.LuceneSearchQueryImpl.fetch(LuceneSearchQueryImpl.java:97)
	at org.hibernate.search.backend.lucene.search.query.impl.LuceneSearchQueryImpl.fetch(LuceneSearchQueryImpl.java:38)
	at org.hibernate.search.engine.search.query.dsl.spi.AbstractSearchQueryOptionsStep.fetch(AbstractSearchQueryOptionsStep.java:207)
	at com.elsevier.kd.repository.fts.YearlyMetricsFtsRepository.getYearlyMetrics(YearlyMetricsFtsRepository.java:23)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:355)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:768)
	at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:138)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184)
	at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:768)
	at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:720)
	at com.elsevier.kd.repository.fts.YearlyMetricsFtsRepository$$SpringCGLIB$$0.getYearlyMetrics(<generated>)
	at com.elsevier.kd.service.YearlyMetricsService.getYearlyMetrics(YearlyMetricsService.java:26)
	at com.elsevier.kd.controller.YearlyMetricsController.search(YearlyMetricsController.java:22)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:255)
	at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:188)
	at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:118)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:926)
	at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:831)
	at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:87)
	at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:1089)
	at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:979)
	at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1014)
	at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:914)
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:590)
	at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:885)
	at jakarta.servlet.http.HttpServlet.service(HttpServlet.java:658)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:195)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:51)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.AbstractRequestLoggingFilter.doFilterInternal(AbstractRequestLoggingFilter.java:289)
	at com.elsevier.kd.appweb.filter.TraceFilter.doFilterInternal(TraceFilter.java:61)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at com.elsevier.kd.appweb.filter.TrailingSlashRedirectFilter.doFilter(TrailingSlashRedirectFilter.java:30)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at com.elsevier.kd.appweb.filter.logging.simple.SimpleLoggingFilter.doFilterInternal(SimpleLoggingFilter.java:46)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.RequestContextFilter.doFilterInternal(RequestContextFilter.java:100)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.FormContentFilter.doFilterInternal(FormContentFilter.java:93)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.ServerHttpObservationFilter.doFilterInternal(ServerHttpObservationFilter.java:113)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:201)
	at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:116)
	at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:164)
	at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:140)
	at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:167)
	at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:90)
	at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:483)
	at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:115)
	at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:93)
	at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
	at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:344)
	at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:384)
	at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:63)
	at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:905)
	at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1741)
	at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:52)
	at java.base/java.lang.VirtualThread.run(VirtualThread.java:329)

What did I do wrong?
Thank you all in advance.

mbekhta · October 21, 2024, 7:09am

Hey,

Thanks for bringing this up. Since you got the AssertionFailure, there’s definitely something to address in Hibernate Search here. Could you please put this in a test case using the template hibernate-test-case-templates/search/hibernate-search-7/orm-lucene at main · hibernate/hibernate-test-case-templates · GitHub and we’ll take a look at what’s happening there… Thanks!

Dmytro_Vasyliev · October 21, 2024, 8:03am

Hey Marko!

Thanks for you reply.
Should I create a PR with my test case?

mbekhta · October 21, 2024, 8:26am

Either a PR or simply add a link to a branch from your fork, both would be fine

Topic		Replies	Views
This iterator can only move forward. Assertion error Hibernate Search	3	11	October 21, 2024
Could not add sortable annotation on some analysed field? Hibernate Search	1	795	July 31, 2018
Sorting not working when update entity Hibernate Search	5	481	June 14, 2021
Cannot apply sort (Version 5.x) Hibernate Search	4	639	November 25, 2020
Sorting on I'd as well as local date time field in hibernate search Hibernate Search	6	1923	June 14, 2021

"This iterator can only move forward" issue

Related topics