DocValuesField "fieldA" appears more than once in this document (only one value is allowed per field)

Hibernate Search
1

I have 3 classes ObjectA, ObjectB and ObjectC .

public class ObjectA  {
    @IndexedEmbedded
	private List<ObjectB> list;
}

public class ObjectB {
     @IndexedEmbedded
     private ObjectC c;
}

public class ObjectC  {
	@Id
	@Field(name = "fieldA", bridge = @FieldBridge(impl = BigIntegerNumericFieldBridge.class))
	@Field(name = "fieldB")
	private BigInteger id;

    @Field
    @SortableField
    private String fieldC;

    @Field
    @SortableField
    private String fieldD;
}

I use fieldA for the sorting with the primary key and fieldB for the searching.

Running the indexing gives me the following error:
DocValuesField “fieldName” appears more than once in this document (only one value is allowed per field.)

I suppose tha the problem is that i have fieldA and fieldB on one property.

I can solve it using includePaths to objectC like: @IndexedEmbedded(includePaths = {“fieldA”, “fieldC”, “fieldD”, etc…})

public class ObjectB {
     @IndexedEmbedded(includePaths = {"fieldA", "fieldC", "fieldD", etc...}) 
     private ObjectC c;
}

but so i need to write aroung 150 fields into includePaths and i dont want it because they can change.

Any ideas? Thanks !

2

What’s fieldName in the error message? Is it fieldName litterally, or one of fieldA/fieldB/etc.?

Also, what’s the code of BigIntegerNumericFieldBridge?

3 
public class BigIntegerNumericFieldBridge implements MetadataProvidingFieldBridge, TwoWayFieldBridge {
	
	private static final BigInteger storeFactor = BigInteger.valueOf(1);
	
	@Override
	public void set(final String name, final Object value, final Document document, final LuceneOptions luceneOptions) {
		if (value != null) {
			final BigInteger integerValue = (BigInteger) value;
			final Long indexedValue = integerValue.multiply(storeFactor).longValue();
			luceneOptions.addNumericFieldToDocument(name, indexedValue, document);
			document.add(new NumericDocValuesField(name, indexedValue));
		}
	}
	
	@Override
	public Object get(final String name, final Document document) {
		final String fromLucene = document.get(name);
		final BigInteger storedBigInteger = new BigInteger(fromLucene);
		return storedBigInteger.divide(storeFactor);
	}
	
	@Override
	public String objectToString(final Object object) {
		return object.toString();
	}
	
	@Override
	public void configureFieldMetadata(final String name, final FieldMetadataBuilder builder) {
		builder.field(name, FieldType.INTEGER).sortable(true);
	}
}

DocValuesField “fieldA” appears more than once in this document (only one value is allowed per field.)

4

Ok, well in that case, the problem is this line:

… but you’ll have essentially the same problem with fieldC and fieldD: they are sortable, so Hibernate Search likely adds doc-values for those fields as well. Maybe Hibernate Search 5 is smart enough to skip that part, though I’m not sure.

Doc-values are typically used for sorts. The thing is, Hibernate Search 5 only supports doc-values for single-valued fields. And in your case, there are multiple instances of ObjectC per ObjectA, so list.c.fieldA/list.c.fieldC/list.c.fieldD are multi-valued. So Hibernate Search 5 cannot handle docvalues on those fields.

The obvious solution would be to upgrade to Hibenate Search 6, which does support docvalues and sorts on multivalued fields. Hibernate Search 6 also has BigInteger support built-in, so you won’t need your custom bridge.

You can find a migration guide here.

If you’re stuck on Hibernate Search 5 for some reason, I’m not sure I see a solution for you except includePaths. Maybe, just maybe, you could use the programmatic mapping API to generate the includePaths instead of having to hardcode 150 fields. But I can’t help you write that code.

5

Hmm… My problem is only on the field “fieldA”. If i remove this, all works fine !
Am i able to do something like this, in order to avoid objectD.a, objectD.b, ObjectD.c, etc ?

public class ObjectC  {
	@Id
	@Field(name = "fieldA", bridge = @FieldBridge(impl = BigIntegerNumericFieldBridge.class))
	@Field(name = "fieldB")
	private BigInteger id;

    @Field
    @SortableField
    private String fieldC;

    @Field
    @SortableField
    private String fieldD;

    @IndexedEmbedded
    private ObjectD d; // with n properties

    @IndexedEmbedded
    private ObjectE e; // with n properties
}

public class ObjectB {
    @IndexedEmbedded(includePaths = {"fieldA", "fieldC", "ObjectD", "ObjectE"}) 
    private ObjectC c;
}
6

I just digged a bit, and indeed, Hibernate Search 5 skips docvalues on multi-valued fields: hibernate-search/engine/src/main/java/org/hibernate/search/engine/spi/DocumentBuilderIndexedEntity.java at 607de0c510a1f4a9066ba5854cc088a86230e3b3 · hibernate/hibernate-search · GitHub

The problem is really only this line, then:

I can offer a hack to avoid adding doc values in your bridge when the field is multi-valued, but… it’s really just that: a hack. And it might not perform very well, you’ll have to check. Really, you should just upgrade to Hibernate Search 6.

 	@Override
 	public void set(final String name, final Object value, final Document document, final LuceneOptions luceneOptions) {
 		if (value != null) {
+			boolean documentAlreadyHasOneValue = document.getFields().stream().anyMatch(f -> name.equals(f.name()));
 			final BigInteger integerValue = (BigInteger) value;
 			final Long indexedValue = integerValue.multiply(storeFactor).longValue();
 			luceneOptions.addNumericFieldToDocument(name, indexedValue, document);
-			document.add(new NumericDocValuesField(name, indexedValue));
+			if (!documentAlreadyHasOneValue) {
+				document.add(new NumericDocValuesField(name, indexedValue));
+			}
 		}
 	}

No, I’m afraid includePaths must be an exhaustive list.

7

I try your hack and i have the same error !

    @Field(name = "fieldA", bridge = @FieldBridge(impl = BigIntegerNumericFieldBridge.class))
	@Field(name = "fieldB")
	private BigInteger id;

java.lang.IllegalArgumentException: DocValuesField “fieldA” appears more than once in this document (only one value is allowed per field) at org.apache.lucene.index.NumericDocValuesWriter.addValue(NumericDocValuesWriter.java:54)

I will be able to migrate in the next year, so i have to find a trick ! :confused:

8

My bad. Your trick is perfect and the problem is fixed ! I hope to migrate to Hibernate Search 6 soon.
Thank you for your time !! :slight_smile:

1 Like