When attempting to update indexed entities, deletions and insertions are occurring instead of direct updates

I created a new entity connected to Hibernate Elastic Search and indexed it. Upon retrieving the indexed data, I noticed that updating the entity using the student ID resulted in deleting the existing data and re-inserting the updated data. To preserve the existing data along with the updated data, I need to merge the updated data with the existing data before indexing the entity.

Student insertion request

{
	"studentId": "1",
	"studentMark": "10",
	"studentName": "LMN",
	"data": [
		{
			"id": "1",
			"mark": "10",
			"name": "ABC",
			"status": "Inprogress"
		},
		{
			"id": "5",
			"mark": "10",
			"name": "XYZ",
			"status": "READY_TO_PROCESS"
		}
	]
}

Upon completing the data insertion, I fetched the latest data from the Elasticsearch server.
Response

{
	"took": 8,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 1,
			"relation": "eq"
		},
		"max_score": 1.0,
		"hits": [
			{
				"_index": "student-000001",
				"_type": "_doc",
				"_id": "1",
				"_score": 1.0,
				"_source": {
					"data": [
						{
							"id": "1",
							"mark": "10",
							"name": "ABC",
							"status": "Inprogress"
						},
						{
							"id": "5",
							"mark": "10",
							"name": "XYZ",
							"status": "READY_TO_PROCESS"
						}
					],
					"studentId": "1",
					"studentMark": "10",
					"studentName": "LMN",
					"_entity_type": "StudentEntity"
				}
			}
		]
	}
}

Update the entity using the student ID.
Request

{
	"studentId": "1",
	"studentMark": "10",
	"studentName": "OPQ",
	"data": [
		{
			"id": "1",
			"mark": "10",
			"name": "HIJ",
			"status": "READY_TO_PROCESS"
		}
	]
}

After updating the data, I retrieved it again from the Elasticsearch server to verify the changes.
Response

{
	"took": 8,
	"timed_out": false,
	"_shards": {
		"total": 1,
		"successful": 1,
		"skipped": 0,
		"failed": 0
	},
	"hits": {
		"total": {
			"value": 1,
			"relation": "eq"
		},
		"max_score": 1.0,
		"hits": [
			{
				"_index": "student-000001",
				"_type": "_doc",
				"_id": "1",
				"_score": 1.0,
				"_source": {
					"data": [
						{
							"id": "1",
							"mark": "10",
							"name": "HIJ",
							"status": "READY_TO_PROCESS"
						}
					],
					"studentId": "1",
					"studentMark": "10",
					"studentName": "OPQ",
					"_entity_type": "StudentEntity"
				}
			}
		]
	}
}

i found that after updating the entity defiantly its deleting the existing data and re-inserting the new changes.
How to avoid this change. i need to get existing as well as the updated data. Please help me too fix this issue

Hibernate Search always rebuilds the entire document and reindexes that. There is no support for partial updates at the moment.

However, this is generally not a problem because the entity is available from the database (or another source, in the case of the standalone mapper), and the document can be reconstructed from that.

I guess in order to help you, we’d need some more information on your use case and some code samples. I’d like to understand why some of the data is not available on updates, in particular. If you’re using Hibernate ORM, it’s possible some part of the Hibernate ORM mapping is incorrect?

Hi @yrodiere

Student Entity

@Entity
@Indexed(index="student")
@Table(name = "student")
public class StudentEntity {
		
	@Id
	@GeneratedValue(strategy = GenerationType.IDENTITY)
    @GenericField(projectable = Projectable.YES, sortable = Sortable.YES)
	private String studentId;

	@Column(name = "NAME")
	@KeywordField(projectable = Projectable.YES)
	private String studentName;

	@Column(name = "MARK")
	@KeywordField(projectable = Projectable.YES)
	private String studentMark;
	
	@IndexedEmbedded
	@Column(name = "DATA")
	@OneToMany(mappedBy = "student", cascade = CascadeType.ALL, fetch = FetchType.LAZY)
	private List<DataEntity> data;

	//getters and setter
		
}

Data Entity

@Entity
@Indexed
@Table(name = "data")
public class DataEntity {

	@Id 
	@GeneratedValue(strategy = GenerationType.IDENTITY)
    @KeywordField
	private String id;

	@Column(name = "NAME")
	@KeywordField(projectable = Projectable.YES)
	private String name;

	@Column(name = "MARK")
	@KeywordField
	private String mark;
	
	@Column(name = "STATUS")
	@KeywordField
	private String status;

	@ManyToOne(cascade = CascadeType.ALL)
    @JoinColumn(name = "student_id")
    private StudentEntity student;
	
	//getters and setter		
}

I’m encountering an issue where, after updating individual values in a data entity, the changes only persist in the Elasticsearch server. It seems after updating the child entity, reindexing is not happening for that child entity.

Okay, this model seems correct and is not particularly exotic. It should work.

I guess the problem lies in the code that performs the entity persist/update.

Maybe you’re not updating both sides of your association? Have a look at this: Hibernate Search 7.0.0.Final: Reference Documentation

Or – less likely, but I’ve seen worse – maybe you’re erasing the content of StudentEntity#data, by doing something incorrect like student.setData(List.of(newData))?

If that’s not your problem, I’d suggest creating a reproducer based on this template: hibernate-test-case-templates/search/hibernate-search-7/orm-elasticsearch at main · hibernate/hibernate-test-case-templates · GitHub Then we can have a look and see what’s wrong exactly.

Hi @yrodiere

@PutMapping(value = "/update")
	public String updateData(@RequestBody StudentModel data) {
		String id = data.getStudentId();
		if(id !=null) {
			StudentEntity findById = dataRepository.findByStudentId(id);
			if(findById!=null) {
				findById.setStudentName(data.getStudentName());
				findById.setStudentMark(data.getStudentMark());
				List<DataEntity> data2 = new ArrayList<DataEntity>();
				for(DataModel model:data.getData()) {
					DataEntity dataEntity=new DataEntity();
					dataEntity.setId(model.getId());
					dataEntity.setName(model.getName());
					dataEntity.setMark(model.getMark());
					dataEntity.setStatus(model.getStatus());					
					data2.add(dataEntity);
				}
				dataRepo.saveAll(data2);
				findById.setData(data2);
			}
			dataRepository.save(findById);
		}
		
		return (id + " Updated");
	}

After updating the child entity, I also update the parent entity for consistency

You’re not though? I don’t see a line like dataEntity.setStudent(findById).

Hi @yrodiere

i have modified my entities as well as my Apis

Student Entity

@Entity
@Indexed(index="student")
@Table(name = "student")
public class StudentEntity {
	
	@Id
	@GeneratedValue(strategy = GenerationType.IDENTITY)
    @GenericField(projectable = Projectable.YES, sortable = Sortable.YES)
	private String studentId;

	@Column(name = "NAME")
	@KeywordField(projectable = Projectable.YES)
	private String studentName;

	@Column(name = "MARK")
	@KeywordField(projectable = Projectable.YES)
	private String studentMark;
	
	@IndexedEmbedded
	@OneToMany(mappedBy = "student", cascade = CascadeType.ALL, fetch = FetchType.LAZY)
	private List<DataEntity> data;

	//getters and setter

Data Entity

@Entity
@Indexed
@Table(name = "data")
public class DataEntity {

	@Id 
	@GeneratedValue(strategy = GenerationType.IDENTITY)
    @KeywordField
	private String id;

	@Column(name = "NAME")
	@KeywordField(projectable = Projectable.YES)
	private String name;

	@Column(name = "MARK")
	@KeywordField
	private String mark;
	
	@Column(name = "STATUS")
	@KeywordField
	private String status;

	@JsonIgnore
	@ManyToOne(cascade = CascadeType.ALL)
    @JoinColumn(name = "student_id")
    private StudentEntity student;
	
	//getters and setter

To prevent a recursion error during serialization, I’ve applied the @JsonIgnore annotation to the Student entity within the relevant Data Entity

insert Api

@PostMapping(value = "/insert")
	public String saveData(@RequestBody StudentModel data) throws JsonProcessingException {
		logger.info("Request !!!!!!!"+mapper.writeValueAsString(data));
		StudentEntity StudentEntityData = new StudentEntity();
		StudentEntityData.setStudentId(data.getStudentId());
		StudentEntityData.setStudentName(data.getStudentName());
		StudentEntityData.setStudentMark(data.getStudentMark());
		List<DataEntity> data2 = new ArrayList<DataEntity>();
		for(DataModel model:data.getData()) {
			DataEntity dataEntity=new DataEntity();
			dataEntity.setId(model.getId());
			dataEntity.setName(model.getName());
			dataEntity.setMark(model.getMark());	
			dataEntity.setStatus(model.getStatus());
			dataEntity.setStudent(StudentEntityData);
			data2.add(dataEntity);
		}
//		dataRepo.saveAll(data2);
//		logger.info(mapper.writeValueAsString(dataRepo.findAll()) );
		StudentEntityData.setData(data2);
		
//		dataRepository.save(StudentEntityData);
		logger.info("StudentEntity !!!!!!!"+mapper.writeValueAsString(StudentEntityData));
		dataRepository.save(StudentEntityData);
		String id = data.getStudentId();
		return (id + " added");
	}

During the insertion process, I’m also associating the Student entity with the Data Entity by setting a reference to it within the Data Entity’s structure

Update Api

@PutMapping(value = "/update")
	public String updateData(@RequestBody StudentModel data) throws JsonProcessingException {
		String id = data.getStudentId();
		logger.info("Id !!!!!!!"+id);
		if(id !=null) {
			StudentEntity findById = dataRepository.findByStudentId(id);
			logger.info("findById !!!!!!!"+mapper.writeValueAsString(findById));
			if(findById!=null) {
				findById.setStudentName(data.getStudentName());
				findById.setStudentMark(data.getStudentMark());
				logger.info("findById after setting  !!!!!!!"+mapper.writeValueAsString(findById));
				List<DataEntity> data2 = new ArrayList<DataEntity>();
				for(DataModel model:data.getData()) {
					DataEntity dataEntity=new DataEntity();
					dataEntity.setId(model.getId());
					dataEntity.setName(model.getName());
					dataEntity.setMark(model.getMark());
					dataEntity.setStatus(model.getStatus());	
					dataEntity.setStudent(findById);
					data2.add(dataEntity);
				}
//				dataRepo.saveAll(data2);
				findById.setData(data2);
			}
			logger.info("before saving  !!!!!!!"+mapper.writeValueAsString(findById));
			dataRepository.save(findById);
		}
		
		return (id + " Updated");
	}

As part of the update process, I’m establishing a relationship between the Student entity and the Data Entity by embedding a reference to the Student entity within the Data Entity’s structure.

Student insertion request

{
	"studentId": 1,
	"studentName": "namjas",
	"studentMark": 10,
	"data": [
		{
			"id": 1,
			"name": "namjas1",
			"mark": 10,
			"status": "READY_TO_PROCESS"
		},
		{
			"id": 2,
			"name": "namjas2",
			"mark": 10,
			"status": "Inprogress"
		},
		{
			"id": 3,
			"name": "namjas3",
			"mark": 10,
			"status": "Stopped"
		},
		{
			"id": 4,
			"name": "namjas4",
			"mark": 10,
			"status": "COMPLETED"
		}
	]
}

Upon completing the data insertion, I fetched the latest data from the Elasticsearch server.
Response

{
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": "student-000001",
                "_type": "_doc",
                "_id": "1",
                "_score": 1.0,
                "_source": {
                    "data": [
                        {
                            "id": "1",
                            "mark": "10",
                            "name": "namjas1",
                            "status": "READY_TO_PROCESS"
                        },
                        {
                            "id": "2",
                            "mark": "10",
                            "name": "namjas2",
                            "status": "Inprogress"
                        },
                        {
                            "id": "3",
                            "mark": "10",
                            "name": "namjas3",
                            "status": "Stopped"
                        },
                        {
                            "id": "4",
                            "mark": "10",
                            "name": "namjas4",
                            "status": "COMPLETED"
                        }
                    ],
                    "studentId": "1",
                    "studentMark": "10",
                    "studentName": "namjas",
                    "_entity_type": "StudentEntity"
                }
            }
        ]
    }
}

Update the entity using the student ID.
Request

{
	"studentId": 1,
	"studentName": "Qwerty",
	"studentMark": 10,
	"data": [
		{
			"id": 1,
			"name": "xyz1",
			"mark": 10,
			"status": "STOPPED"
		},
		{
			"id": 5,
			"name": "xyz2",
			"mark": 10,
			"status": "COMPLETED"
		}
	]
}

After updating the data, I retrieved it again from the Elasticsearch server to verify the changes.
Response

{
    "hits": {
        "total": {
            "value": 1,
            "relation": "eq"
        },
        "max_score": 1.0,
        "hits": [
            {
                "_index": "student-000001",
                "_type": "_doc",
                "_id": "1",
                "_score": 1.0,
                "_source": {
                    "data": [
                        {
                            "id": "1",
                            "mark": "10",
                            "name": "xyz1",
                            "status": "STOPPED"
                        },
                        {
                            "id": "5",
                            "mark": "10",
                            "name": "xyz2",
                            "status": "COMPLETED"
                        }
                    ],
                    "studentId": "1",
                    "studentMark": "10",
                    "studentName": "Qwerty",
                    "_entity_type": "StudentEntity"
                }
            }
        ]
    }
}

I retrieved the specified Student entity from the database using the Retrieve API.

[
    {
        "studentId": "1",
        "studentName": "Qwerty",
        "studentMark": "10",
        "data": [
            {
                "id": "1",
                "name": "xyz1",
                "mark": "10",
                "status": "STOPPED"
            },
            {
                "id": "2",
                "name": "namjas2",
                "mark": "10",
                "status": "Inprogress"
            },
            {
                "id": "3",
                "name": "namjas3",
                "mark": "10",
                "status": "Stopped"
            },
            {
                "id": "4",
                "name": "namjas4",
                "mark": "10",
                "status": "COMPLETED"
            },
            {
                "id": "5",
                "name": "xyz2",
                "mark": "10",
                "status": "COMPLETED"
            }
        ]
    }
]

Still getting the same only.

I may be wrong, but as far as I can see, this is not related to Hibernate Search: Hibernate Search faithfully indexes the content of your entity, it’s just that the content of your entities is not what you seem to think.

This right here. This is you, replacing the content of StudentEntity#data with a new list, instead of merging it.

If you want to merge it, just… merge it? Don’t create a new list, just add elements to findById.getData().