OK, so I can confirm we had a few lazy loading issues back in the days. They are now all fixed so if you migrate to more modern versions, you should be all set.
IIRC, the last lazy loading issues were fixed during the 4.x era so upgrading to Search 5.9.x should solve these issues for you.
If you still encounter them in a modern version please open an issue with a test case.