I ended up creating a custom property entity with
@ConfigurationProperties(prefix = "hibernatesearchpropertiesxxx")
public class HibernateSearchPropertiesXXX extends HibernateSearchProperties
and used it to set the property in the EntityManagerFactory
Properties p = new Properties();
p.put("hibernate.search.backend.directory.root", hibernateSearchPropertiesXXX.getIndexRoot());
emfb.setJpaProperties(p);