TLDR; My bad, Disabling cache in Hiernae works as intended.
Delved deeper into our code base, turns out our Spring configuration get the hibernate properties from an "p:location=“classpath:hibernate.properties” file (in our code base) AND an optional “p:location=file:///${mycompany.home}/configuration/hibernate.properties” file. The latter override settings in the former. In the optional file the caching was set to true.
Thanks for your help, very much appreciated.