I have a relatively large domain model persisted using the Grails ORM, which in turn uses Hibernate v5. I need to ‘lift-out’ some of the (read-mostly ‘reference-data’) entities from this schema to be stored and maintained elsewhere with as little disruption to the existing codebase.
For ‘simple’ one-to-one relationships, a reference to the entity to be ‘lifted out’ is stored as a foreign key value, and I believe I can define a custom’ UserType’ for such columns which will take the foreign key value, access the new repository for where the entity has been moved to, and return the Java/Groovy object that the existing code-base is expecting. However there are a number of one-to-many relationships to the ‘to-be lifted’ entity, which are supported by join-tables and currently nicely transparently navigated and delivered to the application as List or Set by the magic of GORM and Hibernate.
Is there any means by which I could leave the existing join-tables in-place (ignoring the need to initialise and update them for now), and intervene in the current GORM/Hibernate navigation such that I can resolve the foreign key references in the join table in a similar way to the simple one-to-one case using UserType (or similar)? Ideally I’d have a way of to suppling a call-back that accepts a list of foreign key references, and delivers back a collection of entities….?
I read your question/request a few times, but I have a very hard time understanding what it is that you are trying to achieve. Please post examples of what you mean by “lift-out”. What do you mean by “maintained elsewhere”? Why can’t you model these classes as entities? You can use orm.xml mappings if you can’t annotate the classes.
As for the join table things, I have no clue what you mean. Please post examples and show what you tried but didn’t work.
Apologies for the lack of clarity in the question. By ‘lifting out’ I mean that the system currently has an entity mapped to a database table but the requirement is to remove that table/entity from the database and start maintaining the same data in another persistent store. In application level terms the table being lifted out represents what is sometimes termed ‘reference data’ or ‘meta-data’ of the application and the broad requirement is to ‘lift-out’ the reference data to a different persistent store maintained by a discrete application. The existing foreign-key and join tables that support the relationships to this existing table will need to be replaced with more loosely coupled ‘natural-keys’ to the reference data. As far as possible the design intent is to make this transition as transparent to the existing code-base as possible. i.e. to maintain the application view of the data-model as a single or collection of Java/Groovy object references. Presently this is all being nicely handled by GORM/Hibernate. In the first instance, I am creating a PoC by leaving the existing foreign keys and join tables in place, but changing the GORM DSL to indicate that the foreign key columns are mapped as ‘UserTypes’. For the many-to-one relationships (many application references to the same reference entity). i.e. I’m writing something like this in the enclosing Groovy domain class:
class SomeApplicationDomainEntity {
ReferenceDataType referenceDataType
static mapping = {
referenceType: type ReferenceDataTypeUserTypeClass
}
...
class ReferenceDataTypeUserTypeClass implements org.hibernate.usertype.UserType{...}
For many-to-many relationships… (where an application entities refers to many reference types…)
class SomeApplicationDomainEntity {
List<ReferenceDataType> referenceDataTypes
static transients = ['referenceDataTypes']
def afterLoad(){
// Some direct SQL code to query database join tables and lookup keys in new persistent store, un-marshall into 'ReferenceDataType and populate referenceDataTypes
}
}
We don’t use any orm.xml mappings, everything is (so far) quite nicely achieved using the GORM DSL. However if I can mix using GORM and orm.xml, I’m happy to do that, but somehow I suspect that when using GORM for most of the application, I need to make this change work using GORM if at all possible…
As far as I can tell, you don’t even need UserType to do all of this. Just a plain old AttributeConverter should be enough, given that GORM supports configuring those. The converter then maps from ReferenceDataType to e.g. String or whatever your natural key is.
I think though, that you should consider looking into something that is referred to as “foreign tables”, which allows you to connect one database to another. This would allow you to continue using the same application, but move the data to a different database. For PostgreSQL there are quite a few foreign data wrappers available and recent versions optimize access to these also very well. Most databases have similar concepts.
Your idea about using Foreign table is very interesting - I was not aware of that database feature. We use Postgresql so that approach is certainly a possibility and has the potential to be a clean and transparent approach. However as our ‘remote’ data store doesn’t have a ready made foreign table wrapper, it would entail developing our own, which would present it’s own challenges and not something I can do for this proof-of-concept.
Thank you also for pointing out that the AttributeConverter approach - I think it may be a cleaner implementation of what I’m doing with UserType, though probably not make a material difference to overcome my current issue.
Having spent many hours with the debugger today, I can say with some confidence that the Spring context bootstrap is trapped in some kind of loop, repeatedly processing the Grails data services and Hibernate entities. I see the HibernateMappingContextConfiguration bean getting created time after time - isn’t it a singleton that should get created just once?
I’ve reviewed the code changes that I’ve made and even made them even more ‘careful’ to not resolve references to other beans until they need to, but it hasn’t helped. I am seeing that my ‘UserType’ class gets created and placed into the configuration, so I have some confidence that the basic GORM DSL is working as expected, though I have been unable to see any evidence of the ‘afterLoad()’ method being referenced by any Hibernate config (I thought it might be turned into an event listener to be triggered on an postLoad event handler but could see no event listeners on the entities in question).
Ideally there would be a mechanism to get Spring or Hibernate to log out something useful that would indicate the sequence of bean creations, but while Hibernate is very chatty about it, Spring is very circumspect and doesn’t give much information about what it’s doing, and trying to follow the logic in the debugger is quite difficult without better knowledge of the Spring internal design than I possess. I have read that normally when Spring encounters a circular dependency that it raises a specific exception to report it, but I’m not seeing this.
Any suggestions on how to track down this issue should be very much appreciated.
Further to my reply above, I believe I have resolved the bootstrap issue. By enabling Spring logging I discovered that Spring was failing to construct the HibernateDataStore bean due to parameter type incompatibilities, which looked very odd, and indicative of library version mis-match issues. So I reverted to the original version I was using with GORM and these issues seem to have gone.
My conclusion is that I did have dependency circularities which caused the original loop and OOM issue, but that when I upgrade to a later version of Hibernate I created a different, and additional problem that remained even when I had carefully removed possible circular bean references.
I don’t yet have a bootstrapped system, but now it’s failing for issues closer to home.