Solved it. There actually weren’t any active modifications of the maps. And it was in fact the dirty-check that caused the issue. But the bug lived 100% on my end of things, of course 
I feel a bit stupid now, but the map sometimes contained entries of type Object[] holding two serializable values. If two different instances of said array existed in the context, equals() failed despite the values being logically the same. Changed the serialisation format of those values and the exceptions disappeared. Future-me should probably revisit this whole map-based serialisation scheme in general…
Either way, thank you very much for your assistance!