That’s happening because Hibernate compares and looks up String objects via equals/hashCode and obviously, "a" is not equal to "A" and these objects also have different hash codes. Your database on the other hand is probably using a case insensitive collation (I guess you use MySQL?) and will compare strings based on that collation. There is no way to configure Hibernate to do the same, and I would also strongly recommend that you stop rely on something like that by default. Case insensitive comparison should IMO be something that you only do when absolutely necessary, and in that case, do it explicitly.
So my advice is, change the collation of your database to something like utf8mb4_0900_as_cs (in case of MySQL). Note that you might have to change the collation of every column in every table, not sure how changing collations works exactly.
If you can’t change the collation for some reason, at least make sure that the cases match to avoid the issues you are facing, by e.g. creating triggers that do lower(col) or upper(col) on the FK and/or primary key column. I would ask myself though, how it is possible that you have a FK value with a different case than the PK value. Seems to me, that there might be a bug hiding or someone simply inserted this entry by hand.