How to prevent Hibernate to making a column with unique = true automatically

Why do you need TableAb directly in TableDa? If you change TableA to this:

@Entity
@Table(name = "table_a")
public class TableA implements Serializable {
    private static final long serialVersionUID = 1L;
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    @Basic(optional = false)
    @Column(name = "id")
    private Long id;
    @Basic(optional = false)
    @Column(name = "name")
    private String name;
    @OneToMany(mappedBy = "tableA")
    Set<TableAb> tableAbs;
}

you can access the TableBs as well, but through the proper associations. A @ManyToOne essentially is like a FK and a FK requires that the target side is unique, that’s required by any sane DBMS AFAIK.

You could try to map TableAbs as @OneToMany but I guess Hibernate then rightfully forces a unique constraint on table_da(id_a) which you probably don’t want.

@Entity
@Table(name = "table_da")
public class TableDa implements Serializable {
    private static final long serialVersionUID = 1L;    
    @EmbeddedId
    private TableDaPK tableDaPK;

    @OneToMany
    @JoinColumn(name = "id_a", referencedColumn = "id_a")
    private Set<TableAbs> tableAbs;
    
    @ManyToOne(optional = false)
    @JoinColumn(name = "id_d")
    private TableD tableD;
}

The problem here is that JPA/Hibernate forces a bidirectional mapping so that it can keep state in sync. What you want here is not safe. One id_a might refer to multiple rows in table_ab so you can’t map this as @ManyToOne.