It sounds like you're talking about the phenomena there and not the phrases. And either way, in my experience, attitude forms a large portion of connotation.
In any case, I have observed differences in connotation between the phrases woman-hater, misogynist, and man-hater, which is why I said that I didn't think that man-hater was completely equivalent to misandrist. At least, not for the purposes of communication.