“Hate” comes from my own memory, while “what I want to hear” is hard because it depends on the other person’s present state.
What would make me say, 'I’d hate to hear that,' is something I can quickly pull from my own experiences or hurt memories, but what I want to hear requires guessing the other person's current inner state, so I think the right answer isn’t clear and that’s scary.