Wouldn’t those be aspects of one’s self-image that a response might challenge?
Addendum:
I’ve reread your post. You’re saying that one might be angered by the response because the person making it might be those things. Sorry for the misunderstanding.