I think another way that the shame would factor into things is that if I felt I had to feel compersion, I would feel ashamed for not being "big enough" to be unselfish and not feel anything but unconditional positive emotions about the fact my partner was with someone else.