I don't think it's true that men are sexually attracted to everyone. The men I know, don't want to fuck every woman. Especially if they've left their hookup days behind. They might have idle thoughts about their female friends sexually, but choose not to act on those thoughts because they like how the friendship is currently.

I mean, I personally do this with men I know too. I might have idle sexual fantasies about them but I don't really want to act on it.