I also find it interesting that you say that sex is "more vulnerable and intimate" than a conversation, and yet on another sentence you say "people who just want sex" as if it's only physical for them, but deeply emotional for you.
If it's your spouse, why is it hard to believe that they want to have sex with you BECAUSE it is the more vulnerable and intimate experience for them as well, and they want to connect on that deep level with you, and if you're rejecting that it's rejecting the bid for emotional connection, not brushing off a simple physical spasm of pleasure they can get with their hand and "isn't a big deal"?
This is not to say you should have sex if you really don't want to, my point is that yes sex is vulnerable and emotional and that's why your partner wants it, most likely. They want to connect deeply and profoundly. Not just experience physical pleasure with you.
And so if you don't want to do that with them, how else can you connect on a deep profound level beyond what you would do for "just anyone" like you dismissively say for all the other love languages?