I would certainly hate that if it happened to me. I feel like if you're in a sexless marriage, and your partner isn't willing to look into why they don't want to have sex (because sex is an important love language, it's not just about physical pleasure, it's about intimacy and denial of sex is not a denial of an orgasm but a denial of close physical connection and vulnerability), not willing to work together with you to try and become more comfortable having sex, not willing to let you be polyamorous - then you should get a divorce. It's not worth living your whole life without intimacy.

If I stopped wanting sex I would let my husband be polyamorous. However I would want to still connect with him romantically.

However I'm actually the higher libido partner so I doubt that I'll lose my sex drive haha.