That's really sad. It sounds like the way you perceive each other is different. You perceive him as Your Person. Your number 1. He sees you as "one of" his people, and not the priority. That's got to hurt.

I honestly feel as though he must be pretty inconsiderate of your feelings to not even TALK to you for several days after you have sex and when he would know you're in a vulnerable place?

He could have chosen to be a life partner with you (polyamorous or not). But he didn't. And it's not chance that you "found each other in the parameters of polyamory" - he CHOSE to continue seeking other partners instead of deepening his commitment with you. Sure I understand his family is his family. But he could have chosen to invest more of his time into building a relationship with you and instead he chose to date around because his existing partners "weren't enough".

It seems like he will always be that kind of person. And you will be his "escape" from the monotony of "ordinary life", like a mistress except "ethical". But from how you describe things he doesn't seem to want to have you be part of "ordinary life", maybe because having you as the "escape" is more exciting?