This is the key factor here that most polyamorists don't want to admit though. I know you do admit this, but the main reason that I am leery of polyamory in practice for myself (despite being onboard with it in theory) is because sure, I might be able to love multiple people and it not diminish my love for others.

But it absolutely will diminish the amount of time, energy and attention I can give my existing partner. And quality time is my main love language. So if I'm reducing the amount of quality time I can spend with someone, they ARE going to feel it as me loving them less, even if I tell them my feelings for them haven't changed.