I wonder if it's because of the kind of clothing that is associated with wealth? Rich people wear expensive clothing and expensive clothing tends to be designed for the very thin. Supermodels tend to have tiny breasts.

Whereas poorer men may be more interested in how a woman looks when naked, and thus prefer bigger breasts.

There might also be a subconscious association of breast size with "wanting to be nurtured and taken care of". A poor man may want a woman to nurture and take care of him in many ways, including financially. A rich man may feel emasculated subconsciously by the idea of a woman providing for him.