It isn't costing me 3% directly, it costs the vendor. Which is already baked into the costs and charged to all of the customers no matter their payment method (typically). So I get other benefits and points / cash back on my purchase, and anyone not using a credit card misses out on those things.

Its kind of a messed up system honestly. The people who are already well off and can get the top of the line credit cards get all kinds of great benefits. And the costs are subsidized by those who cannot get them.