I think these folks fail to connect the pixel with its purpose. The sites and apps who advertise want to understand who is converting, they provide this information to the advertiser so they can correlate the users who saw an ad to a purchase.
By keeping the purpose vague, it makes it seem nefarious.
I'll give you $5 if you give me access to your email account in full and let me see what you buy, what newsletters you sign up for, etc. I won't even steal any of your data or anything nefarious outside of collect some stats for my own uses.
If you let Facebook do this for free, you should at least take the $5 from me.
By keeping the purpose vague, it makes it seem nefarious.