>somewhere, there is a software developer--with a human name--who built this automatic ban system

They built it because someone in Product or AdTech dreamt it up, sold it to some VP or directory, and got some project + product managers to run with it. That's like blaming irresponsible mining practices on the miners and not the people who are actually deciding to mine in the first place, or even where exactly to mine. Even in those cases, if you try to take some ethical high-road that doesn't maximize profits or serve some VP's ego, you'll likely find yourself relieved of your responsibilities in short order.

Chain of command, but also investor pressure to do everything even remotely legal to maximize profits is more to blame. If devs push back, they get replaced with devs who don't. The money is too lucrative and there are too many qualified people to really have much power to "do the right thing" in most big corps. Sure, you can voice concerns and try to change things, but if the decision is between "do a thing users will hate but will make the company 2% more per year" vs "don't do this thing and hope people will like us more for it"...yeah, they're gonna find someone to get it done.

Chain of command is a weird argument. If you don’t believe in war, then don’t work for a defense contractor. Engineers are a super-powerful enabling force.

In my most recent startup we had many hard days, but I would often say “hey, at least we aren’t designing tactical nukes, we’re helping people get jobs”

Engineers can and should take responsibility for what they work on.

Yes, evil power hungry CEO-types (I’m an eng turned CEO) will work hard and possibly succeed anyway, but that doesn’t mean engineers are absolved of responsibility.

Your proposal seems to me to put socio-economic problems on the back of individuals. Imagine what they would have to do. For every line of code they would have you predict all the socio-economic consequences. I rather think such problems should be solved politically.

At the end of the day we need to pay our bills, take care of our kids, and have something on the side for retirement.

If I don’t implement this system, someone else with less foresight or less alternatives will.

I agree with you for clearly evil stuff. But the system we’re talking about seems in a grey zone.

> For every line of code they would have you predict all the socio-economic consequences

This is like saying for every step you take, you would have to predict if it would kill you or not. This is not how humans generally function. You make simplifying assumptions, such as thinking, "If I go for a walk at night in the jungle, would that kill me or not?" and then take action based on the overall planned trip. The same logic applies to shon's argument: if you write code for a tactical nuke manufacturer, would that contribute to harm in the world? You can then decide whether to work at that company or not, rather than analyzing if the "int count = 0" you just wrote brings the company closer to its (likely) destructive goals.

Focusing on personal responsibility instead of legislation is a dead end.

OK, so I actually happen to be an elected government official, and I additionally have spent a lot of my time working on attempts at getting legislative relief from DRM. And, you know what kind of sucks? At the end of the day, democracies reduce back down to personal responsibility, as you rely on people to vote, lobby, and even donate.

So like, here's the question: if we are going to give the people we know at Google a pass for their injustice because "don't hate the player, hate the game"... are we also going to give them a pass when they don't get involved in the attempt to get the laws changed against Google?

Most of the people I've known who work for Google seem to like Google, in no small part because of all of those benefits they receive due to all of the injustices they are enabling. Even if they feel a bit bad about what Google is doing, they still refuse to get involved because they claim it will get them in trouble at work... which is just the same problem, just shifted around a bit.

At the end of the day, all we have is "personal responsibility": if you keep shifting the blame for the things that happen in the world to the system and keep waiting for other people to come up with a solution, you've done nothing but become a demonstration for the tragedy of the commons.

Capitalism demands that companies do things that aren't good for consumers in order to maximize profits. Through the lens of game theory, a lot more corporate behavior makes a lot more sense. Until regulation makes it more expensive for a company to do user-hostile things than not, they'll keep making those decisions as rational actors. Developers are a fungible commodity. I'm not saying that change never comes from developers, just that when it does, I've found that it was because it was wrapped in a pitch of how to do something another way to make even MORE profit. So long as a thing is legal, or legal enough (risk of getting caught * negotiated penalty amount vs potential profit), what motivation would they have to not do it?

Alternatively, if you allow a legislative environment where bad behavior is rewarded, then—shocker!—bad behavior will be common.

Most people and companies engaging in bad behavior aren't doing it because they're evil, they're doing it because they don't care about good/evil as much as they care about money. If you just change the rules for how to make money (and not end up in jail), they will change their behavior in an instant.

Changing laws may be difficult, but your proposed solution involves changing the brains of a large number of people. Harder, not easier.

Why? More regulation almost always ends up helping the big players.

Are you suggesting removing any regulations?

We've all seen how responsibly companies act in less-regulated markets, and the FTC is already pretty damn toothless.

If legislation helps the big players so much, why do they spend so much lobbying against certain legislation? Why do big time CEOs complain about regulation so much, if it's such a boon?