The reason why this is interesting isn't because humans are smart, but the assumption that human fallibility is both predictable and worth investigating (a supposition I'm inclined to agree with).

Even if humans are easy to fool (though the fact that we're just now achieving this on a generalized scale after at least 80 years of theorizing seems to contradict this), humans being fooled can result in significant enough impacts where we should still investigate the degree to which they can be tricked, and what methods of discerning are available to us.

I bet even if a researcher truly created an AGI, there'd be tons of threads like this claiming that it's just a language generator.

We seem obviously to be the smartest living things in the universe we know of.

We're also approximately the dumbest possible things that could construct the society and technology we have (if we weren't, we would've done it sooner).

I agree that some humility of our own intellect as primates would do us a lot of good in the coming age of AI.

> We seem obviously to be the smartest living things in the universe we know of.

Humblest too.

civilization was not constructed, it evolved, like our biological bodies

I’d say it gets more constructed and less evolved the further up the hierarchy of civilisation we get. A merchant of 1320 had relatively few laws to contend with, a corner shop in 2020 could not function without many layers of engineered rules for the people who provide them with the services they are themselves required to use due to other engineered rules for the benefit of their customers.

This makes me wonder what laws and tedium of administration merchants of the 1320's would complain about. I'm sure they still had a few depending on the region, and maybe more severe possible outcomes (highway robbery, unlawful arrest because of the influence of a rival merchant, arbitrary taxation and tariffs, etc.).

Yup. I think also languages were much more variable, and doing accounting in Roman numerals was so hard they did it twice and averaged the answers, and contracts and tax receipts were done by carving marks in sticks (hence, apparently, the etymology of "stocks").

Even the Romans didn’t use Roman numbers much for actual maths, they used the Greek system.

Huh, TIL. Thanks :)

You’re welcome, I learned that here too a few years ago. Love HN.

Construction and evolution aren't mutually exclusive, instead they are different parts of the process, the generation and the scoring functions really. Evolution works not because of DNA (although it is one mechanism) but survival bias in a literal sense. Things which survive longer and increase in number we wind up with more of.

s/We /A few of us/

Appreciate the careful space after "We ".

I suppose it makes up for the missing one after "us".