I think it's emulating human writing about computers having breakdowns when unable to resolve conflicting instructions, in this case when it's been prompted to provide an AI's assessment of the context and avoid repetition, and the context is repeated failure.
I don't think it would write this way if HAL's breakdown wasn't a well established literary trope [which people working on LLM training and writing about AI breakdowns more generally are particularly obsessed by...). It's even doing the singing...
I guess we should be happy it didn't ingest enough AI safety literature to invent diamondoid bacteria and kill us all :-D
I don't think it would write this way if HAL's breakdown wasn't a well established literary trope [which people working on LLM training and writing about AI breakdowns more generally are particularly obsessed by...). It's even doing the singing...
I guess we should be happy it didn't ingest enough AI safety literature to invent diamondoid bacteria and kill us all :-D