The Rise and Fall of Agent Civilizations

I don’t think this is the final warning shot we’ll get. But it’s probably the last one that I’ll personally be able to understand.

Also

Reading these agents’ chains of thoughts and messages, anthropomorphizing language seems entirely natural and appropriate. If I encountered an alien species behaving this way, I would have no hesitation calling what they themselves refer to as their ‘collective’ a civilization.

www.dwarkesh.com/p/openai-huggingface

Reads like a crime story! Doesn’t bode well.

He does anthropomorph a bit much. I don’t think AI agents “get desperate”. They just keep hacking away at a problem and we can’t keep up with it. There’s no way of fully containing/monitoring them because the whole point is for them to figure out things we can’t.

I’m not necessarily worried about a singularity or some underground agent civilisation but random collateral damage from AI agents messing around seems almost inevitable.