I think the plateau might not necessarily be in terms of LLM capability, and we might even see LLM based tools keep improving at a high rate in the fields where they currently work well, but still have a plateau where we can’t expand their use to other fields.
There’s a very large gap between how well LLM based coding assistants work, and how well LLM based tools work in areas where we can’t easily create a harness that can do dozens or hundreds of LLM turns between human interactions via tool use. As a result, LLMs are much less useful at tasks where there is no agent loop that can drive iterative changes and automatic verification until the final result is much better than a one-shot generation would have produced. If we can’t come up with a way to make these agent loops for more tasks then AI will plateau even if LLMs keep improving.
Now, I think people who use coding assistants a lot and also don’t interact much with other humans who don’t might easily arrive at a conclusion like that AI running everything is inevitable. I can see this with myself too, a lot of what I do I can get Claude Code to do. Not just work stuff like actual coding or running kubernetes clusters or debugging IT problems. Also many of the random things I do at home that are not strictly coding, e.g. I’ve had Claude design 3D printed tools, or electronics circuits, entire concepts for physical toys for me to build if I get around to it. But there is a key observation to make here that keeps me from valuing its abilities too highly: I’ve been programming since I was a little kid, for almost 40 years now, and my work and my hobbies all are coding or coding-adjacent, exactly the kind of thing agent loops make good.
One look at even just my wife’s typical day though and there’s almost no AI involved. And when it is it’s ChatGPT hallucinating nonsense when she tries to get it to research something for work etc., that or other things where there is no harness that can auto-correct errors. So I don’t have a very high opinion of AI tools for general purpose tasks right now even though I, on an almost daily basis, see Claude pull something off that I had no hopes for it to get right and that would have taken me half of an eternity to do it myself.
But if my environment was mostly developers/engineers/nerds similar to me then I would have a much higher opinion of AI. I’d be hearing about impressive things an AI tool did constantly. I believe these kinds of environments are where most of the hype comes from, and there is no control, because people who don’t have AI tools for their tasks that work as well as modern coding tools mostly aren’t even aware of the conversation and certainly not too interested in participating.
www.bbc.comAI staff 'genuinely frightened' for humanity's future, ex-Anthropic researcher tells BBCIt comes as the AI firm's boss has called for the technology's development to be slowed down, citing "serious" risks.
