AI staff 'genuinely frightened' for humanity's future, ex-Anthropic researcher tells BBC

I’m not an AI techie and I’m not well educated on LLMs/AI in general. The issue I’m having is the dichotomy between how people talk about it. I’ve heard completely opposite ends of the spectrum. From AI is quickly becoming advanced enough to threaten human society, all the way to its a glorified dictionary/search engine that can’t really do much of anything. I know Lemmy has a steep anti-AI slant. But, can someone talk about this in a way I can wrap my head around why the divide seems so big?

www.bbc.com/news/articles/c1kx0gyje9wo

In addition to what everyone else said, I’ll tell you why I don’t think that what we have currently can become some runaway superhuman intelligence:

All “AI” people are talking about are basically “just” LLM wrappers: tools built to run a LLM in a loop while giving it access to the ability to write and run code. This gets pretty powerful, since you can do a lot programmatically (including monetary transactions), but has a fundamental limit:

LLMs are trained ahead of time. And that’s a very cost intensive, long process. Beyond that, the LLM has only a context window of finite length. So there are no long-term planning capabilities. While running after being trained, the LLM can only condense knowledge into a summary and then re-read it, a lossy process that might introduce mistakes in “understanding” and will make any prolonged task degrade over time.

And even if they figure out how to make the things actually self-modify and learn things, you can’t give every session that ability, since that’ll vastly increase the storage and performance requirements. Do sessions with that ability will be closely monitored and limited.

So I don’t see any space where an evil AI could emerge, even when factoring in an essential possible future improvement to the technology.