But what when an LLM trains its self?
Thats what I’m saying. They cannot do this, not even close. Even a fruit fly is somewhat adaptive and can respond to novel stimuli, but an LLM effectively cant.
There are tiny experimental models closer to a fruit fly than large LLMs now.
…What about future world models with some adaptive learning loop, sampling stripped out, and so on?
Yeah! That would be fascinating. Then sentience starts to become more of a question.
But thats not what LLMs are.
People like Altman and Modi have grossly overexaggerated what current LLM architectures are capable of, they trained the LLMs to reinforce the illusion as much as they can, and they arent to inclined to change it. Hence all the good researchers have fled to research world models, and are saying transformers LLMs are not a viable path forward.