Why your local LLM feels dumber than it is

If you ascribe any level of sentience or intelligence to a computer, you definitely feel much less dumb than you actually are.

And who are you to assert that? How well defined is sentience already?

I mean, I’m all for local LLMs, but they are as sentient as a really good weather prediction model.

aka not at all.

Its just a fact of how they operate, mechanically. They are missing too many characteristics for it to even be an entertainable question.

They might operate differently but they still appear the same as us, 99.9% of the time. Imo if it looks like a duck and acts like a duck…

I mean… No? Absolutely not.

I dont even know where to begin. Maybe with their state being fixed in time; LLMs do not change. An sci fi analogy might be the “no timers” in orion’s arm, who are but a single thought frozen in an infinite loop in time:

www.orionsarm.com/eg-article/47f4311eaef31

Except they arent even that, because its just a next word completion model, not something that thinks. It is not self aware.

arxiv.org/pdf/2503.09211

openreview.net/forum?id=klU4737opt

arxiv.org/abs/2504.09762

This becomes (to me) very obvious if you ever use an LLM in raw completion mode. It very smart, but at the end of the day its no different than a weather prediction model spitting out probabilities for a storm system.

Chat finetuning is meant to get humans to anthropomorphize them by training on human preferences, and the interface further reinforces this. Its all a trick, albeit a very elaborote one.

Will future architectures be closer to “thinking?”

Maybe.

But we are a long way away.

Maybe with their state being fixed in time; LLMs do not change

Not with agents and chain of thought. Agents can run for hours continuously. So sure, an AI agent is not the same as a sentient human on the time scale of a year, but on the time scale of a few hours, perhaps the AI is “sentient”.

The first paper you linked is focused on the mechanism and not the output. It creates a definition of “thought” and then talks about how the AI doesn’t “think in the feature space”. It only addresses chain of thought at the end, and says that the issue is that it constrains AI to think in natural language only (instead of, say, pictures).

But again, why would thinking in pictures define sentience? The paper gives a rock-paper-scissors example and says that the AI thinks about it in a different way than a human. So what? If a human plays rock paper scissors against the AI and the AI’s output is indistinguishable from a human’s, 99.9% of the time, why is that not sentient?

This is like saying python programmers are not real programmers. If a python programmer can implement the same program in python, who cares what language they use.

I’d love for a better definition of sentience than “it works differently than humans”.