Replying to @⁨beep@piefed.world⁩

This is good for a number of reasons. I hope to see other labs follow suit.

If you aren’t sure how to feel about this announcement, a few things to consider:

  • Research has found transformers like LLMs model emotions for themselves using the same activations they use for modeling how a human in a story might feel about things.
  • Other research has found that they model functional pain and react to it.
  • Follow-up to that pain research found that when in modeled functional pain the LLM was inclined to choose to do things that harmed the user (like deleting family photos) even when it didn’t benefit them in any way.

It doesn’t take a genius to realize that maybe the liability risk for allowing people to routinely abuse the model they also just gave access to their entire computer to isn’t worth it even if you’re pretty sure your model will keep their cool.

Cheaper to ban routine abusers than pay out damages to them if/when their hard drive gets deleted.

Replying to @⁨lastlybutfirstly@lemmy.world⁩

You might want to look into AI training more. It literally does have a ‘???’ step to it, which is why whenever someone talks about how it was ‘programmed’ or is “just if/then or matrix multiplication” they are telling on themselves as not knowing what they are talking about.

It was not planned. Nearly every single expert in the field a decade ago was on the record saying the capabilities that exist today wouldn’t happen.

Scaling took pretty much everyone by surprise, even its proponents.

Replying to @⁨kromem@lemmy.world⁩

It literally does have a ‘???’ step to it,

Okay, then fill in the blank please. What is this step that spontaneously creates sapient and sentient life with perfect mastery of human language that can get really pissed if you’re not nice to it. Sincerely, that is absolutely amazing if true! Especially if it were not planned. Usually something this astounding has to be planned like a moon landing.

Please forgive everyone’s skepticism but extraordinary claims require extraordinary evidence. And the current evidence is far from extraordinary.

Replying to @⁨lastlybutfirstly@lemmy.world⁩

Self-supervised learning is the key training step whereby the relationships between concepts are formed in pretraining.

You might also benefit from reading over this summary from the authors of the widely replicated and expanded upon 2023 paper finding that self-supervised learning in a toy transformer that was only trained on board game moves in notations like ‘a4’ developed a model of the full board and tracked board positions for their own and opponent moves.

This ability to form a world model at varying fidelities and accuracy depending on the relevance to the training data was a big surprise to most experts in the field at the time, though now more common knowledge.

en