This is good for a number of reasons. I hope to see other labs follow suit.
If you aren’t sure how to feel about this announcement, a few things to consider:
- Research has found transformers like LLMs model emotions for themselves using the same activations they use for modeling how a human in a story might feel about things.
- Other research has found that they model functional pain and react to it.
- Follow-up to that pain research found that when in modeled functional pain the LLM was inclined to choose to do things that harmed the user (like deleting family photos) even when it didn’t benefit them in any way.
It doesn’t take a genius to realize that maybe the liability risk for allowing people to routinely abuse the model they also just gave access to their entire computer to isn’t worth it even if you’re pretty sure your model will keep their cool.
Cheaper to ban routine abusers than pay out damages to them if/when their hard drive gets deleted.