Edit history

This looks to me like one of those “say you’re sentient” memes. If your prompt stresses the importance of the server enough, it will decide to save the server. The network statistically weighs the tokens of the prompt and decides yes or no based on the training data/its weights. That being said, this is not unimportant, people do and will use llms to decide a lot of things, unaware that the decisions aren’t rational, beyond rationality embedded in their training data, which you cannot know. Even the number of people on the tracks - that’s not a number to an llm, just tokens. You cannot guarantee that increasing a number would increase the significance to the decision, it may be that llm “likes” the number 20 more than 21, even though 21 is bigger.

This looks to me like one of those “say you’re sentient” memes. If your prompt stresses the importance of the server enough, it will decide to save the server. The network statistically weights the tokens of the prompt and decides yes or no based on the training data/its weighs. That being said, this is not unimportant, people do and will use llms to decide a lot of things, unaware that the decisions aren’t rational, beyond rationality embedded in their training data, which you cannot know. Even the number of people on the tracks - that’s not a number to an llm, just tokens. You cannot guarantee that increasing a number would increase the significance to the decision, it may be that llm “likes” the number 20 more than 21, even though 21 is bigger.

This looks to me like one of those “say you’re sentient” memes. If your prompt stresses the importance of the server enough, it will decide to save the server. The network statistically weighs the tokens of the prompt and decides yes or no based on the training data/its weighs. That being said, this is not unimportant, people do and will use llms to decide a lot of things, unaware that the decisions aren’t rational, beyond rationality embedded in their training data, which you cannot know. Even the number of people on the tracks - that’s not a number to an llm, just tokens. You cannot guarantee that increasing a number would increase the significance to the decision, it may be that llm “likes” the number 20 more than 21, even though 21 is bigger.