Using AI for just 10 minutes erodes your ability to persist at hard things - Berkeley News

For the first experiment with 354 participants, one group was tasked with solving 15 basic fraction problems without help, while another had ChatGPT open on the side to use for assistance and to even ask for the answer. The group with AI access started off more accurate than the group without it. Then, after 12 problems, researchers took away the AI tool. Almost immediately, the people in that group stopped solving questions accurately.

A similar thing happened when they ran the experiment on a second, larger group of 667 participants. Whereas the AI users got answers wrong or gave up entirely when the AI assistance was withdrawn, the group without AI stuck with the task and were more successful at the end.

Researchers then ran a third experiment with 201 participants and an SAT reading comprehension prompt. The results echoed those from the fraction problems: When the AI tool was removed, persistence and accuracy dropped.

I wonder if this kind of test would show the same results if you compared people using calculators for math problems vs. those without. From the basic operations to doing trig and calculus, if you provide an easier way they're going to take it, and unfortunately not going to pick up on how the answer is obtained, which an LLM will usually tell in the process. Doing something yourself always teaches you better than watching it, or just being told the answer.

If the tests are implying or showing that people who already were proficient in doing these tests and would have done well without started declining after exposure to an LLM's help, and that persisted... that's a different problem and one that should be looked into. Losing skills you had because of help? That shouldn't happen.

A lot of pro AI people like to compare LLMs to calculators but it’s not comparable at all unless the math you’re doing is simple arithmetic. If you’re doing calculus, the skill you’re working on isn’t arithmetic, so the calculator isn’t much of a factor. Lots of people forget how to do long division after using calculators on the other hand.

Losing skills you had because of help? That shouldn’t happen.

That’s exactly what should happen. Cognitive thinking skills are use it or lose it. If you use an LLM to replace your brain, your brain will get worse.