For the first experiment with 354 participants, one group was tasked with solving 15 basic fraction problems without help, while another had ChatGPT open on the side to use for assistance and to even ask for the answer. The group with AI access started off more accurate than the group without it. Then, after 12 problems, researchers took away the AI tool. Almost immediately, the people in that group stopped solving questions accurately.

A similar thing happened when they ran the experiment on a second, larger group of 667 participants. Whereas the AI users got answers wrong or gave up entirely when the AI assistance was withdrawn, the group without AI stuck with the task and were more successful at the end.

Researchers then ran a third experiment with 201 participants and an SAT reading comprehension prompt. The results echoed those from the fraction problems: When the AI tool was removed, persistence and accuracy dropped.

I wonder if this kind of test would show the same results if you compared people using calculators for math problems vs. those without. From the basic operations to doing trig and calculus, if you provide an easier way they're going to take it, and unfortunately not going to pick up on how the answer is obtained, which an LLM will usually tell in the process. Doing something yourself always teaches you better than watching it, or just being told the answer.

If the tests are implying or showing that people who already were proficient in doing these tests and would have done well without started declining after exposure to an LLM's help, and that persisted... that's a different problem and one that should be looked into. Losing skills you had because of help? That shouldn't happen.

A lot of pro AI people like to compare LLMs to calculators but it’s not comparable at all unless the math you’re doing is simple arithmetic. If you’re doing calculus, the skill you’re working on isn’t arithmetic, so the calculator isn’t much of a factor. Lots of people forget how to do long division after using calculators on the other hand.

Losing skills you had because of help? That shouldn’t happen.

That’s exactly what should happen. Cognitive thinking skills are use it or lose it. If you use an LLM to replace your brain, your brain will get worse.

Over time yes, but this says skills were lost during a single session or short term. It was a random group of people, so unknown on what skills they did have before. I think what they observed wasn't a permanent loss, but a placebo effect of having extra help and it was taken away, so for the rest of the test they felt lacking.

My example of calculators was implying one that could do the test work. All calculators have trig functions, and some have calculus abilities. Would a student forget how to find cosine if the calculator was taken away, or do a derivative of a function? Not in one test. But if they were using it first half and then had to do it themselves for the rest, I imagine there's some psychological stuff going on that makes you feel like you've lost that skill.

They were tested on fractions

Did you use an LLM to summarize the study for you or something?

And reading comprehension.

We're discussing the effects of the test, not what the test was.

The content of the test is relevant to the outcome

Tech evangelists rely heavily on logical fallacies. False equivalence is one of the main ones. “A thing happened before therefore this tech is nothing to worry about”. They’re redirecting the argument to bait people to try to prove why calculators are bad. But of course the past already happened and nobody is going prove otherwise.

but it’s not comparable at all unless the math you’re doing is simple arithmetic.

That’s what they were studying on the math side

For the first experiment with 354 participants, one group was tasked with solving 15 basic fraction problems

Also

If you’re doing calculus, the skill you’re working on isn’t arithmetic, so the calculator isn’t much of a factor.

Calculators would be a big factor in solving calculus problems. Haven’t done calculus in a while but if I recall correctly you are breaking down or rearranging the problem into smaller steps that are more basic arithmetic. One part of a calculus problem could be finding out what the square root of 7 is , if you have a calculator to do that you’re more likely to solve the problem more accurately and to not give up on the problem, which were the two metrics this study was looking at.

Realistically, you would just write root 7 as a coefficient of your answer if you’re performing this math long hand. No one really expects anyone to be able to calculate things like that.

It’s funny because I actually think learning math the old fashioned way is super important. I used to struggle with math a lot as I’d always had calculators. I went back as an adult and got some old 1980s math textbooks based on Soviet pedagogy (widely considered the best for math at the time) and the impact has been huge. I’ve been building from the ground up math skills that got entirely skipped in school, and feel much better about it now. Calculators are useful but when LEARNING math they should not be used imo. Except maybe to check an answer after you already get it. Doing difficult things is good.

Doing something yourself always teaches you better than watching it, or just being told the answer.

A neat result from pedagogical studies is that being given the worked solutions to new problems to study from is much more effective for learning the material than simply practicing.

Students who were assigned practice problems performed worse on later tests than those given the same problems and all of the worked-out solutions for them. The explanation from the research was - when learning a new topic, immediate practice isn’t beneficial because the student doesn’t yet know what to practice. If they are lost when trying the problem, they can easily end up practicing the wrong skills by not knowing where to start. Those with the solutions have some of the cognitive burden removed, and are better able to internalize the task and generalize it to later problems.

I completely agree with you on LLMs and I think they’ll be a disaster for education. But I’ve always found it interesting that being given the answer can be useful for learning in certain circumstances.

Immediate practice on something you haven't been walked through yet? I guess the way I phrased it made it sound like jumping in yourself with no guide is better. You definitely need to be taught first, or at least as you go, and not figure it out on your own. The point was that simply watching someone do something isn't enough, you have to develop the process of getting to that point yourself. But only after seeing that route, otherwise you will take lots of wrong paths.

There are different ways to learn, some work better for certain people than others.

I would think that the best way to learn is neither of these options, but to be taught how to do the work by someone who understands (like a teacher) and then made to do it after it has been thoroughly explained to you and practiced with you.

That’s what works best for me anyhow, that or literally reading the textbook, lol, but that’s less efficient time-wise.

That was one of my gripes with my experience in college vs hs. In my hs at least, we were taught the most difficult combination of a problem and then everything that we practiced would be able to be worked out from there. In college, we were often taught the simplest example and somehow expected to derive the most difficult version of it on the problem sets. That’s how you get students who study really hard for an exam and half of them fail. They never taught the proper way to solve the problem!

I was out of college by the time LLMs came out but I still believe they could be helpful in this scenario if fine tuned and limited as a tutor not an answer-giver. Obviously if you have access to a good human tutor (who doesn’t call you dumb like my friend had at the time), but I didn’t have access to one.

A neat result from pedagogical studies is that being given the worked solutions to new problems to study from is much more effective for learning the material than simply practicing.

Finally, something to justify my still-existent hatred of my middle school geometry teacher for literally telling us ‘the theorems are on the board’ any time we asked her for help trying to solve a problem or for her to show us an example problem. Why the fuck were you even teaching.

Calculators don’t help with reading comprehension (3rd Test)

Also, they’re not talking about trig and calculus, they’re talking about fractions

You know (hopefully), like 1/5 =0.2? Or 6/18=1/3=0.333?

Yes, I'm aware of what they tested and what fractions are...

I was just expanding the point to other things that are comparable. And an LLM for a reading comprehension test is exactly what a large language model would be best suited for, so in essence a language "calculator".

Doesn't change the rest of the discussion, which was about the brain's ability to jump back into things when the crutch is taken away.

I don’t think it’s fair to say that fractions are similar to trigonometry and calculus aside from the fact that they’re all types of math

Very few people do trigonometry or calculus in their heads tho, I’m sure most people who know it well could do a cosine on paper, but that’s a much higher level operation than a fraction

Um, you know all of those were developed before calculators, right?

I learned both trig and calc on the principles in how they were derived, so yes, when I was learning and using them, I could do some of it in my head or with a few figures on paper.

But I never said "similar" at all, I was using them as comparable subjects that could be done by hand or with aid.

You're so obsessed with calling out someone on not mentioning fractions you've missed the point of the test, article, and this thread. They use fractions because it was above simple math but not so high level that some of the random people picked might not know it. Which going back to the real discussion, having randoms whose skill isn't known even halfway through the test might have been a flaw in the test.

Um, you know all of those were developed before calculators, right?

What does the timing of the advent of the calculator have to do with what I said?

But I never said “similar” at all, I was using them as comparable subjects

I would contest that they are comparable subjects. Entry level piano lessons and music theory are both technically of the same subject, but wildly different in practice. One is suitable for beginners or lay people, the other is not.

You’re so obsessed with calling out someone on not mentioning fractions you’ve missed the point of the test, article, and this thread.

You are projecting your own obsession of using the poor comparison of “AI = Calculator” onto me. That’s why I keep bringing up fractions, it is an easy-to-understand (for most) rebuttal to your logically fallacious proposal.

They use fractions because it was above simple math but not so high level that some of the random people picked might not know it.

Exactly, that’s why your comparison doesn’t work.

Which going back to the real discussion, having randoms whose skill isn’t known even halfway through the test might have been a flaw in the test.

What changes would you make to the experiment to remove the flaw you’re perceiving?

How do you expect that change would impact the outcome?

As I've said before, the abilities of the ones with ChatGPT assist don't seem to be known, as they started off with help. The assumption is that they might on average be the same as the others, but we don't know that. A simple fix to start is to offer the LLM help in the middle of the test for them, see how the curve of ability goes up and down, and compare before and after.

Not answering the rest, I was more interested in the study and what it was claiming and not arguing about missing context of commentary. I think the study may be right in suggesting people today have more trouble understanding things than they used to, but it's far deeper and older than LLMs. They are just the latest part of the bigger problem.

I don’t think that change would negatively impact the experiment, but for what they were trying to measure I’m unsure of how necessary that step would be.

They weren’t measuring how smart or good at math the participants were, they were specifically measuring changes in focus and persistence (so basically the amount of effort put forward). The findings show that people tend to just give up when you take the LLM away, and I think that with your proposed change you’d still get largely the same findings.

I would agree that LLMs aren’t the entire issue, but they are making the issue far worse at an exponential rate.

I’d be interested to read a study about what factors and/or traits predispose someone to trust an AI, or allow themselves to become dependent on it.

They weren't measuring the math ability levels, but the changes in what those skills were per individual. But for the assisted ones, we don't know what they started with, so there's no base level to compare. If they're comparing them with the others, then that's a problem in the study since we're not measuring their math abilities.

What would a companion study reveal that does the same thing for the math part but just offers calculators instead? Probably the same? If it was dramatically different then there'd be a case that LLMs are affecting the result in some other way and it's not just the human nature of leaning on what they lost. Or what if that person had a tutor/help with them but they left for the rest of the test? Should be similar results, right?

What would a companion study reveal that does the same thing for the math part but just offers calculators instead?

I think that depends on the difficulty of the math in question. If it was something that most people don’t know how to do by hand (like your trigonometry example) you’d probably see a more severe drop off.

I believe they chose fractions because it’s something that everyone who’s gone through the public education system should know, and they’re more relevant to (most people’s) lives than trigonometry (people use them to split bills, calculate tip, etc.)

If you got a group of people who all should know trigonometry (due to their profession perhaps), and then ran the calculator test, it would be more analogous.

What would a companion study reveal that does the same thing for the math part but just offers calculators instead? Probably the same?

That would not be my hypothesis personally, but perhaps I’m biased, I intentionally do most of my basic math mentally and only use the calculator for more complicated equations. If I was in this study I would want to see how much of it I could do without the calculator first… perhaps for the study the calculator use would be compulsory, but I wouldn’t just give up when it was taken away if I didn’t need it in the first place.

Or what if that person had a tutor/help with them but they left for the rest of the test? Should be similar results, right?

I think that would depend on the knowledge the subject had before entering the testing scenario, and also on the quality of the tutor in question. Personally, I don’t always approach things in the conventional way, so a tutor could be more of a hindrance than a help if they disliked my methodology.

I think that would depend on the knowledge the subject had before entering the testing scenario

That was my main point on this test being an interesting question starter, but not being conclusive enough because we don't know. The subjects were purposefully random and not evaluated before the test.

fractions because it’s something that everyone who’s gone through the public education system should know

Yes, well. I don't think I need to comment on how bad the US system has become. I wonder how much higher the assisted people did while they had the help compared to others?

I don’t think I need to comment on how bad the US system has become

No, I think we’re in full agreement on that point, lol

Some kind of comparison group would have made the result more informative.

Replace this with math and calculators. Similar results? Then it’s not AI’s fault, but a known phenomenon.

Of course, they would never even get the funding for this “experiment” if they didn’t have AI fearmongering in the title.

You can’t do fractions without a calculator?

I assume you’re also a fan of AI then?

A calculator doesn’t stroke your ego and you know that. That’s why you like and use AI.

Of course, they would never even get the funding for this “experiment” if they didn’t have AI fearmongering in the title.

In what world are you living in where the people with money are against “AI”?

Now you need a human control. Provide a human assistant to do the same.

Is this a problem of AI or a problem of dependency.

Cause from my understanding of the topic this same phenomenon happens when people are given too much assistance of any kind form any source.

Not many people will actively drive themselves if given a solution with low barriers.

It’s such a common thing it’s been a trope among noble classes though out the ages. If you coddle someone they go full potato.

LLMs are a competency chipper and credibility solvent.

My observation from having to deal with entities which have engaged with LLMs, is any competency or credibility they may have had, is eroded proportionally to their level of exposure. It’s analogous to experiencing a traumatic brain injury.

Losing cognitive ability is bad enough, but it’s devastating when they never possessed it to begin with.

Deferring all your work to anything or anyone would do the same thing.

That’s not a llm unique problem but a byproduct of not engaging your mind or skills. You can still do that while also using an llm or taking on human help.

If you hired a human to do everything for you, you also end up becoming a potato.

The problem is that llms make it too easy for people to actively choose that route because of lack of self discipline or desire to improve.

But a lot of technology is like that and not just computer tech.

On the other hand I have seen llms write better emails (with better understanding of the subject) than less competent colleagues. Some people are already very intellectually challenged to begin with and were able to hide it well.

They would be in the devastating category.

Yeah they basically disappear behind their new tool, as the only contribution you now see is the one coming from the plagiarism machine.

LLMs give people ADHD?

That’s not what ADHD is

News about LLMs gives people a false understanding about ADHD?

Their false understanding was likely there before they read the article.

Impossible.

It sounds like it’s messing with the reward center of the brain which is what ADHD is all about.

ADHD is a neurodevelopmental disorder, it’s not something you can gain, it’s a defect that happens from birth or during development. It’s a physical difference in the brain.

You can simulate “ADHD brain” with drugs and conditioning, similar to how PTSD alters people’s brain chemistry.

The human brain is plastic/adaptive and can change, depending on the cost/benefit of action/reward conditioning. Just like how people with ADHD can see some improvement with training, a neurotypical brain can see some ADHD-like changes with the same sort of (anti?) training. That’s what I’m suggesting is happening in this AI study: People were rewarded for using the AI initially (dopamine response) but then that reward (easy way out) was taken away, and they performed more poorly as a result.

It stands to reason that their reward center was trained to behave like an ADHD brain in that it could tell there was not going to be an easy way to gain a reward from the activity, so it didn’t bother putting in the necessary resources to achieve the outcome it would have accomplished if they were never given AI in the first place.

The usual dopamine response was overridden, thus behaving like an ADHD brain.

ADHD would be better described as “executive dysfunction”, essentially the issue is not that people with ADHD cannot pay attention, it’s that they have difficulty controlling where their attention is focused. People with ADHD are actually better at focusing than neurotypical people, they just can’t direct that focus, and it is captured by whatever the person with ADHD finds most interesting in a given moment.

ADHD does not negatively impact intelligence like AI use does, rather it impacts executive function.

If you gave these AI brainrot victims the medication commonly used to treat ADHD, it would not assuage the effects that the AI has had on their cognitive abilities.

I feel like they could have (and probably did) say the same think about search engines 25 years ago.

Though this does seem a bit worse, depending on how you use it.

Yes, I was going to ask, how does this compare to with/without internet access. Coders were also likely more persistent when they could only read the tool’s manual and keep trying, vs getting the answer from StackOverflow immediately.
I’ve heard there’s been a decline in memorization skils since the internet, but I have not verified.

getting the answer from StackOverflow immediately.

StackOverflow didn’t have immediate responses and the responses were *how to do the thing*, not literally writing the code for you. The internet and support communities were like reference books and asking a colleague for advice, the normal way people *learn*.

LLMs are like having someone else do it for you. You have to actively try to lean something from how it did it and most people don’t, they just take the answer and move on.

Stackoverflow definitely had code you could just copy and that would be the end of it. Not always though.

Dictionaries have definitions you can just use!

You act as if people never just copied and pasted stackoverflow without reading it lol (we’ve all done it)

And code that you didn’t write is someone else doing it for you.

And if you don’t believe me, I have a bash file you can curl | bash.

I have zero patience with coding to begin with and this is the first time I can even start to learn anything.

Imagine a help function that actually does help.

We learned how to use the library pre-search engines. Libraries were stacked with text books. Also, education was cheaper, so you could simply learn to code in a classroom.

I don’t agree with this study and I am not a fan of AI. Replace the AI tool that they used with a calculator. You’d get the same results. Here you are legally cheating on some task because you were told to do it - breezing right along thinking of something cool you are gonna do later and then … WHAM ! Cheaty tool gets yoinked. Now fuck - I have to think ? I’d probably think fuck it, I’ll give it a half asssed shot because fuck them for making it not fun.

Yea, if you compare the post-AI test group data with the initial control group data before they had practiced, the AI test group literally did about the same as the non AI group. And you are very correct about the fact that yoinking the easy solving tool makes skipping jump because why would you do a thing that feels pointless and is harder then it was before? I hate the enviornmental and technofascist implications of AI, but people need to stop trying to make brain degradation from AI a thing, it is never going to be a thing. You just don’t really learn from it if you don’t want to y’all, like everything else. It’s not that fucking complicated. This is just a technology witch hunt.

I am begging people to look at the ACTUAL studies and analyze the variables and results critically.

While I think studies like this are bunk, I think it’s a mistake to assume that there is no real effect on cognitive health.

We know that using LLMs is addictive. I can’t imagine that long term use is going to be as benign as long term use of a calculator.

It’s not going to be benign because of how much the AI can do. I caught someone using a calculator to figure out what 15% of 7000 was the other day. They haven’t calculated anything in their head in a decade so simple mathematics have become a chore.

Add that concept to basically 50% of tasks we do on a computer and the future starts looking grim.

AI is basically the devil whispering in your ear that you can be as lazy as you want, I for one find it harder and harder to resist. Told myself I’d code by hand a few hours a week to keep up my skills but that went about as far as going to the gym every week. My monkey brain loves taking the easiest path.

This is what they say about people not learning assembly and jumping straight into C++ or gasp Python

I mean you can see the same thing all the time with caudled kids. Kids who grow up with a bunch of help and servants and waited on if they aren’t pushed they just end up helpless.

This isn’t an LLM problem. It’s just a known issue with dependency. It’s been a problem as far back as history goes.

It’s just before it was a problem that mostly the rich were allowed to have. And now it’s cheap enough that everyone gets to have it!

There’s a huge difference between someone using agentic ai to solve real problems quickly and someone asking a chatbot for life advice. A lot of people are asking llms for life advice, and don’t get me wrong it’s a great research tool IF you already posses critical thinking skills and know when to question it, but schools are introducing generative ai to middle schoolers before they even gain any critical thinking skills.

I agree, my main issue is people treating AI as some pseudo psychological evil. It’s just an answer box. People who have issues from it already needed help. If AI didn’t make them worse, a forum site would’ve.

I think the bigger problem is a potential for a loss of morale. We’re getting to the point where engineers can do more with less effort but it’s also less rewarding. I haven’t been this sad about doing my job in 8 years.

Anecdotally, I have software I wrote entirely by hand, started by hand and ended with ai, and AI from the start. Anything I had ai touch in the last 6 months has caused this for me. I barely have the attention span to read what the agent is planning to do. I can only conceptually review at a high level and direct it that way. It gets worse every week.

I think part of it is me not spending enough time setting things up so it writes code like I would, so I am reading a strangers code really. But even then, I find myself reading it for no more than 10 seconds and being like “ugh just ask the AI”.

I have caught myself a few times about to ask the AI to change something that I could easily do myself still.

I know that if I stopped using it for a week and sat down with the code I had generated, I would understand it, but the thought of doing so has somehow become exhausting. It’s a real issue.

It sounds like you enjoy coding more then managing.

Not enjoying something is going to lead to no attention span related to that thing after all.

Which is a pet theory why I think so many software guys and long time Linux users say this.

Fundamentally it’s just not fun. It’s not interesting. Even if the AI can produce perfectly fine and working code. It’s not great cose, but it’s also not buggy in a way that gives you challenges to solves problems to understand.

Ai is just mundane. It removes the enjoyment form computers in exchange for mediocrity and functionality. It works “well enough”. And that is more of a sin then anything else when it comes to long term engagement.

Personally I can stare at a prompt for 8 hours straight and debug and improve the harness, the system, the thing it makes, manage it, fiddle with its memory set up, make entire new workflows or systems for it.

The actual product it makes tho? Boring and I tend to forget about them a week later.

The process is boring as fuck to most people AND what it makes is boring.

So either you need to find the process fun, or make something with it you actually personally use day to day.

I had the same problem with 3d printers. They are cool as fuck. But the stuff I made was… Meh cause it wasn’t useful I just made it cause I could.

The only reasonable take in this thread is getting downvoted. Lemmy is so over.

the calculator doesn’t get 1+1 wrong…

AI can and will be wrong and then tell you it’s right and that youre wrong and stupid for questioning it. you

AI doesn’t get 1 + 1 wrong anymore… but yes it can still be wrong. Most commonly if you ask it the wrong way. A calculator can do the same if you forget the parentheses, or mess them up slightly.
My feel is that the degree to which a wrongly inputed question can get a wrong answer is proportional to the complexity of the question you are trying to get an answer to. Kinda like computers made it possible for a simple mispelling of someone’s name to have much more massive consequences than before computers.

AI does get things wrong, but it owns up to it when pointed out, and often self corrects once it has observed that something is wrong with the work. Your characterization is straight-up wrong.

“The bcrypt hooks caught only GOG web traffic (blob_007 is literally plaintext GET /token?.. HTTP) — the save crypto never touched bcrypt.dll, confirming it’s done inside the exe by statically-linked OpenSSL. That also means the ChaCha state I scanned is likely TLS traffic, not the save. Two parallel paths now: brute-force the record layout against AES-GCM (OpenSSL’s default AEAD), and locate the AES tables in the exe to find the cipher code:

No AES tables at all — the game uses hardware AES-NI, so table-xrefs are out (and that also means my earlier AES-schedule scan is the right detector… except it never finished; that was the hang). Let me nail the layout facts offline and check my tooling:”

a computer doesn’t ‘own up’ to anything.

This is pathetic. I have been working with an AI, and there are many bugs and oversights that it acknowledges and addresses, ultimately creating a final product. Frankly, you are in denial.

If I had a choice of interns to hire and you were one of them, I would have ruled out hiring you. This is because your lack of objective reality would undermine any project, be it purely human or involving AI.

Any series of events where something makes an error and then though a process of trial and error or reasoning from facts to arrive at a correct destination. Then accurately state the error, the reason why it was made then the steps to correct it. Is “owning up”.

Your giving moral weight and ethics to a phrase used in a way that was not intended to carry any in context.

At best your being stubborn, at worse your trolling.

We can all agree the dumb slop machines aren’t people and anthropomorphicizing them is dumb. But take the statement in the way he clearly intended and he’s not wrong.

Models are at the point they will noticed they are wrong and self correct 9 times out of 10. If given the resources, tools, knowledge to know it’s a error and see it.

The bigger problem is bad user input. Ask a stupid question get a stupid result. Even if the best case is a mediocre result. Thats still good enough .

Replace the AI tool that they used with a calculator. You’d get the same results.

Then go do the study and publish it. Should be simple as shit.

Replace the AI tool with a calculator. You’d get the same results.

Correct. I used to be able to calculate square roots in my head, then they let us use calculators the next school year, and soon I was no longer able to do so.

The problem is that what calculators due to math LLMs do to thinking in general, causing much deeper and extensive damage to the brain.

Also, calculators are good at maths. They produce correct results.

LLMs aren’t good at thinking (or even capable of doing it at all), they’re very good at putting the most statistically likely (given the texts they were trained on) word in front of a sequence of words. They don’t necessarily produce correct results, they produce results that wouldn’t look out of place in their training set (unless you’re an expert in whatever the text is about, in which case it’s often absolute bullshit that sounds correct to a layman).

We’re replacing working brains with autocomplete engines that (if they were capable of caring about anything, which they’re not) couldn’t care less about the correctness of their output, just that it looks correct.

And it remains to be seen if the people injured by this will ever be able to recover anything close to normal brain function. Personally, I doubt it. Once it’s gone, it’s gone.

Why have our oligarchs or billionaires devised systems and work cultures of difficult tasks and productive struggle, when our government’s documents claim we, all of us, have a right to pursue happiness?

Researchers recently found that spending just 10 minutes relying on an AI tool significantly impairs the user’s ability to focus and persist with difficult tasks…

…

The findings backed up what anecdotes and surveys have increasingly found in recent years: An overreliance on AI for answers erodes our cognitive fitness. When AI produces answers in seconds, we get used to that speed; anything that takes longer feels inefficient. And relying on AI also removes what the team called “productive struggle,” which can lead people to find their research strengths and passions.

Jokes on you. I never persisted at doing hard things because I had no idea what I was doing. Now I get into all sorts of shenanigans because I do what ai tells me to do.

Pretty much my situation. My local AI has whipped up some compatibility enhancements for a 2006 game, and about four save editors. Right now, I got it researching The Oregon Trail 2021’s character sheets for editing.

When a Qwen4 Flash is released, I will be looking in recompiling some Windows 3.1 games.

Shenanigans.

That’s not 37 pieces of flare!

I don’t know the details of this study but it sounds like a basic result if you get people to rely on something for most of an activity and then you take it away vs those who are doing it manually, they’ve been given more progressively difficult challenges but they’ve been actually practicing and doing the work and hence internalizing it better.

It’s like those open versus closed book exams where those who actually study can do much better than those going through a book during the exam to figure it out. It’s about internalizing and practicing a material.

Not really surprising results. I guess the empirical verification is good.

Not doing fun Maths exercises often enough erodes my desire to do start fun Math exercises. I don’t use AI tho

Driving a car for just 10 minutes erodes your ability to walk to the grocery store.

funnily enough over reliance on cars CAN cause long term health issues if you really are using them for stuff like a trip to the store in the corner, along with all the increased and more fatal accidents and destruction of accesibility that comes with car dependence, so you’re literally proving the (admittedly sensationalist) headline correct

Wearing shoes for 10 minutes erodes your ability to walk to the store barefoot.

LLMs are useful for branching into certain areas I am unfamiliar with or that have a lot of content. I’ve got lots of regulations and white papers for my work, and it’s helpful to point me in the right direction very quickly or assist in coming up with topics/ideas when I have very short deadlines on projects. Recycling (plagiarism) of company products is actually promoted. We also have very corporate lingo and it helps me rephrase my content in that style. Basically it helps me do stuff that would probably otherwise be too hard or impossible for me to do in the timeframe I have, so kind of the opposite of the study? At least in the way I use it. But the study results kinda line up with the other studies talking about IQ decline in newer generations for the first time in history, if you buy that.

So basically its a tool for checking corporate checklists instead of actually doing good things?

There’s a flip side to this. I am persistent. But busy, and a realist. Since AI, I have been able to accomplish things I always wanted and planned to, but never got done before. I’ve been wanting a contractor’s license to expand my business. But I’m busy. AI is the tutor that helped me get it done. I’ve been loving Linux and open source, had a handful of raspberry Pi projects, but was never quite confident enough to go full on self hosted. Thanks to AI, I am fully self hosted and loving it. Also, I own a service business, time is money. Guess who has more time since AI? Define persistence in the context of my experience with AI. Put in other synonymous terms, am I faltering, yielding, “going fallow”, atrophying? I don’t think so.

Oh shut up.

Using a car erodes my ability to persist at using a horse and buggy. Using a powerdrill erodes my ability persist at using a screwdriver. Using a microwave erodes my ability to eat cold food.

Speaking as someone who uses all kinds of technology, AI included, I welcome the erosion.

It just makes me solve harder problems…

We all knew this was going to be the case. You look at the people around you feeding their entire lives to it, surrendering their will to it completely, you knew it was going to fry their brains extra fast.

But if I want something very specific out of it and don’t tell it my life story it’s going to be beneficial for me.

The idiots use it like google, have angry letters written by it citing things that don’t exist etc.

I am under no illusion that this is more than autocomplete, a summarizer a coder debugger etc.

Using a lighter for 10 minutes erodes your ability to persist at starting a fire with two sticks to :)

Truth is in the middle somewhere. I can still fix/make the things I made 2 years ago, but i’m going to use AI tools to do things that it’s passable at while the tokens cost me $20 a month.

My homelab went from bare minimum with not enough time to ansible jobs to deploy any product I want to play with in a week.

Yeah. I don’t reall like using AI but it has helped me a couple times with homelab and other PC related issues when even after 45+ minutes of searching the web I found nothing useful and barely anything pertaining to my issue at all.

Agree. With homelab being a hobby but time being a scarce resource; with AI I’m able to get past the things I hate that are mundane and that I would never find time to improve while still doing the things I enjoy with the hobby.

Also working in a job where there were never enough resources to do the tasks that were high priority, meant we also never improved on technical debt. Now that has turned: we are able to use AI assistance to deliver a higher throughput while fixing more tech debt. Just a different take than the “AI produces low quality code” position.

It’s not going away. It’s a tool. Learn to use it as a tool, not a replacement for your brain.

You’re a tool.

Just stop.

Y’all know it’s not going away. Y’all can bitch on lemmy but the billionaire class ain’t getting rid of it.

This article is a perfect example of how legitimate research gets turned into sensationalist nonsense, and the post on Lemmy is making it worse.

The study involved 1,222 participants across three experiments. Researchers had people use AI to help with math and reading comprehension, then tested them when that assistance was removed. They found that participants were more likely to struggle with accuracy and persistence when they had to work independently.

That’s an interesting finding. It’s worth discussing.

But this whole fixation on “just 10 minutes” is ridiculous. Ten minutes of AI use under specific experimental conditions does not mean that ten minutes of ChatGPT permanently damages your brain or destroys your ability to think for yourself. The study did not demonstrate that.

There’s a pretty significant difference between showing that people can become reliant on assistance and struggle when that assistance is suddenly taken away, and claiming that AI use fundamentally erodes your cognitive abilities. One is an experimental finding. The other is an exaggeration of what that finding actually means.

And this is where I have a problem with the article and the way it’s being presented on Lemmy. The actual research is interesting enough without having to dress it up in sensationalist headlines. If the point is that people need to struggle with difficult problems to develop their skills, then make that argument. It’s a perfectly reasonable one.

But “using AI for 10 minutes erodes your ability to persist at hard things” is a hell of a lot more dramatic than what the research actually establishes.

There’s a difference between reporting scientific findings and manufacturing a narrative around them. We should be capable of discussing the potential downsides of AI without resorting to clickbait and misrepresenting what the research actually says.

This is a really smart take on things.

Yah this should be pinned

Yah. Turtally.

“Students who cheat on their homework don’t learn as much” is a better headline.

TLDR: LLMs are better information aggregators than Google, which has become a mess of ads and SEO spam. Using AI to complete homework isn’t inherently cheating; homework is practice, not an exam. If schools want to measure independent knowledge, they should use supervised assessments instead of take-home assignments. And if an assignment offers little educational value, perhaps the assignment is the problem, not the tool students use to complete it.

I think the more important takeaway from this study is that people produce better results when querying LLMs than when using traditional search engines. To me, that points to a problem with conventional search more than it demonstrates a detriment to using LLMs.

Have you tried Googling anything in recent years? It’s a bloody mess. Google increasingly prioritizes what it wants you to see, and more importantly, what advertisers have paid it to show you. Finding genuinely useful information has become an exercise in digging through sponsored results, SEO spam, and pages that exist primarily to generate ad revenue.

LLMs, for all their faults, are arguably better information aggregators at this point. That isn’t necessarily a ringing endorsement of AI, considering that they aren’t particularly good at searching either. It just illustrates how bad conventional search has become.

I wouldn’t interpret using an LLM to complete a task as inherently equivalent to students cheating. I’d argue that LLMs are simply another tool for finding, processing, and synthesizing information. If students can produce better results using those tools, perhaps we should be asking why we’re so determined to prevent them from doing so rather than examining the tools themselves.

And frankly, you can’t really cheat on homework. It’s homework, not a supervised examination. Students should be able to complete assignments using whatever resources are available to them. If the objective is to determine what students can accomplish independently, then schools should assess that through supervised examinations rather than take-home assignments.

More importantly, there’s a broader question about whether we’re assigning too much homework in the first place. Research has repeatedly found that its educational benefits depend on factors such as age, workload, and assignment quality. If an assignment provides little educational value beyond producing a completed worksheet, then perhaps the problem isn’t that students are using AI to complete it. Perhaps the assignment itself isn’t worth doing.

Have LLMs become better than Google?

Or has Google just gotten shittier than LLMs?

My SearXNG instance never lies to me, and returns results better than Google used to before they fully enshitified

Have LLMs become better than Google?

Or has Google just gotten shittier than LLMs?

LLMs are better at most search tasks than Google has ever been. No contest, it is in no way close.

I disagree

Truth. There is a lot of obscure searches that would go nowhere if I tried to use old-school searching. AI is the way forward when it comes to collecting lots of info and giving a cliff notes. Even better, the AI can then go into detail.

AI is the glasses of the information world: it lets people focus on getting shit done, every day.

Kagi is able to reach into Japanese websites, aggregate their information, and translate a summary of the contents for me. I used it to learn a bit about an old SNES kusoge, called Saiya no Densetu. Apparently you can get Goku to arrive before the Ginyu Force fight, but paradoxically, he has to be very low level.

This is information that a traditional search engine can’t easily grasp and present.

They make browser extensions for translations, and my SearXNG can index Japanese sites. I use it to find and download obscure Japanese music albums and ROMs. AI is a very inefficient way to achieve that result.

Aren’t most search engines just LLMs anyway these days?

No, a lot of them have LLMs integrated into their search engine, but they themselves are not LLMs.

The search engine I use (SearXNG) is what’s called a meta-search-engine, meaning it runs my query through several different search engines (which I can toggle on or off at will), then parses the results from all of the queries engines, strips out all of the slop bullshit, and returns me actually relevant links.

I think if you wanted to you could attach an LLM to a self-hosted SearXNG instance, but I’m not putting LLM fuckware on my system. I prefer my computations to be deterministic.

Weak.

Eh. Go sit and spin.

Now say it without crying.

Imagine needing a computer to put your own thoughts into words for you.

You used an AI service to determine if my comment was written by AI, and the service itself admits it can’t reliably determine that. Do you see the irony here? You literally used AI to guess whether I used AI and then treated that guess as fact. That’s some next-level nonsense.

I will say this however in my own defense not that I have to, I use Google keyboard which has a built-in AI editor it helps me to write long responses quickly.

I will use any and all tools that I deem necessary to complete any and all tasks regardless of your idiotic criticism.

It is my position that a machine cannot reliably determine whether a string of characters that would look identical whether produced by machine or human was produced by either. While you could very well be correct, your method for making such determination is—in my view—severely flawed.

Edit: I generally dislike LLMs and other generative AI, just as I dislike the “AI detectors” that often use them in an attempt to determine whether any given text or image was produced by such. I personally know someone that would score high from having developed habit of writing in clear, generic style for work over many years.

It shows 100% for Frankenstein

Read the text of the post. Are you admitting you can’t distinguish LLM output when you see it?

Lol, offloading your lack of criticism of a point to an llm.

Seems you’ve been cooking for longer than 10 minutes…

“Using a compiler for ten minutes erodes your ability to write raw machine code”

This is… Unfortunately completely true. ☹️

Agreed, I tend to get more interested in topics and learn more than I expected, for new neural networks. I think it’s like any tool and using it right can be useful. Our brains can make new connections learning about different topics and how they are related, systems thinking and problem solving. We can search online and read the sources directly too, and verify more with increasing importance of the topic. For example, going to defend yourself in court, do a lot of research from quality sources and still reconsider lol. Just like large claims require large evidence, same idea for other resources including llms, and no need to avoid it like it’s some new Witch panic. The energy people use daily mindlessly absorbing video streams is more than text based llm use and follow up confirmations.

Learning new things and recognizing connections between topics is beneficial for the brain, as it strengthens neural pathways and enhances understanding. This process, known as neuroplasticity, allows the brain to adapt and improve its ability to learn over time

Forgive me, but i’m not understanding why you find the headline to sensationalize or over state the actual findings of the study. The abstract of the paper directly says:

we provide causal evidence for two key consequences of AI assistance: reduced persistence and impairment of unassisted performance.

I’m struggling to discern a meaningful difference between “erodes your ability to persist at hard things” and “reduced persistence of unassisted performance”. Forgive me for not understanding but I would appreciate your explanation. The paper’s very own conclusion actually uses the term erosion:

Here, we show that AI interaction can result in significant impairments in independent performance and persistence – capacities that are foundational to life-long learning. If brief exposure produces measurable erosion, the cumulative effects of daily AI use over months or years may be profound and difficult to reverse

I found a definition of erosion: “the gradual loss of something as a result of an ongoing process”. Trust can erode and be rebuilt; soil can erode and be restored.

I agree that 10 mins of foundation model assistance likely doesn’t “permanently damage your brain or destroy your ability to think for yourself”. However, in my reading of the Berkeley news piece, I find no claim of this specific kind. Does the text of the press release explicitly claim the duration of effect is long-term or lasting? Does it explicitly claim the the erosion is permanent? Again, I find myself confused what you are arguing against.

Of course I’ll be happy to explain, the title of the article states:

Using Al for just 10 minutes erodes your ability to persist at hard things - Berkeley News

The findings of the study do not indicate that using an LLM (AI) for ten minutes removes your ability to do hard things. The study showed that it was harder to perform tasks without AI assistance after having access to it. In fact, people who used AI for ten minutes before performing difficult tasks actually performed better than those who did not.

The ten minutes was simply used as a control so that everyone had the same amount of time with AI. It had nothing to do with the erosion of their ability to perform difficult tasks. The conclusion the researchers drew, which is up for interpretation, was that prolonged AI usage may potentially impair people’s ability to perform harder tasks without assistance.

The title heavily implies that just ten minutes of AI usage makes difficult tasks harder to perform, which is not what the study showed or concluded. Therefore, the title has been intentionally sensationalized as clickbait. It objectively misrepresents the findings by encouraging a different interpretation and conclusion than what the study actually demonstrated.

The study itself is interesting, but apparently, the researchers’ findings weren’t sensational enough on their own. The people posting these articles still felt the need to mislead their audiences into drawing a different conclusion from the one the research actually supports.

“The study doesn’t show it removes the ability to do hard things” Again, who exactly are you arguing against? Nobody said this… Neither the press release nor I

“People who used AI for ten minutes before performing difficult tasks actually performed better than those who did not.” I’d gently encourage a quick re-read of the paper, as the statistic results directly contradict your claim. Here’s experiment 1 (fraction math problems):
ALTimage

The solve rate measures the rate at which the participants correctly solved the problem, and the skip rate measures the rate at which the participants chose to skip the problem rather than inputting an answer. We observe that with AI assistance (i.e., the first 12 problems), participants in the AI condition were significantly more likely to solve problems correctly and less likely to give up compared to the control participants. However, when the AI was removed, their performance dropped and they were also less likely to persist with problems (as seen by the increased skip rate).

Here’s experiment 2 (same fraction problems, but with an added pretest): ALTimage Finally, experiment 3 (SAT-style reading comprehension): ALTimage To clarify, people who used AI did worse than the people who never used AI on the test.

“The title heavily implies that just ten minutes of AI usage makes difficult tasks harder to perform, which is not what the study showed or concluded” and Again, just to make sure we aren’t proceeding on a misunderstanding here, the headline says “persist”, which means to refusing to give up. The definition of persist is not that of perform. Further, I suggest you re-read the research paper. The abstract directly states, “impairment of unassisted performance”

“The conclusion the researchers drew, which is up for interpretation, was that prolonged AI usage may potentially impair people’s ability to perform harder tasks without assistance.” I think you may be confused about how experimental studies work in research. In the experiment, when they controlled the variable of AI assistance, the experimental results showed statistically significant results proving the hypothesis that test solve rate of the control group was greater than the test solve rate of the AI group.

The results of the study are not in question, nor am I arguing against them. I am criticizing the sensationalist title being used in this Lemmy post.

The study does not say that using ChatGPT for 10 minutes erodes your ability to perform hard tasks. You yourself acknowledged this in the information you provided from the study.

So I guess the question is, what exactly are you arguing against?

Once again, to reiterate, my complaint is about the misleading clickbait title.

The vast majority of people who see this post aren’t going to read the study. They’re going to look at the title, nod their heads, and move on. I’m simply trying to provide some context.

Seems like a biased study yea. Why not get someone to do something manually first without AI, then give them AI to see if they do it better or worse?

Yeah my first thought looking at the title was literally… But I’m doing an extremely hard task while the dumb slop machine is doing an entirely different task.

I’m not just sitting there waiting around not thinking.

So how is that causing me to lose the ability to persist at hard things.

The article after reading it raises some good points but that head line is nonsense.

Thank you! This makes me think maybe our brains reward themselves for finding easier ways to do something and LLMs become just that sometimes. But at the same time atrophying our existing skill. We need brain gyms like we do physical gyms.

I read a study that people used to focus on remembering information they learned, but given the opportunity (Internet, books) instead they focused on remembering the location of the information they learned so that they could access it later

The latter is a great approach for knowing where to find knowledge you don’t know. Like understanding how a library is organized makes finding something easier so if you haven’t previously learned about something you know where to find it. It is basically how research is done.

Knowing how to filter what is important and what isn’t is another skill that helps to apply that approach, and LLMs don’t provide anything that helps with that kind of filtering.

What a load of shite.

If I asked an intern to do something and then review their work is that eroding my ability to work?

The actual study does not make the claim this stupid “article” does.

My managers seeming plenty eroded rn 😅

Software devs becoming managers, who then lose the ability to reason about software dev is a good example of the anwer being "yes, sometimes".

As someone who has a disorder and is prone to hyper fixation, this isn’t a concern. The largest issue is that AI often claims they know how to do something, and it’ll get you about 80% the way there.

The at the 80% mark, it admits the thing you’re doing can’t be done with the software or tool it recommended.

I was going to argue against this, but i don’t feel like doing the research myself.

I’ll reddit your lemmy so good baby

That’s no tr

I used AI one time to find out what time it was in LA… now I can’t tell time Edit: it’s jokes. cmon lighting up a bit. Sheesh!

Using AI for just 10 minutes erodes your ability to persist at hard things

Did Claude give you erectile dysfunction?

Depends if you outsource your thinking or if you use it as a tool.

This is a human problem, not an AI problem.

‘a study’

How ironic is it that you couldn’t be assed to click one link and read two paragraphs to find said study?

arxiv.org/pdf/2604.04721

ya i mean a single study doesn’t mean anything, these result need to be vetted and replicated.

ITS CALLED SCIENCE

so until that happens your moronic opinions dont really matter.

stop spamming my inbox.

blocked.

lol, I hope he gets to touch some grass one day.

Yeah. People study shit like this.

Gonna ask your friend Claude how to feel about it?

wow you have anger issues.

Naw I just look down on people who would so willingly shed their agency to a fucking machine build by sociopaths.

you also make elementary assumptions and project them on strangers over the internet in a petulant manner.

Poor baby. Gonna cry that you clanker-lovers can’t fathom having to think, and do, things on your own?

Cry me a river.

dont need a new study, 1 study already said people using AI during HS or COllege makes them inability perform at thier grade education level.

Yeah, but reproducible results is a hallmark of a scientific peer review

Can you shorten your sentence please, oh wait, I’ll copy it to AI.

It is so sad that people always look for shorter means to a problem solution, I understand it is coded in our DNA to exhaust the least but over the long term it looks increasingly worrying.

You’re offloading your thought to an LLM there is no way you don’t come out a bit dumber.

I’ve been using LLMs with a project that I simply don’t have enough time for. I’ve built the core of it during most of last year. And in the past 6 months or so I’ve been using LLMs to help me out with some of the more tedious stuff. Half the time it doesn’t quite get it right, and some of that time it’s just flat out wrong. It’s usually when I get the LLM to try to hunt down some bug or something. After a couple tries, I give up on the LLM and figure out the problem myself.

I genuinely don’t believe this “offloading your thoughts to an LLM” idea is universal. For someone still in school and learning? Probably 100% true. But for someone established in a skill, I don’t think it’s true, at least not entirely.

I don’t use LLMs (especially now during nosloptober), but I’m also trying to apply this to web searching. Like instead of doing a search for “how many minutes in a month” I’ll realise I have all the tools and information to figure it out and just do that.

Or checking with a local manual before searching online

I remember the times before search engines. I used to have to wait for the weekends and walk 40 minutes to the library to look stuff up if I had questions. I remember how I even though encyclopedia britannica being available online was revolutionary and I spent hours reading just because I finally had access from home. It felt like it wouldn’t always be there, so I tried to read as much as possible so I would just know stuff. I’m kinda nostalgic for it and where I live now, the library is only a few blocks away. You’ve inspired me to try to recreate that a bit.

Also, correlation does not equal causation. So the title is attributing AI to eroding your ability to persist, when in reality there was no prior test to ensure the individual could do the task at all. Which is almost like what they’re proving is that AI gives people the ability to do things they otherwise would not be able to do with out it.

Giving someone a tool to start and then taking the tool away mid task isn’t a good way to study the impact of the tool.

Ironically, the people that agree with the headline won’t bother to critically think about the study.

when in reality there was no prior test to ensure the individual could do the task at all

Why would such a test be needed? I presume the researchers did the reasonable thing and assigned the participants to the two groups randomly.

but ironically most of my time using AI is for “funny” stuff

Surely if I offload work tasks onto someone else then I would achieve the same effect? I don’t think this is AI exclusive.

It depends on what you’re doing. I deal with lots of data entry and when there are corrections to be made, I had to scour through troves of information which could also cause data entry error when i am going through with it. In other words, it’s “secretary work”. AI had been helpful on that front to minimise the tedious work that does not add value or improve productivity.

AI is only good at secretary work. But for tasks that require critical thinking and analysis and creativity, don’t rely on AI.

Man getting people to listen to me when I essentially talk about the same thing you brought up is impossible. It is great for these kinds of things but I just get shouted down by all of the “AI bad no matter what” people.

I’ve come to realisation that since many Lemmy users are IT workers, they feel resentful in training AI to replace them. Many had been laid off after all since the (irrational exuberant) coming of AI. That’s why I don’t begrudge many Lemmings hating AI as something that could never do anything good. I’m lucky enough that my work isn’t going to be replaced by AI. But for how long?

For something that uses immeasurable amounts of stolen data to be trained on and function, without credit or compensation, and breaks so many laws I can’t list them, I think it’s pretty unethical to use. Or something that burns immeasurable amounts of potable water and electricity to function at all, I think its pretty unethical. For something that artificially generates scarcity for chips we desperately need, I think it’s unethical. For something that authoritarian governments are lapping up as a form of propaganda, disinformation, misinformation, surveillance, I think its pretty damn unethical. All so people can write an email 10 seconds faster, in a tone that’s not yours, to be sent to a AI to answer that email. For the people who say “I think it can be used for good” or “it’s just a tool!” What kind of tool sacrifices so much of which we take for granted? Are these things not important to you? Look at that list again! Fuck, look around, your silly chatbot is not changing the world in the way you want. Maybe you get some minor benefit from your chatbot, but the world shifts further to a dystopian shithole for your marginal gains.

Say you’re in the desert and miraculously you found a glass of water, but it is filled with shit particles. Ya, it’s a glass of water, but it’s filled with shit. And you’re out here thinking to yourself “hmm, there’s water here, and I can drink it. So what if it’s filled with shit? I can use it.” #fuckai

This study actually measured the impact of knowing that your job could be much easier on performance.

Thanks for smoking out more people to block.