But when I get fixated on something, I want the correct answer. I’d hate to learn the wrong answer and repeat it to someone - I’ve done that before and I hate it. That’s why I avoid AI on principle. I want the right answer and AI won’t give that to me about things I know about so I won’t trust it on things I don’t know about.

This is the thing about LLMs and their users I genuinely don’t get. Like, objectively, being able to input plain speech into a computer and get an almost immediate, detailed, 95% accurate answer is incredible. Genuine technological marvel.

But 95% accuracy is functionally useless for all but the most basic, unimportant tasks. Outside of a coding context (a big exception, to be clear), are people really satisfied with that?

The majority of human history at all levels of science, philosophy, and mathematics. The best you could ever hope for was 95%. Hell you were lucky to break 90% accuracy.

Because the only people you could ask questions was the people in the room with you right then and there or the people you could actively physically go ask.

If you were extremely lucky you could write to someone who MIGHT know more. Or may there’s a book about it that someone knows exists.

95% can and frequently is very much over kill for almost everything anyone will need for their day to day life for random trivia or even most important work.

Very few fields need more than that, and few still is it a matter of life or death.

The part that people fuck up is not being open to learning more and being corrected.

Let’s be honest, most sources online are not 100% accurate. Even before LLMs, find any random article about something you’re an expert in, and there was a good chance you’d spot some inaccuracies. Most of the time, those inaccuracies aren’t a big deal, and when they are, you should be consulting an expert for those.

For me personally, I would rate AI accuracy at about 75%. And for some of my use cases, that’s a pretty decent starting point. Things with no consequences, like trying to quickly find the relevant console command for the game I’m playing. I can get instant feedback on whether it gave me correct info, and if not then I hit Google.

I would never trust AI, at least in its current state, to tell me the truth for anything of consequence.

Let’s be honest, most sources online are not 100% accurate.

Most scientific reviews cite papers in which only the title was read, and often the titles are not justified in the papers.

If you can confirm it, it doesn’t matter what percentage is correct.

When solving computer problems, using an LLM to guide you to the correct answer doesn’t even need 95% accuracy. Every step it gives you have to execute manually, so if something is hallucinated you will notice really quickly. And if by the end your problem is solved, then all the hallucinations didn’t matter.

You could do just as well googling Stack Overflow back when Google wasn’t shit.

But now both SO and Google are shit…

You should avoid social media. You aren’t immune to wrong answers here either. You can just read it from another person and carry it with you. Best get offline!

Perfect example, wrong answer. Stay on the Fediverse, at least until Eternal September gets here.

You’re immune to the Gell-Mann Amnesia Effect!

For what he’s trying to do, it’s not even a horrible use-case. But as someone is a ready to go to the mat on fact checking, he’s definitely not anywhere near arduous enough about it.

He’s not treating it like aunt marge. You wouldn’t make a tv show around finding out truth, then go ask your aunt Marge and not follow up on her answers.

AI for general information gives you approximate directions at best, maybe you got a wikipedia page and maybe you’re reading 297 regurgitations of some asshole that didn’t know what the fuck he was talking about on reddit who got reposted over and over.

The best way I’ve seen it put is that ai is the perfected form of the wisdom of the masses. It’s 90% right 90% of the time. But the most important part is the last 10%.

90% is good enough for most things in your day to day life that have no consequences. But it’s like people have forgotten what it was like before smart phones.

Back when your best chance of a good answer was asking who ever was in hear shot if they knew and just accepting that as good enough. You never treated it as fact just as the best answer your going to get at the moment.

For anything important you would go to the damn library.

It’s 90% right 90% of the time. But the most important part is the last 10%.

No LLM model system card claims such high rate as 90%. Fable claims in its system card, what, 40% hallucination rate?

arxiv.org/pdf/2512.10791 yep (no Fable score here)

Only they’re not “answers”. AI does not provide information, it tells you what you want to hear. It even makes up fake scientific papers as “references”. So you think your info is backed up by peer reviewed sources. Nope!

The better models absolutely will cite real scientific papers that are relevant to your discussion. It doesn’t hallucinate as a matter of course, it can hallucinate some of the time. You can’t trust what it gives you 100%, you have to verify it. But it very often gives useful answers and information.

That’s just not true.

It even makes up fake scientific papers as “references”

Broski, it’s not early 2023 anymore. Thinking models with internet access have been able to provide public sources just for over a year now. Technology evolves; shocking, I know…

They do far better today than they did three years ago, but they still get stuff wrong constantly. I’ve had it give me the source for claims, and the source says the exact opposite of what it claimed. This isn’t a quantized to hell model running on my own hardware, but Opus from Claude.

My go to is to get people to ask it something they know a lot about, and see how bad inaccurate it is. If it gets something you know about so wrong, how wrong is it for everything else?

ok, but they will give you a different answer to a question a day later if you delete previous prompts.

One of my favorite economics bloggers for decades started sourcing his information with LLMs. He was such a bright guy. 😭

Oh they’ll give “answers” alright. They could just be wrong or untethered from reality.