posted in Technology
Why your local LLM feels dumber than it is
forum.level1techs.com/t/why-your-local-llm-feels-dumber-than-it-is/253917
This really killed a wish to run local LLMs. So fiddly…
Not any more than any other local service. Ollama makes it really easy to get started. The ministral models are small but still useful.
Replying to @frongt@lemmy.zip
sleepingrobots.com/dreams/stop-using-ollama/
And thats just the tip of the iceberg.
Also, while Minstral is fine, it is pretty much obsolete…
Sleeping RobotsFriends Don't Let Friends Use Ollama | Sleeping RobotsOllama gained traction by being the first easy llama.cpp wrapper, then spent years dodging attribution, misleading users, and pivoting to cloud, all while riding VC money earned on someone else's engine. Here's the full history, and why the alternatives are better.Yeah, I’m just saying it’s a really easy way to get started, without being overwhelmed by formats and quantizations and top-k and stuff. You run it, it goes, then you can move to something better.
Replying to @frongt@lemmy.zip
Its sleazy software, and IMO its defaults provide a really terrible 1st time experience. It convinces 1st time users that locally run LLMs are terrible, and goes out of its way to be incompatible with other stuff.
I think LM Studio would be a much better “1st time” recommendation. Its not perfect either, but its lightyears better, not an active con like ollama, and its smoother to “graduate” from it to croco.cpp or llama.cpp or something.