posted in Technology
Why your local LLM feels dumber than it is
forum.level1techs.com/t/why-your-local-llm-feels-dumber-than-it-is/253917
This really killed a wish to run local LLMs. So fiddly…
Replying to @majster@lemmy.zip
You shouldn’t let that discourage you. VLLM is really finicky, but there are easier backends to use (though which one depends on your hardware).