Mainstream News Anchors Now Talking About the AI Bubble Bursting at Any Moment: "I think the entire AI boom is investing in the wrong future."

“more clear that the real future of AI is small language and open-weight models running on local desktop computers” So where are we with those smaller models? I haven’t been following them. Are they any good?

Qwen3.8-flash-next is making waves since its near frontier or frontier (according to some i havent used it) that can be run on an ancient 8GB video card (12GB or more is recommended. 16GB is best.) with around 32GB of RAM using Strata.

Strata

It’s a mixture of experts MoE model, so while it’s around 66GB (depending on which quantization you decide to run) you’re only really using about 6B parameters per token processed, meaning the VRAM requirements are a lot lower.

I personally don’t see the value proposition in paying $4/1M tokens when i can generate around 30tok/s on 500W of power. I pay ¢13/kWh so to generate 1M output tokens takes me 9.25 hours and costs $0.60.

I’m also running an ancient video card and have a new one in the mail. If i got 200 tok/s even at higher power use i’m still a fraction of the cost of paying for API access and I want that GPU for other things regardless.

I feel lile medium and large businesses have a strong incentive to rent or buy inference using open weight models for a fraction the cost of paying OpenAI or Anthropic. Especially if you can pre load them with everything about your org to make them better suited for internal use.

The vast majority of uses can use a open weight locally run model. I think you would only want to pay for frontier models for really complex tasks with lots of steps, and even then there are usually ways to be smart about how you task agents with work to get better output and use less money.