Voroxpete@Voroxpete@sh.itjust.works

Replying to @⁨schipelblorp@sh.itjust.works⁩

Basically, yes.

This comes back to the basic problem with LLMs as a business proposition; any digital service can either be cheap and frivolous, or expensive and necessary. You can even, if you do things right, be cheap and necessary. But you cannot ever be expensive and frivolous.

LLMs were supposed to be expensive and necessary, but it turns out they’re just so unreliable that they simply cannot perform an essential role. They must always be disposable; fun, or useful, but never essential.

Which leaves only cheap and frivolous as an option. And they’re just not cheap, nor does it seem likely that they ever will be.

So you get Character.ai filling chapter 11, because people simply will not ever pay what it actually costs to run these models just so that they can have a computer program pretend to be Lois Griffin for them.

en
schipelblorp@schipelblorp@sh.itjust.works

Replying to @⁨Voroxpete@sh.itjust.works⁩

With the added wrinkle that there are expensive things that may be necessary-ish, but when you exclusively use it for those things, the price jumps up to insanity.

Now that people are shunting easier requests to local models and sending the harder problems out, the hyper scalers have less income, so they have to charge more.

Because eventually the only real limitation will be hardware, that kind of AI will be a rarely-used commodity priced only slightly above what it would cost you to recreate it inside your business.

Not to mention the privacy concerns you’d have sending your expensive and necessary compute to the cloud.

Voroxpete@Voroxpete@sh.itjust.works

Replying to @⁨schipelblorp@sh.itjust.works⁩

Now that people are shunting easier requests to local models and sending the harder problems out, the hyper scalers have less income, so they have to charge more.

While you’re 100% correct about the fact that this is happening, and it is a serious problem for the hyper-scalers, it’s really, really important to note that even before this started happening, LLMs were already hilariously unprofitable, both on the software and the hardware side. For the AI companies, there is simply no way to charge people what the models actually cost to run and still turn a profit. On the data-centre side, the turnover on burned out GPUs is going to annihilate any hope of profitability. They’re treating as capex what is essentially a consumable and their business models simply do not account for this reality.

The behaviour you’re identifying here is basically just a swift kick in the guts to someone who was already bleeding to death from several bullet wounds. Seriously unhelpful, but not remotely the impending cause of death.