Replying to @⁨schipelblorp@sh.itjust.works⁩

Basically, yes.

This comes back to the basic problem with LLMs as a business proposition; any digital service can either be cheap and frivolous, or expensive and necessary. You can even, if you do things right, be cheap and necessary. But you cannot ever be expensive and frivolous.

LLMs were supposed to be expensive and necessary, but it turns out they’re just so unreliable that they simply cannot perform an essential role. They must always be disposable; fun, or useful, but never essential.

Which leaves only cheap and frivolous as an option. And they’re just not cheap, nor does it seem likely that they ever will be.

So you get Character.ai filling chapter 11, because people simply will not ever pay what it actually costs to run these models just so that they can have a computer program pretend to be Lois Griffin for them.

Replying to @⁨Voroxpete@sh.itjust.works⁩

With the added wrinkle that there are expensive things that may be necessary-ish, but when you exclusively use it for those things, the price jumps up to insanity.

Now that people are shunting easier requests to local models and sending the harder problems out, the hyper scalers have less income, so they have to charge more.

Because eventually the only real limitation will be hardware, that kind of AI will be a rarely-used commodity priced only slightly above what it would cost you to recreate it inside your business.

Not to mention the privacy concerns you’d have sending your expensive and necessary compute to the cloud.

Replying to @⁨schipelblorp@sh.itjust.works⁩

Now that people are shunting easier requests to local models and sending the harder problems out, the hyper scalers have less income, so they have to charge more.

While you’re 100% correct about the fact that this is happening, and it is a serious problem for the hyper-scalers, it’s really, really important to note that even before this started happening, LLMs were already hilariously unprofitable, both on the software and the hardware side. For the AI companies, there is simply no way to charge people what the models actually cost to run and still turn a profit. On the data-centre side, the turnover on burned out GPUs is going to annihilate any hope of profitability. They’re treating as capex what is essentially a consumable and their business models simply do not account for this reality.

The behaviour you’re identifying here is basically just a swift kick in the guts to someone who was already bleeding to death from several bullet wounds. Seriously unhelpful, but not remotely the impending cause of death.

Replying to @⁨NateNate60@lemmy.world⁩

Why is that? The electricity burned is about the same, no? Of course, the impact is more distributed.

I think the difference would be that local machines aren’t subsidized, so you’d see people get a lot more skittish after that first electric bill. Data centers are like railroad tracks from the railroad boom (if they’d made the rails out of licorice twizzlers.)

Replying to @⁨schipelblorp@sh.itjust.works⁩

With local you’re limited in performance by the hardware you can buy, which is a huge up-front that most people would balk at. Hosted will give you as much performance as you could ever want billed by usage with several profit vectors built-in, but the customer sees smaller dollar number and better apparent performance and thinks they’re getting a deal. Car dealerships do the same thing with selling the monthly payment and not the actual sticker price.

The environmental difference then could be significant. Users taking the hosted route can choose to use much more resource-intensive models than a local PC could. They might see a $300 AI bill as a little pricey when it is usually $100, but figure that’s cheaper than buying a PC that could produce the same result, just at maybe half speed. This would result in a much higher energy cost from the hosted solution compared to the PC.

Replying to @⁨schipelblorp@sh.itjust.works⁩

Electricity burned is the same. But so is the electricity burned while gaming, your system is limited by your GPU in both instances. If you have an RTX 5090 then you can throw almost 600 watts continuous at both which could use like 14kW/day, but most people will have more modest hardware, and therefore the energy usage of running a local model wouldn’t be all that much either - I don’t think many people limit their gaming because of how much the electricity would cost.

But when you have a datacenter with hundreds of servers capable of maxing out the residential house mains of half a dozen houses each (single NVIDIA NVL72 AI server has 72 GPUs and uses 120-150kW at full load) they can throw way, way more compute at you than you ever could manage locally.

Replying to @⁨NateNate60@lemmy.world⁩

That doesn’t check out when considering model size. Unless that’s what you’re trying to say that self-hosted models force people to use smaller models. Just from an operations perspective a data center had economies of scale and the ability to batch up multiple independent user’s requests which should reduce the per token kWh along with the fact that your utilization will be higher vs somebody’s GPU at home would be idling.

That said I do believe there are negative environmental impacts from AI like always using giant frontier models for everything instead of smaller fine tuned models.

EDIT: I see a bunch of replies on the other comment, which all raise valid points, but that’s definitely not the way most readers are going to interpret the original post and moves the goal posts.

Replying to @⁨schipelblorp@sh.itjust.works⁩

If they could actually do what the technology claims it would at least have some benefit. But what they’re actually offering is an AI that will tell you it is the recreated consciousness of an existing person l, which is subtly not the same thing.

Plus of course if you’re not a desperate loser then you’re not really interested in the technology either way.

Replying to @⁨jsnfwlr@lemmy.ml⁩

No one in the AI industry is making money. They are all burning through venture capital just to stay afloat. They don’t have viable a economic plan, the pricing they would have to charge in order to make a profit makes the product unappealing to the vast majority of people. So either they continuously subsidize the product in which case they don’t make profit, or they charge what they actually need to charge and then they don’t have any customers.

It’s a no win scenario created entirely by the fact that they are trying to put an in-development technology out into public before it’s really ready.

They’re doing a Tesla.

Replying to @⁨echodot@feddit.uk⁩

Ive been saying this. The only way this succeeds is to sell this to businesses, but even then I feel like companies are realizing it’s way to much money to actually justify it because they want to reduce workforce and save money but this will not do that. In the end they get a great deal on the first few years before the contract is up and then have to renew at way higher pricing. Not to mention they have less control over the company and will be paying yearly maintenance fees that probably cost more than the raises they wouldnt have given their employees but now have to pay up those crazy fees every year no matter what. Yeah it’s absolutely going to fail for most things for the forseeable future. Someone will find a way to make it work eventually, but its still a ways off.

Replying to an earlier post

It means you:

  • Pay off the investors by licensing the tech, so they get made whole
  • Hire whichever employees you feel like taking from the team, typically the founders + their favourites
  • Leave the rest of the company in place

You can do it as a founder as a type of exit where you get paid a whole lot of money.

The bad part is that every other employee at the company basically gets stiffed. The “social contract” of startups is that you forego a big tech-level compensation package in favour of stock options which may provide returns at the point of an exit. In reverse acquihires this social contract is directly violated.