GenAI as it currently stands is a fancy text predictor. You ever had your phone suggest the next word in a message you’re typing? It’s that, on crack.

When you really wrap your head around the fact that that is all it’s doing, it loses a lot of its appeal imho. Especially for the cost to do so.

To be more specific (for anyone interested), the next word predictors are usually a type of model called an LSTM (at least I think that’s the most common). This model type has been used for a long time for dealing with sequential data. In 2014 there was a famous paper introducing an attention mechanism. This was a rather brilliant, though relatively minor extension to how LSTMs work. Essentially between each step of an LSTM it generates some data representing the model’s knowledge of the sequence to that point. The attention mechanism looks back at these intermediate values and determines how relevant each state is to the current point in the sequence and pulls in the most relevant bits. This vastly improved the memory of the LSTM over longer sequences.

In 2017 there was another famous paper “attention is all you need” which said something to the effect of “the attention mechanism is doing all the work, we don’t need the rest of the LSTM we can replace it by running attention between all point combinations in the sequence.” It’s actually significantly slower to run as the model grows, but much much faster to train because it’s not intrinsically sequential. This is the transformer model that’s the basis of all our LLMs.

Obviously some massive simplifications here but as despite being fairly anti AI, I do love the engineering behind it. So yeah, pretty literally a fancy text predictor, but it turns out when you throw all the compute you can muster at a fancy word predictor is makes the world go crazy

That’s still limited to LLMs. A lot of work being done is not even in any language. It speaks election orbitals and bonding energy. This is used for biomolecular and pharma purposes.

This AI work is normally not classified as “GenAI” which is what I was responding to, and the parent was specifically talking about LLMs. Usually GenAI specifically refers to LLMs as well as image/video creation.

The big data centers aren’t for the most part being built for pharma research.

Most part, no. Significant chunk, certainly. Here are some of the larger ongoing contracts:

​Eli Lilly & Insilico Medicine ($2.75 Billion)

Merck KGaA & Valo Health (Up to $3.0 Billion)

Roche & Manifold Bio (Up to $2.0 Billion)

AbbVie & Gilgamesh Pharmaceuticals (Up to $2.0 Billion)

Novartis & Relation Therapeutics (Up to $1.7 Billion)

Isomorphic Labs (Alphabet) with Eli Lilly & Novartis ($2.9 Billion Combined)

Eli Lilly & NVIDIA ($1 Billion)

Just simply ask yourself, why are all the AI companies discussing going public now? I hope you would agree that AI as it currently stands is far from the human brain replacement it was sold as. Outside of a few very specialized fields it’s basically an email generator. They’re out of training data for all intents and purposes. AI generated content is so ubiquitous now that you can’t use most data moving forward without painstakingly checking it all, and AI is becoming increasingly harder to distinguish cheaply or easily. The widespread adoption has poisoned the well. So AI is as advanced as it’s going to be, and it’s not worth its valuation. They’re all racing for the exit and IPOs are their last hope for their backers to sell and get out before the markets stop being irrational. I hope I’m wrong but that seems to be the writing on the wall.

Edit: they’re also already posturing the current administration for a bailout deal.

Microsoft contradicts your argument on all sides.