‘Doom Loop’: OpenAI and Microsoft Admits LLMs Are Destroying the Web and Built on Theft
Executives working on AI at Microsoft and OpenAI admitted what its critics have been saying all along: Large language models are predatory pieces of technology that have been built on what a Microsoft executive called “an astonishing theft of unprecedented proportions,” and the “largest theft of labor in human history.” An internal Microsoft document said generative AI products have created a “doom loop” that is killing “the entire web.”
Those statements and a series of other mask-off moments feature heavily in an unredacted court filing that was unsealed Thursday in the behemoth New York Times vs OpenAI copyright lawsuit that has been winding its way through the court system for years. In a filing asking for summary judgment (basically, a filing with the court asking it to rule), lawyers for the New York Times laid out a series of admissions made by Microsoft and OpenAI executives in documents and depositions that until now had remained either sealed or redacted at the request of Microsoft and OpenAI.
It’s easy to see why the AI companies wanted to hide this from the public. The statements, taken together, are some of the most damning indictments of the ways LLMs were trained, how they worked, and the immediate threat they pose to human labor. It is a reminder that even as AI becomes more powerful and companies try to shift the narrative to the supposed existential risk of “superintelligent” AI, the tools they have already built were created by stealing from human creativity and labor and are by definition existential threats to the human labor market.
“Our AI content strategy has started a ‘doom loop’ that will hurt the performance of our models and the entire web at the same time: It is highly unusual that an end-product threatens the economic foundations of its essential suppliers, but that is the situation we have created for our LLM business with respect to its ‘content supply chain,’” the document said.
Microsoft executives, including CEO Satya Nadella, testified under oath that after ripping content from the New York Times and other news sites, clicks to those news sites fully cratered, falling by more than 90 percent on Bing.
Documents obtained during the court proceedings found that OpenAI created “a hack to get around nytimes paywall,” to which OpenAI cofounder Greg Brockman said “ah, nice.” Microsoft executive Brent Hecht wrote that LLMs steal content “without ways of distributing economic value down the supply chain, [which] necessarily threatens the economic stability of those who create the content.”
That’s because, thus far, they get away with choosing not to distribute any of their trillions of dollars to the suppliers of the information they consume - money has only gone to the suppliers of hardware and power, and to influencing politicians and rewarding investors. That’s their choice, and they should not be allowed to continue to make that choice. Good luck to the NYT.
I think the way they are operating now is commonly called predatory pricing.
I don’t know if there is a potential for a profitable business that covers all the costs. Possibly the direct costs, but I think not if they had to fairly compensate those who created the intellectual property they consumed to train their models, and much less likely if they had to pay the indirect costs. They might, some day, compensate creators a little, like Google shares a pittance of their ad revenue with some creators. But it is unlikely to be anything near a living wage.
AND it’s subsidised by too big a supply of compute power - Microsoft prints demand for Azure, by buying ownership stakes in openAI through the grant of Azure credits. There’s too much available compute.
From an economic viewpoint AI companies are not worth is as it is now. OpenAI and Anthropic are hundreds of billions of dollars in debt and will never make a profit. Same goes for the AI subsidiaries of Microsoft, Meta, Google etc, except they are just leeching off of the mother companies and hiding their figures among their finances. Paying creators for their data would make very little difference in their overall finances.
This is why I’ve been arguing to anyone who will listen for like four years now. If this tech is so amazing, someone will figure out a way to make money off of it. Right? The fact that it’s been 4+ years and no one has made a damn cent off of it should be making more people skeptical. The fact that 4 years ago it was widely celebrated even though no one had a plan to make a damn cent off of it should have made more people skeptical back then. It’s weird that I have to keep arguing this with folks.
NYT is transphobic and pro-genocide. I hope they go bankrupt, and papers with fewer conservative biases survive. Unfortunately, the opposite will happen. Billionaire-backed news will run at a loss to manufacture propaganda, while independent journalism dies.
That’s kind of the funniest part in fact: these companies are destroying their own viable business segments. Bing was a huge growth driver for Microsoft. Less clicks is bad for them. They make more money on Bing ads than they do on LLMs. Same with Google.
Reddit is getting crushed atm after it sold access to its data to train models. Chat bots make visiting these websites pointless, without replacing that traffic with anything they can meaningfully monetize.
The more popular Gemini is, the less money Google will make. The market has already shown how much people are willing to pay for Ai subscriptions, and it isn’t all that much. None of these companies have found a way to make ads viable in LLMs either. They are beyond self sabotaging themselves at this point. It really is a doom loop.
So the admission of intentionally circumventing the NYT paywall is where this becomes a real actual IP infringement issue for them. From a legal perspective, that is literally as crazy a thing to come out in discovery as Anthropic’s torrenting an enormous chunk of their training corpus. These are IP infringements, totally irrespective their use in training GPTs. I cannot imagine having to represent these idiots.
Devil’s advocate, self regulation does not work. 9/10 companies can agree to cool it and if 1/10 presses on full steam ahead for whatever reason, then they can still do all the damage and they will be ahead of the curve.
We need laws and regulations that hold all companies accountable.
Microsoft, in a policy document, wrote that generative AI could “significantly disrupt the employment of the very people who generated the data on which the foundation model was trained […] LLMs are a product that destroys its supply chain.”
If that’s not the very definition of a categorically unsustainable business model, I don’t know what the fuck is.
That’s not quite the thing. They know that people in various societies in the course of history could create artifacts getting much less in return than they’d get today in a market without LLMs. They just want those people to become crops in their fields, so to say. To reap and not sow.
There’s the obvious problem with this, that there still are alternative business models of closed circles and ordering artifacts, not accepting offers to buy them. And, of course, that either it’s IP violation of ultimate proportions, or if it’s not, then they are going to have too much competition to get anything out of it anyway.
So it’s either oligopoly with regulatory capture, or too much competition to make this profitable.
They apparently want to make some theft legal and some not, a bit like copyright protection entities in ex-Soviet states, which are usually controlled by former pirates legalized (with ex-pirate electronic libraries which are now legitimate stores, except I’m confident they didn’t get consent of most authors whose books are being sold, similarly with movies and such), hunting their “honest pirate” competition.
So - no, at the point this is mainstream, the only regulation of this should be aimed at enforcing existing laws against them. Then one can think of some, but before existing laws are being firmly enforced - no.
Except enforcing existing laws can mean fines bigger than their capitalization and some life sentences for all chief people involved, and these are all “too big to fail” companies, so I don’t even know.
If these companies are, figuratively, sentenced to death for doing this, then it’ll be a process of the century. It’ll define future. A bit like with Standard Oil and United Fruit Company. Or so I think.
Except it might go differently, in their favor, then it won’t be a very nice future.
I mean there is not very much new data to train on, and only so much refining you can do (and you don’t need to be one of the big endebted companies to do that).
“All by itself” usually doesn’t happen, “it” just stops looking like a problem for those not destroyed by “it”, and those destroyed by “it” stop existing.
Standard Oil’s monopoly wasn’t ever truly dealt with in my opinion. We’re still seeing the fallout of people who got way too rich off it. I’m sure some asshole Rockefeller has had a meeting on Venezuelan oil in the last year.
Like, intellectual property doesn’t matter to me at all with respect to anything I may have produced that has been scraped. I am, however, really not liking the idea that parodies on highly niche technical subjects that my me or my friends might have been created might have been added to their “collective knowledge” or whatever and it will be used to synthesize answers in the future.
Not because it’s anything so powerful (this is not a flex, I swear) but that’s got to be an infinitesimal fraction of all the absolute trash that’s been fed to the machines without scrutiny.
Anyone with 2 braincells saw this from day 1. Unfortunately the timeline lined up with USAians electing a president with less than 1 braincell. So there was no one to stop them when it was time to stop them.
They destroyed the web forever for the entire world, but owned the libs for a couple years, so it’s worth it I guess.
Gonna get real weird soon when they run out of rich new training data and they begin to consume their own shit. When that happens their entire model will collapse and if the bubble hasn’t popped already that may be what causes it.
They’re already doing it. They call it distilling when they take it from another llm. Pretty sure most content was already stolen in the early days and they now rely on distillation and stealing new content.
Why do you think there has been such an emphasis on hacking with LLMs lately (especially by OpenAI). They figured out all those vulnerability databases were an excellent source for training material. In the past they scraped those, but just for general language training. Now they’ve specifically trained the models on the information within. Some team figured out how to use that data to train a model and have testing scenarios automated so they could write a good reward function. It wasn’t that they figured out the models are good at hacking, they ran out of content and found a new source of good data.
With all the books they’ve been scanning I wonder if the next thing is going to be a writing assistant or editor or something like that. Even though writing good books is an art form and the actual writing down of the words is the easiest part (still not easy tho).
These companies are starving for content and they’ve not just poisoned but absolutely destroyed the content well that is the internet. Given they were already hitting diminishing returns hard, it doesn’t matter too much to them probably. But more compute and storage has also been hitting diminishing returns hard and customers are complaining about the cost. So they are getting a bit desperate on how to improve these things at all.
Pretty sure they mostly use the pre-AI internet (that they scraped and kept) and synthetic data currently. Probably trying (and failing so far or we’d have heard) to adapt to using video as training material at the moment, but developments there will likely apply to robotics at some point. Here’s hoping the current chuds have crashed and burned before then and that some sanity has taken over from unfettered capitalist oligarchs dreams of computer slavery.
Nah, it’s in the name, Large Language Models, what gets marketed as ‘AI’, are trained on text. That’s why they’re ripping up secondhand books (and copyright law, but that never stopped them) at the moment. No one has really cracked Large Vision Model yet, although there are some primitive versions being attempted in robotics, mostly they translate to text, which is why general purpose robots aren’t a thing yet.
I think we’re talking two different things. Like, for example, they can create data using ALPRs to create a history of a person’s location to train LLMs.
It’s either agi or model collapse. For every AI system actually. If you have a high enough adoption you start to muss original data so if your AI is not self sufficient in time it will collapse, because it will be trained on its own data, which just is an incentive loop
in a manner of speaking - you always end up there. not by design though. models operate via continuous refinement and you can only optimize a model so much until it is a mess and you need to figure out where to roll back. so you either get shit like semantic drift or variance decay and you can whack a mole it to an extent but then you hit the rlhf wall when the model starts gaming its reinforcement framework and the fat lady sings.
the only more or less workable way to keep it under control is maintaining a closed loop small-scale environment - kinda like NotebookLM where you upload documents and that’s all there is to work with - outside of that it is a mess.
“Our AI content strategy has started a ‘doom loop’ that will hurt the performance of our models and the entire web at the same time: It is highly unusual that an end-product threatens the economic foundations of its essential suppliers, but that is the situation we have created for our LLM business with respect to its ‘content supply chain,’” the document said.
funny thing is that this ouroboros thing been known ever since LLM concept became a thing. that’s why it took forever to take off beyond r&d experiments. basically, the only remotely adequate way to apply LLM is what NotebookLM does - which literally parsing an uploaded document and reiterating it in some preset format. that’s literally it.
They knew what they were doing every step of the way. They are criminals that need to be disarmed and locked away. And we need to create a new Internet from scratch somehow thanks to these donkeys.
technically, it’s the hardware that helps humans destroy the planet. and that hardware is astoundingly inefficient in what they’re trying to do with it. if you need to stack a literal data center with GPUs - maybe you need to develop some new tech.
humans destroy the planet with gear. that’s what i meant. and it’s the humans who think they can build superintelligent AI with tech and gear so woefully unfit for that it is not even funny.
Pretty optimistic to assume LLM’s weren’t designed to kill people. Especially since we know that Israel and other countries literally use AI tech to kill people.
The current AI push is specifically for killing people. The billionaires know their days are numbered, and that they’ll eventually vibe called to answer for their crimes. They’re trying to build a system to protect their power and wealth that doesn’t depend on potentially disloyal guards and militaries.
I once looked for a tutorial on a process in a Minecraft mod and there was a site that gave out incorrect information that was obviously written by an LLM.
this website used to pop up in pretty much all web searches i made about retro gaming. looks like it’s blacklisted from qwant now, since i couldn’t find it anymore by searching and had to dig through my discord messages to find the link
“To give you an idea of the size range, Kirby’s Adventure, the largest officially released DS game, takes up 768 KB, while smaller games like Tetris weigh in at around 8MiB. The maximum recorded storage capacity of a Nintendo DS cartridge is 64 MB.”
and then right after that “Size Range: 8MiB to 512MB”, followed by:
“According to various sources, the average DS game size is 32mb to 64mb, with some games being larger or smaller. The actual average would be hard to calculate, but it’s likely somewhere between 128 KB and 256 KB.”
I did a search for the torque spec on a castle nut last week and the second result was for a nut (as in, food nuts that you eat) website and the llm had gone all in on nuts and added a page about castle nuts. It even generated an image of a castle for the top because… castle nut.
It proceeded to provide vague instructions and a torque recommendation of 2x the actual spec.
Then there was the other one about a small 4 stroke engine where it stated “other sites will say you don’t need to mix oil with gas but you absolutely must!” (You don’t and shouldn’t)
… I wonder how many people have fucked up their shit by trusting llm garbage without knowing better.
Running into this same problem but with parenting. My wife is constantly asking me to “Google if it’s good/bad if our baby is doing” X. And I’ll find 20 slopposts from dozens of “parenting” websites that all give conflicting answers. This week, I asked my parents for their copy of Dr. Spock’s Baby and Childcare because we need some solid reference. Sorry kiddo, you’re going to get raised like it’s 1987 because 2026 sucks.
Try the 1940’s when the book was first published. Most of what he said, like talking to and listening to your kids and treating them like a human is really good. After a quick review of criticism it doesn’t appear there is much worthwhile other than some of his earlier views that did change later on (like male circumcision).
Copy right is ridiculous, ideas should be shared freely.
I’m glad ai is ruining the internet.
They all deserve nothing less for being brain dead capitalist drones. The primary concern with ai is it taking your useless crap without paying you?!
Your idea is worthless, but the big players ideas are hoarded and you are worried about your little shit drawings or crap stories being eaten by ai. Luddite donkeys.
But they charge for their “services” and if it was ultimately a net positive I’d be rallying alongside you, but they just suck and have made everything else they touch also suck.
Copyright is ridiculous and AI shows us how little it matters to the government and corporations if it means getting what they want.
I am not glad AI is destroying all the little guys on the Internet. Fan made and hobbyist sites are being scrapped to death because tons of AI bots are scrapping the same content over and over again astronomically increasing the cost of running a website. This is complete bullshit and should be treated like the attack it is.
I liken it to something crazy like companies sending you texts that your phone company forces you to pay only they sent you like 100,000 texts and now your phone bill is for thousands of dollars. It is an impossible burden they are imposing.
Yeah once the mainstream is fully saturated they’ll turn their gaze to smaller, more “authentic” places. Imagine if they could have a boot infiltrate a tiny little private chat between a dozen people by building up the appearance of an organic presence on the fediverse / indie web.
Now advertisers (and potentially, malicious state actors) have a little spy and influencer in your group.
All the world’s a forum,
And all the bots and users merely shitposters;
They have their memes and their wojacks;
And one poster in their time submits many posts.
Slop commenting. The litany of AI slop websites that have popped up, polluting search results. The absolute flood of AI slop pull requests being submitted to open source projects, to the point at which many open source maintainers are buckling under the weight of garbage being thrown their way. Bots everywhere. The list goes on and on and on.
The internet is already a dessicated husk of what it once was.
Something others haven’t touched on is the burden all the scrapers are putting on small websites and forums.
Lots of niche/hobbyist site runners have repeatedly talked about how the scrapers inefficiently scrape the same pages, images, and files endlessly. Until they crash the site, run them out of bandwidth, or run up their bills till they cant afford to host a site they’ve been running for many years before this nonsense.
All in the futile pursuit of more to dump into the machine because of the belief if they just had more data all the issues with the models would finally be solved.
Actually surprised how candid some of these statements are. Maybe points to more internal resistance than I would have assumed… like, they see the problem.
Their focus on threats to media companies is probably borne out of fear of litigation, so maybe they only pay a lil lip service to how they also hurt “content creators” (people).
Could also be that they know that when talking to the representatives of runaway financial automatons like multinational corporations, they have to appeal in terms the machine will understand.
The best succinct description of llm that I’ve read was “plagiarism machine”, because that’s basically what they are: automated plagiarism. At best they create a collage of other works, at worst they copy one work verbatim, but they never create something original because they can’t.
Copying something is a lot easier and thus cheaper than creating an original work from scratch, so genuine creators cannot hope to compete on cost. Which leads to less people being able to afford to earn a living from creating works, which leads to less original works being created.
Short term that’s great for the neoliberal company executives: fire the expensive artists, create cheap slop with the plagiarism machine, and thus maximize profits now. That the plagiarism machine becomes stagnant because not enough new original works are being created is a future problem, by which time the current executives will have already jumped ship.
But while it’s great for the neoliberal business model, for the rest of society it will suck. So now that we have confirmation that the AI executives know how bad their products are, and that they were just publicly lying about it in the style of tobacco executives, will anything be done about this “astonishing theft of unprecedented proportions”? Personally I doubt it, there’s too much regulatory capture.
I’ve started to think of LLMs like a worm, running around copying and pasting gobbledegook. It’s the ultimate filter, the gibberish curtain of advertising
Let me assure you that I’m very lazy. I will passionately start using LLM’s as soon as they can do things that I need done… but it seems like that could take awhile.
Transport isn’t the issue, but both those protocols provide low-bandwidth hosting requirements. NomadNet is straight out of the 90’s bbs territory, and I2P is slow enough on its own to preclude anything too fancy. It’s forcing us to go back to the small distributed hosting models, which I consider a good thing. They’re also resistant to censorship and, in some cases, reasonably privacy-conscious
You could try to do some kind of bridge server side or just add straight up identity/send/receive client side. You need RNSD running somewhere to hook.
I honestly don’t see a point where we can move past this without Sam altman, nadell, musk, etc all facing mandatory life time sentence without parole, work release etc.
Could you imagine Elon Musk running a fryer or injection mold for 8 hours? Who am I kidding? he’d never pass the piss test required to run the injection mold.
I’m sure many of them think they would just prove us wrong, and because they are so smart that they would just rebuild from the ground up on their own.
My concern is they could convince another rich person of this, and get them to invest in their newest scam.
There’d have to be some sort of stipulation that made it so nobody could do that. I don’t know how you’d do that, but it could probably be done. Are there any laws that prevent people convicted of fraud from doing the same thing again? Maybe something like that.
I would love to see them roofing houses or working in a foundry. Hell, even seeing them trying to be a janitor or something.
I’d prefer if they were forced to pay royalties for all the work they stole to the people they stole it from. And, like, have someone actually force them to comply. They’d have to hire a Microsoft-sized compliance department just to figure this shit out and track payments.
Yeah, but with which money? All those AI companies are not and will never be profitable (and they’ll be the reason for the next stock market bubble pop)
weird how Chinese AI firms aren’t saying this - but then, they don’t have a massive fiscal crunch where they have to take delivery of trillions of dollars worth of contracted compute, that they haven’t yet found a market for . . . .
What seriously worries me about these companies is the fact they somehow act like what ChatGPR is doing is 100% out of their control. Dude, you built the LLM, you trained it on stolen data.
Stop acting like “oh evil LLM is built on theft and is hacking all these companies and will doom us all, whatever will we doooo?!” when it was your damn company that started this whole crap and you’re always blabbering about general AI and how this time it will definitely arrive by 2029 if they give you another 10 billion dollars. I can’t with these clowns. Remember when Musk had us believe he wanted to save the world and came from poverty? Yeah, good times
404 Media‘Doom Loop’: OpenAI and Microsoft Admits LLMs Are Destroying the Web and Built on Theft"Millions of people around the world will soon consider large models ‘hoovering up’ all their work to be an astonishing theft of unprecedented proportions."
