Artificial intelligence labs are in a new arms race to buy up millions of rare books, slicing them open, scanning the pages and pulping the remains — sparking concerns that the last remaining copies of out-of-print texts are being destroyed on an industrial scale.
ISBNdb notes that “print books from the pre-LLM era are structurally guaranteed to be free of this contamination”.
“Millions of the most valuable books have never been digitised. They exist only in physical form, scattered across library shelves, used bookstores, and out-of-print catalogues. We get them to you at scale.”
I really hope the AI bubble pops soon and takes at least some of these stupid companies with it. They’ve been operating at a GIANT loss for years at this point, how much longer will this go on?
The way I see the situation is that every single AI company can clearly see the massive wall they are speeding ever faster towards, they see it and can also see that it is a massive wall, thick and sturdy.
There is however one thing that keeps them pushing the accelerator down hard against the floor, the delusion that they alone are smart enough to find the prefect weakpoint where they can smash through to the promised land of massive promises, while everyone else is just flattened against the wall.
It is extreme narcissism combined with yes men powering this shit.
Investors are starting to notice that it’s all smoke and mirrors in terms of the profits. I am reasonably sure it will pop this year. I will be utterly shocked if it doesn’t pop by the end of next year. It’s just so comically and obviously unsustainable.
This is gonna go down as the biggest Ponzi scheme of all time, I swear.
Depends on how much power the financiers have to fabricate capital from debt. In modern economics that’s not necessarily a limitation. I do hope it will be tho.
As his faculties return, Robert (who has always been technophobic) must adapt to a different world, where almost every object is networked and mediated-reality technology is commonplace.
…
However, jitter and latency are considerable problems with this basic network when long distances are involved. In the novel, Robert Gu develops an algorithm that partially compensates for these technical deficiencies, and might ultimately allow the inclusion of haptics.
This feels like it would be a little Mary-Sue to me, but I haven’t read the book. Is that transition handled well?
I found it a bit of a stretch that a cranky technophobe English professor could, or even would bother to, create an algorithm the whole world of math and computer nerds overlooked. But I don’t let preposterous characters get in the way of my big-ideas SF when I can help it.
Anthropic described the project as “our effort to disruptively scan all the world’s books” in an internal memo, which noted “we do not want it to be known that we are pursuing this project”.
That would imply preserving the original. No, they are having their AIs revise history and its literature to whatever they deem ‘acceptable’ in their new world order, hence the rampant book banning spree they’ve been on as of late.
if this was about Number Go Up, they'd probably make more money selling the despined books or something instead of destroying them and selling (or do whatever they're doing with) the raw pulp. this sounds like an expensive process and there's no guarantee that expanding these datasets is going to improve their models in a quantifiable way, let alone a profitable one. they must just get some kind of perverse pleasure out of making the world a worse place
The bookseller explained that in April, he suddenly went from selling around 20 books per week to hundreds — including obscure and out-of-print works.
“It benefits me financially as well as by clearing out old inventory that is otherwise unlikely to sell,” he said.
I hate the AI slop as much as the next guy, but these are books nobody wanted. They were going to rot on the shelf until being thrown out. Just being rare doesn’t mean it’s particularly valuable.
Hopefully they are retaining the digitized versions for release at some point in the future. I’d imagine they are, since they’d still need the raw data to train future models.
You can’t determine what knowledge will be useful or valuable to someone in the future. Archaeologists have poured over receipts and shopping lists from ancient Egypt that were thrown away. It’s why the Internet Archive backs up all the random webpages they do. There’s no justification for mass destruction of books, especially since it’s being done to train broadly useless LLMs.
Of course they won’t be releasing these books later on. These companies want to hoard this text data so their competitors can’t access it. It would also be illegally distributing copyrighted work. Aside from destructive scanning being faster, the fact that a book is destroyed makes the digital copy “transformative” and therefore legal if it isn’t further copied or distributed. A judge ruled on this recently. I don’t mean to advocate for copyright enforcement, but that’s the reality here.
I think China has a national repository of AI training data that companies can access if they contribute to it as well. Doing the same thing in the West would at least blunt some of the harm the AI craze is causing, but that would have made too much sense.
Bernie’s plan for government control of AI companies made a lot of sense in this regard. And would mean those vast scanned libraries were public domain, or at least publicly stewarded.
To be fair, we don’t know what books those are, anyone can publish anything as a book these days. It doesn’t have to be good or even true. For example, Jordan Peterson shits out some books, and I personally would not mind if all copies were shredded and forgotten.
Yeah, part of me thinks this is part of why they are desperate for these LLMs to take off.
Get everyone to trust it, then just manipulate it to saw whatever fascist lies they want as truth. Look at how Elon gets all pissy everytime Grok starts going to the left on its ideas.
As much as I think this behaviour is fucked up, once they buy the books it’s legally property of the company. And it can be destroyed without consequence since it’s legally property of the company.
I agree there is no current law for it. It just feels like there should be some kind of obligation to ensure records are preserved. With heavy enforcement for mass operations of record destruction.
The whole point of post-enlightenment government is to serve the public good. But then it gets captured by capitalist interests (as Karl Marx explains in Das Kapital) at which point the government serves capital.
Issue is, the AI accelerationist cult has planned out every scenario, and they have a very good plan to amaze politicians. That’s why the bubble haven’t burst yet.
To be honest, what surprises me most of all is that, after three decades of intensive digitisation of analogue media, there are still purely analogue copies in existence at all!? In that respect, I see this more as a benefit than a threat, as some people here are suggesting.
Absolutely. At the very least, everything of relevance, that is, anything that has been catalogued in libraries, for example, should be almost entirely digitised. We may well find out soon whether AI companies can still uncover anything of relevance amongst the other books, i.e. those that have not been widely published or catalogued. en.wikipedia.org/wiki/Book_scanning
The book needs to be not copyrightable at the moment, then you need to have a copy, then you need to put work into scanning and OCRing it, then to place it in somewhere accessible. It’s a pretty narrow window. Some books that get published now get no digital release or it’s put behind a heavy price tag.
Does anyone remember circa 2014 that radical jihadists in the middle east were destroying ancient and medieval artifacts, including stone heads? And I remember here in the liberal west we thought that was awful.
This is not the only thing that makes me think of that. Also Trump’s recent calls to reduce the size of US national parks and open them up to oil and uranium mining exploitation.
Rare doesn’t necessarily mean culturally valuable. How many of these rare books are actually just old instruction manuals, travel guides, training materials, phone books, and dictionaries?
If they did have a copy in Anna’s Archive, why the fuck would AI companies find physical copies, buy them, rip them apart, scan them, and then digitize them before destroying them?
Remember, you don’t get the digital copy. It’s only used to train their models.
Absolutely. Every AI company has Anna’s Archive in their training data already. They want to differentiate their models by having more training data than their competitors. There are way more books out there that have never been scanned than those that have.
Yeah we remember. Because that’s a literal fucking warcrime. You’re not allowed to destroy historical sites.
While I agree that destroying books is bad - even common ones - it’s also not technically illegal. If I want to burn a first edition of whatever book worth thousands of dollars… I’m legally allowed to do that.
I can also see the benefit of having a digital version of a rare book that can be enjoyed by thousands as opposed to sitting on a shelf. But yeah, definitely hold companies accountable for destroying them. There has to be a better way to do that work and still have a useable book afterwards.
I don’t trust AI companies to not alter the texts of those books if there’s something in them that does not align with their worldview.
So any book added to the online archive(s) by these companies is suspect.
The point of bookburning is that it would only be accessible to AI, bringing forth it’s monopoly on said first edition. No one would upload it to IA in the process. It is deleting one source of knowledge out of existence.
That is the justification and not the reason. Its faster and cheaper to cut the bindings, and they would have to store the books while they are using the digital copies to comply with copyright.
If they were archiving digital versions so they could be read by thousands in the future, this would be a totally different conversation.
They don’t even need to be as selfless as the internet archive - just follow Googles lead and allow people to search and view selections the archive, with automatic opening of the archive when you are confident copyright is expired. Even a closed archive owned by a third party with a dead mans switch to open it in the future when the business model is done would be something.
Sorry, but I did an oopsie. I watched the video linked, they destroy the books to scan them. It’s really sad. Though I’m sure they would destroy them anyway.
They take apart the books to scan them. There’s no scanner in the world that deliberately destroys the paper it’s scanning. They pulp them because that’s more advantageous to them than reselling or archiving the disassembled pages.
Fair use loophole, they claim they convert one physical book into one digital with no illicit copies, while also supporting the delusion of AI being an equivalent to a human reader. It is weird on so many levels but no one of noticeable weight asked them wtf.
Copyright. They mustn’t make a copy. There is legal precedent that confirms it’s okay to transform the copy you bought into digital format. The Internet Archive relies a lot on that. I think they actually litigated it in the first place. So that’s why the copyright heads are going so absolutely apeshit. If no one’s charging you rent for using some data, then it’s “unethical”.
I was misremembering. Google won the precedent, when they were suing over Google Books. IA had only 1 big lawsuit and were forced to settle. The non-destructive scanning is dicey.
You people have no idea how much I want AI banned all across everything. I don’t care about anything at this point. These people are EVIL EVIL EVIL EVIL. Pure breed CRIMINALS
Fucking 90% of the books I read are “rare”
“Oh, but there’s SOME benefits to having AI”, says some person on the internet that had it shoved down their throat by their boss. No. Fuck you. Fuuuuuuck you all the way.
That’s sort of the joke. These aren’t treasured first editions. They’re not shredding the Guttenburg Bible. These are single prints sitting in warehouses and abandoned libraries or dumps that nobody has read or carried about in decades. They’re rare because they never got a second edition.
And the process of digitizing the book means disassembling it. So the copy (that nobody wanted) is destroyed as a result.
The article is pure clickbait. These books aren’t valuable, save that they were produced pre-AI. That’s why you can get them in bulk on the cheap.
They are buying up used books at rates that push up the cost of used books. That is enough to piss off people legitimately.
We also are fully aware they are not careful. Yes, they will occasionally shred the last few examples of a book that might have otherwise become useful in the future, and there’s no indication that they are archivists. They will eventually destroy or loose this training data and the book contents will be gone to future generations.
They are buying up used books at rates that push up the cost of used books.
Used books are practically free to begin with. Libraries will often give away whole storage containers of books that they don’t want anymore. Bookstores will sell books for pennies on the wholesale market.
Half-Price offers Books By The Yard to be used entirely for decorative purposes. $50 for 40-60 books, largely organized by color and binding style.
a book that might have otherwise become useful in the future
We have vastly overproduced the physical reading material relative to consumer reading habits. We have vastly overproduced certain genres and subject matter material. We have overproduced individual volumes. These books are not useful in any material sense. No more than the clothes we overproduce or the snacks we overproduce or the automobiles and electronics we overproduce.
We are swimming in trash. We produce mountains of it. We fill ravines with it. Stop treating hording as a virtue.
if you want any old used book, sure. if you want a specific used book, maybe you might have to spend ten to twenty to a thousand (i have needed some rare texts over the years).
you want a specific used book, maybe you might have to spend ten to twenty to a thousand
Show me the thousand dollar copy that an AI company destroyed and we can revisit this.
Nothing in the article identities an actual valuable work being destroyed. They’re just “rare”, which can be anything from unpublished manuscripts of failed writers to drafts of screenplays that never got produced to limited run publications that never sold a copy.
idk dude. i have a rare first edition that i absolutely treasure. single run, too. it’s a local therapist’s little comic book he drew. he gave a copy to my mother when her parents were divorcing. really effective at explaining masking, just uses different words.
if someone took my copy and pulped it, i’d [disproportionate response. very disproportionate response]
That’s how life works, people come and go. We can’t remember every single person that’s ever lived forever, but people can have keepsakes of those important to them in their lives.
Its not sad that more people didnt get to make that specific connection, because there are countless opportunities to connect with people now.
I think we would be better off forging human connections with those around us than trying to make those connections with the remnants of those long passed.
The bible is also a rough example, because its been modified and translated over the last 2000 years. I’d argue there isn’t much connection to those who originally wrote it, and I don’t think the intent of religion is human connection anyways.
i have a rare first edition that i absolutely treasure. single run, too. it’s a local therapist’s little comic book he drew.
Cool, I guess. How many of these did the AI company destroy?
My niece wrote a ten page short story, had it bound, and added it to her school’s library - a project every kid in her grade level did. If she doesn’t claim it before she graduates, it goes into an archive. And eventually on to a dump.
if someone took my copy and pulped it
If they snuck into your room at night, sure. If they walked off with it from a yard sale, idk if you get to complain.
Okay, now consider my nonexistent idiot sister (my sisters are all cool as fuck) stole my copy, how family does. And I ran the first search in a decade for this book, and the rare bookshop that got pulped last week has (had) the only other extant copy. And then my nonexistent idiot sister died in a fire with my book. See why I might be mad at this straw man?
It depends on why the books is rare, some books are rare because they are not popular while other because they were banned. For example I know a novel were the English translation is easily available, but the original language is rare because it was banned in the original language in multiple countries. Don’t forget collecter artbooks that were never digitised.
Lots of them are uninteresting and useless to 99.99% of people IIRC (examples I saw in a French article a few months ago was very precise history of some random villages in Spain or something similarly ultra specific)
When destroying books, we don’t actually know what will be important in the future. That’s why archivists spend so much time archiving mineutia without trying to curate. Those are different jobs and you can’t curate for the future.
Yeah I’m not saying these books are objectively uninteresting and useless and that they should be destroyed to feed some water-hungry plagiarism machine, but that is why they are not being re printed : barely anyone is buying them (despite them being rare)
Im assuming for pre digital era books, they don’t have the equipment (printing plate) anymore. But yes, presumably anybody could just print the scans with a modern printer?
Keep in mind that books often become culturally important decades after their print run, sometimes when the copyright is ambiguous.
And also keep in mind there are technically, an insane number of “rare” books out there, but important rare books might only be important to people without the money to pay for another printing.
I could almost forgive this prrocess if they were /also/ archiving these books (leven if limited ike Google books does), but they won’t, because they are so focussed on staying ahead of compeditors based on the “knowledge” in their training data.
Well, I’d be surprised if they’re throwing away the digitized versions, they will want to use them in training all of their future models. If they go bankrupt maybe someone more willing to share will get the data
These companies need quantity over quality. They’re gonna buy the cheapest books in bulk they can.
Contrary to what some might think. I’m not taking a stance for AI. I’m just pointing out, no one cared about these books 1 month ago. No one wanted to buy them. No one wanted to read them. That’s why they’ve been sitting on a shelf in a warehouse collecting dust.
They’re not valuable in any sense of the word. They’re only “rare” in the sense that they sold so poorly, there was no point in printing more.
This is such a silly thing to glue on the hate-train. “They’re destroying books no one wanted to read!” Oh no… anyway
Popular doesnt always mean good. Unpopular doesnt always mean bad. There are countless little poetry books that only circulated through clubs. Texts circulated through writing groups that contain works of authors before they were famous.there are rare books you’ve never heard of that a collector would pay thousands for because they are one of the few people who know its value. Old folios that crossed oceans as the libraries behind them burned in petty wars that sat in attics for generations before the family found it only to send it up wholesale in an estate auction.
The point is that these assholes buy it all up by the truck load and process it indiscriminately. All to avoid copyright, and to feed their environment destroying hallucination machine.
Yeah, when I had a library close by I’d go all the time and find books I’d never heard of to read. It was great. Just because something doesn’t have a publicst pushing it on everyone doesn’t mean it’s not good or valuable. These ghouls destroying everything to feed their slop machine disgust me.
You’re doing an awful lot of speculating. The texts circulating through clubs, are circulating. These books were not.
Do you have any idea how many books we casually destroy? Books that never got sold, that are just wasting space for the next new book. The only difference is that unsold books being burned for energy isn’t manking the news. It’s just another thursday
Actually im not. I deal with old books every day. What youre talking about is the modern publishing space where books get pulped by the millions because they are textbooks that weren’t picked up by enough schools. Or theyre no longer the latest edition. Even regular consumer books get pulled for not having enough sales.
The used market is still full of these books even after being abandoned by the publishers. But even that market is still driven by demand. Used bookstores only buy books if they think they’ll sell. But the thing is they still bulk sell those books that dont have market value because people will buy huge pallets of books for pennies on the dollar.
Yes libraries also recycle books that have been on the shelves too long. And it’s hard to sell old library books because of the extreme use they are put through. They do library sales for old books pretty regularly. Most of the time they are pulping books that are worn out. Which still suggests a demand, worn books are read books.
What they’re doing is taking old texts from warehouses that are known to be abandoned copyright, since stealing modern writing has proven to be unpopular. And so unknown old works are basically free game. And at best they are saving it digitally, which is good, but then selling access since after destroying the physical copy so as to have a monopoly on the data. And moatly theyre selling it to ai companies to use as training data. So basically theyre squatting old rare text that they have no actual right to.
Buddy, how many books do you think are written every year?
If you started today. And did nothing but read until you died. You would not even have finished all the new books that were published last year.
Point being. There’s an enourmous amount of books printed every year. Most of them doesnt sell very well. So they are sent back to the publisher. And the publisher does not have room to keep every copy. So they’re destroyed. Maybe recycled into other paper products, maybe burned for energy. Regardless. What AI companies are buying is but a drop in the ocean.
And it has nothing to do with people “not reading”
It has everything to do with not reading you muppet, any literate person understands that you don’t need to destroy a book in order to gain the information from it. A child could understand that there’s no reason to destroy the books other than laziness or malice.
To scan every page manually is going to take hours. Where as if you remove the spine and separate the pages, you can feed it automatically in minutes.
And since they want to process hundreds of thousands of books… well, you’re literate so in sure you can do the math.
They’re just books…
And again. For the fifth time probably, publishers destroy more unsold books than these companies. And have been doing so for the past 50 years.
No one seems to care publishers destroy old unsold books. But as soon as it’s AI companies then all of a sudden it’s a crime against humanity.
No, publishers don’t go to the stores. The stores send the books back to the publisher.
What are my arguments? Can you even tell me?
Because what I’ve said. Is that we already do destroys millions of unsold books. And the various filings, rapports, inquiries, and fucking weather data, isn’t sold at bookstores to begin with. So the risk of those being bought and destroyed are zero.
So… you have not cared one bit, that publishers destroy unsold books for well over 50 years. But when these companies do it, it’s somehow terrible.
“Book burning is good actually because no one reads them”
Summarized by biological human intelligence
Typical fucking anti-human attitude. If it doesn’t make money then it’s useless.
You do know that books (the majority in existence probably) aren’t meant to be sold or read by the general public but? Like historical ledgers, accident inquiries, patents, scientific findings, family trees, weather data?
I’m sorry you didnt know the vast majority of unsold books are destroyed and recycled into something else. What did you think we did with them? No one wants to store 10’000 unsold copies of Robert Holms, “The straight circle”
What they’re buying are the cheapest literature they can find. Old unsold books no one wanted for decades.
There’s not a single scientific finding we have not digitalised, which you can buy the originals of. We’ve digitalised every single patent filed. Ever.
And weather data? Seriously? You think they’re buying weather data published on paper only? Where do you even buy that?
“I personally have mixed feelings about all of this,” one small bookseller told tech news website 404 Media last week.
The bookseller explained that in April, he suddenly went from selling around 20 books per week to hundreds — including obscure and out-of-print works.
“It benefits me financially as well as by clearing out old inventory that is otherwise unlikely to sell,” he said. “I’ve been well suited for these sales with inventory from overseas and foreign language books. On the other hand, I don’t like the end-use, and I don’t like that uncommon books are being pulped.”
Then stop selling rare and out of print books to an AI company, you fucking doorknob.
Combine this mass burning of humanity’s canon with the onslaught of AI generated nonsense and think about what is doing to do our history and knowledge going forward.
Humanity is being dragged into a black hole by tech-fetishist demons.
It’s a second dark age approaching. It’s slow, maybe not in our lifetime but with the enormous pressure of climate and demographic crisis add the regression of human cognition and destruction of researchable truth and you will get a collapse of the global economy and a shift in priorities (survival first) that will bring a dark age.
To be honest, I’d be MUCH less against this practice if this shredding at the very least included saving the original scans at at least 300 dpi for everyone to access, freely.
Ideally, they wouldn’t cut them by the spine, but even giving the despined ones to an archive/library would be just above the unpermissible line.
Despining, scanning and public access archiving is borderline permissible.
Also, if every AI company does this, then it’s more books gone. If they cooperated they’d only need to do it once. Good scans of all books in the public domain would be so useful for us all as society.
While that would be nice, unfortunately it would almost certainly run into the very same legal issues that are also causing them to pulp the books after scanning them.
IIRC destruction of the physical copy to prevent resale while still possessing a copy is required for book scanning to be free use, or otherwise not encounter problems with copyright law. Proceeding to share/distribute those scans, or especially trying to charge money for them, would encounter those exact same copyright barriers.
I would assume they aren’t bothering to scan it if it’s in the public domain. Much easier to just grab a copy off archive.org or any other website hosting such things, and that’s assuming the entirety of the public domain isn’t already in the training set.
Crazy how we had all this information available and all we did was lock it up and make it for sale. The internet was an awesome place we shared ideas easily and without massive corporate “sponsors” now the owners of the internet are the corporations and they are just riding t till the wheels fall off.
doubt many ‘of the most valuable’ books haven’t been digitized or aren’t available on amazon or ebay for a pittance. when i need to find something on anna’s archive, it’s usually because it’s some bullshit, tween pulp romance novel from the late 70’s.
There’s a lot of incredibly old books that can’t even be opened to be read without damaging them; but those are already locked up in museums and shit for study and preservation.
