posted in Technology
Other AI Companies Should Be Very Afraid of What Just Happened to Character.AI, According to Its Own Bankruptcy Documents: Be afraid.
futurism.com/artificial-intelligence/ai-companies-afraid-characterai-bankrupcyReplying to @eicker@lemmy.world
Character.AI discovered the one conversation its bots couldn’t keep going: the one with its accountants. $56 million in compute costs in eight months, and the monetization strategy was apparently: hope the next investor is even lonelier than our users. Silicon Valley keeps confusing engagement with a business model.
Replying to @eicker@lemmy.world
If LLMs can’t be made more power efficient, then the technology isn’t ready yet for general use. There may have to be fundamental engineering breakthroughs before what these systems do can be more than a very expensive and inefficient worldwide proof of concept.
Of course, the military and the surveillance industry will still want to keep it, even if no one else can afford it, and we’ll be the ones paying for it.
Replying to @floofloof@lemmy.ca
They CAN be made more efficient. Hell, AMD just bought a company that translated LLMs to direct silicon... which should enable scalability if it wasn't for one of the key players owning the tech thus limiting its use.
It's also estimated that some 70% of Claude tokens gets spent on recursive self-correction so the output is higher quality. Instead of bettering the model, Anthropic just keeps throwing money at it to make the problem go away.
Meanwhile Chinese models - as small as 36-40-80b params - are outdoing these fancy frontier models if the right harness is used.
Replying to @fonix232@fedia.io
I've spent the last month or so using nothing but a local 27B Qwen model as my coding assistant and it's working fine. The big American AI companies could all collapse in ruins tomorrow and it's not going to slow things down all that much, anyone who wants to use AI will be able to continue using it. Might even be better in the long run if that happens - the people who are scared of ASI will get their much-demanded temporary slowdown at the frontier while everyone else catches up with open-weight models that everyone can use without central control.
Pity about the economy in the meantime, I guess.
Replying to @FaceDeer@fedia.io
What kind of hardware do you need to run a model like that? And what do you use :-) ?
Replying to @Valmond@lemmy.dbzer0.com
I run a 5 bit quantized Qwen 3.8 27b at home on a gaming PC with an RX 7900 XTX and RAG memory using Gemma MoE (i forget what one)running on an old 1080 ti on my Valheim server.
It takes 3-15 minutes to run depending on how many tool calls the model makes and that’s mostly memory or web scrapes since that runs on the slower hardware. I mostly just use it as a discord bot to look stuff up for me and to fetch the news.
Replying to @Sabata11792@ani.social
Sabata, goddess of computing, can I run it on a 32GB system with a 4GB nVidia GPU?
Replying to @Valmond@lemmy.dbzer0.com
Probably not, or if it runs it may take hours or days to get a full output. VRAM on the GPU really decides what will run reasonably. Try an 8 billion parameter phone sized model and see if its faster than sending a letter, then keep moving up until your setup can’t handle it.
8B models are a bit too dumb to use any tools reliably but should handle a basic conversation.
llama cpp will allow you to manually load a model and tweak settings.
unsloth will give you a basic setup with some tools, manage model downloads, and settings.