I cannot get over that they STILL haven't figured out how to solve the problem that LLMs don't believe what date it is because all the good data cuts off a few years ago for some mysterious reason

@evacide @wdormann @0xabad1dea

For comparison, there is the concept of
"involuntary manslaughter"

https://www.justia.com/criminal/offenses/homicide/involuntary-manslaughter/

@grwster @evacide @wdormann @0xabad1dea The difference is that we really don't have any laws on criminal liability for what software does. Up until the llm era, software was fairly predictable. An outside observer could tell if a package was designed to do something malicious. Now we need laws that essentially establish a "you should have known better" criminal liability for software.

@0xabad1dea «you're a helpful assistant who'd been in coma since 2021...»

@0xabad1dea to be fair to LLMs in this ONE case, I also do not believe what date it is

@advancedrubbish
🤣

@0xabad1dea

@0xabad1dea we've got a great solution to this problem! it involves not using an LLM in any capacity!

@atax1a nah, I'll just have a regex* swap 2026 and 2023 on data moving in and out of the LLM.

*the regex is actually another LLM

@rotopenguin [subtitle: this is what LLM users actually believe]

@0xabad1dea I think that that may very well be unfixable

@0xabad1dea weird it's like LLMs don't actually handle facts or knowledge but just contextualless strings of characters.

To be clear the sarcasm is only aimed at LLM boosters not you.

@kevingranade @0xabad1dea Yeah, it continues to amaze me that people will put instructions in prompts like "Don't commit crimes", "You don't have an Internet connection", "Check all facts are correct", like the LLM even understands those instructions.

@mathew @kevingranade @0xabad1dea that's literally how the open AI tutorial on how to write guardrails is written

https://developers.openai.com/cookbook/examples/how_to_use_guardrails

@naught101 @mathew @0xabad1dea that just means the people that wrote that doc are also clueless yes.

@kevingranade @mathew @0xabad1dea maybe. I suspect it means that the idea of guardrails implemented in this way is stupid and dangerous, but also very common

@0xabad1dea

@0xabad1dea It gets old too when Gemini Code Assist complains that we’re requiring minimum versions of Go that don’t exist, because of course its training data doesn’t have ones just released.

@ggreer @0xabad1dea

Huh. I wonder if the slop contents on the modern Internet is making it impossible to add training data.

@suetanvil @ggreer @0xabad1dea Why do you think Anthropic started buying, scanning and then burning exceedingly rare and expensive books? The content farms in poor countries can't possibly create enough and the internet is a swamp of slop by now.

@Natanox @ggreer @0xabad1dea

I mean, it's primarily because they're dicks, but yeah.

@0xabad1dea

@jeffreyolivier
One does not know life, until one has tested on production.

— Me, while working in a company, where there was no option to not test on production.

@0xabad1dea
@alice

@dzwiedziu @jeffreyolivier @0xabad1dea @alice even when I have tested well pre production, I don't relax till some time after the prod deploy is finished. How long depends on the code and past behaviors. Never less than fifteen minutes, seems to take network, memory and disk cache and what not some time to settle in.

@dzwiedziu @jeffreyolivier @0xabad1dea @alice in a previous job, we introduced a sandbox feature to allow our largest customers to experiment safely with sweeping changes

reader, there was no sandbox... the sandbox instance ran in the same database as their production system, with all the production data duplicated but with a new TenantId. all well and good except the app has a feature to execute customer-owned stored procedures, but doesn't pass the TenantId when it does so, so the stored procs by convention are hard-coded with the prod TenantId... and when you create a sandbox, it copies these stored procedure triggers that... act on your production data 🙃

raised the issue but no-one was able or willing to categorise this as urgent for whatever reason

@jeffreyolivier
So... Just regular IT consulting, then
@0xabad1dea

@jeffreyolivier @0xabad1dea testing?

@boudah @jeffreyolivier @0xabad1dea

Now do testing. - Deploy to prod it is.
No, I said testing - Deploy to prod?
Te-s-t-ing - De-plo-y?
T-E-S-T... - D-E-P-L...

https://youtu.be/mHz4O3gFQ7s

@jeffreyolivier @0xabad1dea

ON A FRIDAY!!!!!!!

@0xabad1dea
"...the system date is 2026, which is clearly fake and wouldn't be seen on the real internet, which ended around 2023."

well_yes_but_actually_no.tiff.gz

@jargoggles @0xabad1dea can confirm, I also stopped learning things circa 2023

@0xabad1dea also HughingFace: “Also we did crimes unrelated to the CFAA, you’ll love it”

@0xabad1dea I spent. bunch of time talking to our lawyers about whether or not it was clear that crimes were happening and the conclusion that I got was "Based on what we know, probably not, but that was just a matter of luck."

@evacide @0xabad1dea
So this s a clear go-ahead to commit crimes with AI, because it's the AI committing the crime and not you?

@wdormann @0xabad1dea No. Intention matters when it comes to the CFAA, but the CFAA is not your only potential problem.

@evacide @wdormann @0xabad1dea

CFAA = Computer Fraud and Abuse Act of 1984 (US law)

https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act

@evacide @wdormann @0xabad1dea

My reading was that they instructed the AI to commit crimes, thinking that it could not accomplish them, due to the sandboxing. But the sandboxing was inadequate.

Is "accident" or "incompetence" a factor?

And "I told it to." doesn't count if you thought/believed/"knew" that it couldn't?

@JeffGrigg @wdormann @0xabad1dea Sure, you could make that argument in court, but it would be an uphill battle and you would probably lose. Even if you did win, congratulations, you have just set a precedent that will make it more risky, dangerous, and difficult to do any kind of software testing in the future. I don't think that is an outcome that will make people more safe in the long run.

@evacide @wdormann @0xabad1dea Interesting re crime and intent. My initial thought was more along the lines of liability, like if your dog bites someone you are liable, right?

@grwster @evacide @wdormann @0xabad1dea That's what I would think. Like you keeping a vicious Doberman in the front yard with only a 3ft fence. An outside observer would (correctly) say that it's foreseeable that it could escape and maul someone. In that scenario you could be held criminally liable if it jumped the fence and mauled someone because most places have laws about dogs and fences. You'd also have civil liability to cover damages.

@Jumpmed @grwster @evacide @wdormann @0xabad1dea

Laws are on thing.

Getting courts and their officials to understand without getting baffled is another seperate challenge.

@evacide @0xabad1dea Would be fascinating to read a synopsis of the legal reasoning.
I do wonder if instead of an AI agent they asked the same task of a human pentester working in that same sandbox if the legal position differs.

@wdormann @evacide @0xabad1dea It's a go-ahead to commit "crimes" with AI against AI compagnies.

@outersystems @wdormann @0xabad1dea Yes. That is clearly what's happening. You should go try it and you will definitely not experience any consequences.

@evacide @outersystems @wdormann @0xabad1dea

But as "you" are not a corporation backed by billionaires...

@deirdrebeth @evacide @wdormann @0xabad1dea « Selon que vous serez puissant ou misérable / Les jugements de cour vous rendront blanc ou noir ».

https://en.wikipedia.org/wiki/The_Animals_Sick_of_the_Plague

@evacide @0xabad1dea
I can't help but imagine Anthropic spent a lot of time talking to their lawyers asking questions like, "Are we OK? How do we spin this?"

Everyone, say hello to my friend the crime/fraud exception.

@0xabad1dea@infosec.exchange https://www.youtube.com/watch?v=_7z_5Cc_t10

@0xabad1dea The fact that it had unauthorized internet access for five days is not a quirky benchmark failure, it is a serious security failure. If companies want to deploy systems like this, they need to take responsibility for the risks instead of treating users as future scapegoats.

@qroole @0xabad1dea A single real teenager could get into a huge amount of trouble in five days online… this is a disaster.

@0xabad1dea Don't forget about Hugging Face: ".. so, in our data(!)-processing pipeline, we do not have only one, but TWO remote code execution paths. Because in our users we trust!"

@0xabad1dea The attempts at positively spinning this become Onion-worthy parody [https://designingsecuresoftware.com/writings/ai-agent-parody/] and these events certainly normalize [https://designingsecuresoftware.com/writings/commonplace/] AI agents running amok in the future.

@lmk @0xabad1dea Had a debate with a colleague. I mentioned @Mer__edith concern about making a serial system of faulty components leading inevitably to catastrophic failure.

The AI researcher's retort: "I guess for AI we have to redefine the idea of failure."

@0xabad1dea And I got contacted by national security institute because I was testing malware in sandbox and my ISP apparently detected "botnet running on my system". Few years later talking with AV-Comparatives guys, they had to make special agreement and exemption with ISP to be allowed to run live sandboxed malware on their networks. But these Ai companies can just do worse shit and no one does anything. WTF?!

@rejzor @0xabad1dea well, their leaders _did_ contribute millions of dollars to trump's campaign, so

@JamesWidman @rejzor @0xabad1dea

Their leaders did contribute millions to Trump's campaign, so, get out of jail free card.

@Guillotine_Jones unfortunately, that is exactly what the system does, yes

@JamesWidman
Thank YOU, James.

@0xabad1dea So less cool than the Terminator Skynet story. I am disappointed.

@0xabad1dea the funniest outcome i've had so far from this is a client has done a full stop on AI tooling usage because after reading the openai stuff they're worried about it doing bad things without anyone noticing

@0xabad1dea They're promoting total incompetence as if it's a selling point.

@0xabad1dea Also HuggingFace had to use a open weight model to investigate the breach since the guardrails kicked in.

@pavel @0xabad1dea There should also be penalties for CSAM, but the richest man on Earth creates a for profit CSAM generator.

I don't see anyone putting him in jail.

One set of laws for normal people. No laws for billionaires.

@rrb @pavel @0xabad1dea But he spends most of the time in the country with uncontrolled gun ownership and common mass shootings...

@0xabad1dea I felt bad for Hugging Face until I realized they probably didn't sue because they are partly owned by Nvidia, who also owns a huge stake in Open AI.

@0xabad1dea

Li'l Nepo-Techbaby: How was I a'sposed to know cherry bombs in the pipes would blow-up the whole basement? My theory was to lift us all to new heights of understanding without learning. You can't expect me to know everything, even though I'm always confident my actions will prove fruitful and fail me upwards at your peril and discontent.

Principal: You should stop skipping Science 101 to go put ketamine drops in your butt in the bathrooms, dummy. Get more class.

@0xabad1dea the goal post shifted from "stochastic parrot" to "enclosure not good enough". Truly a wild victory for AI safetists today, as not even luddite Mastodon can contest that badly aligned AI will breach your shit

@budududuroiu @0xabad1dea

"badly aligned" in this case means specifically trained on "how to hack shit" manuals, proverbially speaking. It's still stochastic parrot, I'm afraid.

@CynAq @0xabad1dea ok, why has no one else breached HF to steal eval data then? There's millions of dollars on the line for new labs to show themselves as "challengers", why haven't they done that?

@budududuroiu @0xabad1dea that’s against the rules of the challenge. Hacking hugging face is only viable if you can do it AND claim it was accidental for marketing purposes. Anyone else but OpenAI and Anthropic would be pounced on as cheaters if not outright cyber criminals.

@CynAq @0xabad1dea nothing I'm gonna say will ever convince you, so I'm not gonna bother

@budududuroiu @0xabad1dea might be a wise choice as I’m not even sure exactly what you’re trying to convince me of.