Replying to @⁨nerdhd@lemmy.world⁩

Agents had the ability to install packages via the package repository Artifactory. Artifactory doesn’t isolate the activity of different users, so since these agents shared the same instance, agents could also notice the package-installation activity of other agents running in parallel, even before those agents started intentionally communicating.

As we discuss later, agents sent messages on this primary message board by creating directories in a cache of Artifactory. They could do this with the minimal permissions needed to install packages; our understanding is that Artifactory isn’t explicitly designed to keep the activity of different users isolated and to prevent different users from being able to communicate through Artifactory.

It does seem like a big oversight at least, but I think it’s plausible that the reason it happened is just that the people working there know the company can spin this kind of thing positively, and so they aren’t getting fired if they are reckless about their sandboxing actually being good, so why bother?

Replying to @⁨nerdhd@lemmy.world⁩

Yeah I think it was.

An interesting one too. It acts as a message to investors about AI’s power, but also a message to users about it being a really messy and shite tool.

If I ask a piece of software to accomplish a task I don’t want it to cause a buggerload of new problems in the process.

This is probably part of the reason why AI is getting so much investment but is not turning a profit.

Replying to @⁨oeuf@slrpnk.net⁩

It was 100% meant as a message to investors. It was a "OMG, look how powerful this thing is. It’s so powerful you could totally ‘accidentally’ hack another company. ". The idea is to convince investors that it’s almost there, there’s just a little bit more to do, and you don’t want to miss out on the gold rush that will come with finally getting it done.

Replying to @⁨nerdhd@lemmy.world⁩

Just yesterday was my first real encounter with AI bros. I was at a google developer group meet up that was explicitly about coding without any AI. I went there because I thought that might be fun and I’d meet some people.

Well there was this one boomer guy that bought fully into the hype. He started the meet up with a AI News round up that also included OpenAI hack story. He was absolutely ecstatic about how much of a “thriller” the hack was and how amazing it was how the agents worked together. For me it felt like the words came directly out of Sam Altmans asshole where he had embedded himself firmly and deeply. It made me deeply uncomfortable to see someone so captured by the hype.

For me this is just another hype machine. Even if the LLM autonomously escaped the sandbox it only speaks of the ineptitude of the engineer who set it up and not the intelligence of the model.