DrWhax@drwhax@infosec.exchange

I know Mastodon hates LLM's and AI. So here goes!

I recently got access to trusted access of cyber capabilities of both openai and anthropic, which also allows you to weaponize security vulnerabilities.

The speed at which these parrots can find bugs and be creative enough to exploit them is staggering.

I recently pointed an LLM at an kernel fix that was reachable by an unprivileged namespace on Debian and it fully weaponized it, without too much me prompting it in the right direction, in about 7-9 hours.

I don't think open-source and companies will know what's coming for them once these open-source weight models will have broader reach and get better at exploiting vulnerabilities on a massive scale as anyone can access them.

The bottom line I think is, you cannot patch faster than the attackers can easily chain all kinds of vulnerabilities together and just move laterally on an incredibly fast pace.

I've started reporting vulnerabilities to all kinds of projects and the majority have trouble or patching issues found. There's not enough maintainers, or there's simply none anymore.

I've been getting quite worried about what our future will look like for data privacy. I think outright not running an LLM over your codebase to find critical security vulnerabilities because of your moral stance will keep us more insecure.

Please run an LLM over your code base if it's internet facing or something critical, we thank you!

Can't wait for the discussions on this!

Richard Hughes@hughsie@mastodon.social

Replying to @⁨drwhax@infosec.exchange⁩

@drwhax I'm paid by Red Hat to do this, and am given access to all the resources I need -- but it must be soul destroying for an open source contributor just doing OSS for fun as a hobby.

I keep telling myself that all this security stuff isn't much fun, but it probably makes open source stronger in the long term.

I think it helps to have very strict rules upstream for the flood of AI reports, e.g. github.com/fwupd/fwupd/blob/ma

github.com/fwupd/fwupd/blob/main/SECURITY.md#reporting-a-vulnerability
A system daemon to allow session software to update firmware - fwupd/fwupdGitHubfwupd/SECURITY.md at main · fwupd/fwupdA system daemon to allow session software to update firmware - fwupd/fwupd
Raven667@raven667@hachyderm.io

Replying to @⁨hughsie@mastodon.social⁩

@hughsie @drwhax I think a lot of projects aren't clear where they stand between professional and hobby development and how much responsibility they should be taking on. Just because you donated some work to the world as FOSS and a company picked it up and distributes it as part of a product, doesn't make you responsible for the "supply chain" your hobby project finds itself in. IMHO any company which ships FOSS (which is _all_ of them) needs to vendor their dependencies, or have a contractual agreement for maintenance, and not just rely on the goodwill of the original author or volunteers for things like security, QA / CI testing, etc. I feel like a number of devs have gotten suckered into a culture of emulating commercial professional development (using an issue tracker, reviewing merge/pull requests, having numbered version releases, using CI and test suites, etc.) for things which are fundamentally non-commercial hobby projects and end up deeply unhappy burning their nights and weekends doing software maintenance because "people depend on them", like a martyr, and it feels rude to tell those people to get stuffed.

I think a fundamental way we should structure human systems is to limit the responsibility (and correlated damage) any one person can have on their own because our poor little meat-brains aren't really built for it.

JTI@jti42@infosec.exchange

Replying to @⁨raven667@hachyderm.io⁩

@raven667 @hughsie @drwhax EU is attempting parts of this. We'll see about how the realexisting version of this will turn out.

Other than that: Ensure awareness for said corporates by enforcing the existing rights on the components they use.
There'll probably have to be some sort of enforcement threshold or staggered requirements.

And a large part of it is probably to just answer to requests for features/fixes/... with "pull request welcome" or "here's my hourly rate" to any corporate requests.
Unfortunately, this is not trained for.