It’s not free; doing so provides Anthropic with free access to training data for its models.

I think it’s safe to assume that any public code on the internet is already in their training dataset.

At this point anything that is public in general, probably all the posts in the “fediverse” have been taken already

probably all the posts in the “fediverse” have been taken already

I don’t doubt this, as it’s trivial to do (by design). You don’t even have to scrape anything.

New posts and comments in a Lemmy community (or Mastodon profile or any other ActivityPub-based Fediverse app) get pushed to every server where federation is allowed and at least one person is subscribed to it. You can spin up your own server, subscribe to a bunch of communities, wait a while, and over time your PostgreSQL database will be populated with profiles, posts, comments, etc.

The code they’re analyzing is free and open-source though, so they can already use it this way. One of the freedoms of free software is that people can study it and use it however they want, for whatever purpose they want.

Isn’t it good that they’re sharing data with the projects instead of just keeping it for themselves?

One of the freedoms of free software is that people can study it and use it however they want, for whatever purpose they want.

Given that Anthropic and OpenAI derive their profits from their models being closed source rather than open source, I don’t think they should be assisted in devouring freely-accessible data in their endeavor.

Not all developers may see it this way, but those contributing code to LLMs are training them for free at the expense of career opportunities for fellow developers, all the while Anthropic, OpenAI, and other AI companies rake in money for maintaining access to content that isn’t theirs.

I don’t think they should be assisted in devouring freely-accessible data in their endeavor

How is this assisting them, though? I see it as two different options:

1. They use the open-source code without giving anything back; or
2. They use the open-source code and help improve the project by reporting security issues.

In any case, a fundamental feature of both free and open-source software is that anyone can use it, regardless of if you like them or agree with them or not, without discrimination. You could instead use a license that forbids AI training, but then your code would no longer be free or open-source.

One of the freedoms of free software is that people can study it and use it however they want, for whatever purpose they want

Open source still has licenses, this is incorrect. They are breaking hella licenses left and right. Sorry for the necro, tho.

In return your code is theirs.

and they haven’t already scraped the whole of OSS already?

Assuming yes, it would’ve been outdated, and this give them access to fresh new code for a foreseeable future in the case of Microsoft blocking access from other AI to make it exclusive for Co-Pilot.

They don’t have to offer this to get new versions, that’s kinda the point of foss software, it’s there for anyone to download and use however they like whenever they want.

Hot how ever they like though, that would be code under MIT ant the like, not all FOSS. Where the other licenses actually fall?

No, the GPL doesnt put any restrictions on what you can do with code, just on that code and derivatives of it have to be released (source code included) under the GPL as well.

The courts have tended to find that training LLMs on data you have legal access to is fair use and transformative.

Loling at the previous comments here… As if AI labs don’t already vacuum all open source code they can find

It‘s pointless. The pace at which games are decompiled with AI in the modding community tells me that each and every software will be figured out and broken into within a few years or even months. The digital world will be turned upside down. All data could be leaked, entire hedge funds and banks could get wiped with a mouse click and governments could be toppled over night when pension funds of millions suddenly disappear. The AI ouroboros is a vortex that could swallow the entire internet including everything we put in it. Moving fast and breaking things may just break our way of life forever.

Things take years to build, minutes to erase.

De-compiling isnt magic that lets you do whatever you want with software. It just works well for games because DRM has the insane model of “we give you the encrypted payload and the key to unlock it, but you can only unlock it in the way we want” which is fundamentally insecure. Most software isnt like that.

Wait you mean just because I find the deposit api for my bank I can’t send it a fake 100m deposit into my account? 😜

“The outputs of this opt-in vulnerability scanner will be fully model-generated, without human review or triage,” Anthropic explained. “This will enable faster and more frequent scanning, but means that it is possible reports will be incorrect or invalid.”

When I saw the headline, I was wondering about this specifically. This may make this service not super useful.

My experience with AI security reviews is that they’re fantastic at finding faults, but they always find a list of things to complain about. If there are no real/serious faults they’ll start finding things that kind of have the same shape as a security issue, but really aren’t if you dig into them. I’ve regularly had an LLM generate a list of 10-15 issues ranging in severity from “nits” to “critical” where none of them were actual issues.

Periodic reviews seem like they could get annoying really quickly, becoming more of a maintenance burden than a help.

AI security reviews is that they’re fantastic at finding faults, but they always find a list of things to complain about.

Same experience for me. At work, I always review the output because a lot of times it finds something that can be noteable but not relevant to the scope of what is being reviewed. Even with guardrails like instructions to not go beyond the stated scope, a human absolutely still needs to read and check the findings at the end.

Yeah. I did use an open weight model LLM to scan an app that I use and it came up with a critical severity issue and I checked on it first to see if I could reproduce the problem before ever submitting it to the developer as a problem.

Turns out it was actually a legitimate security problem that has now been fixed.

I am sorry, to enroll a primary maintainers need to open a Pull Request with a Yaml file containing their email address?

And people do that? Here is an example PR: github.com/anthropics/oss-scanner/pull/…/changes

Did Anthropic just created a largest ever publicly available collection of email addresses of primary maintainers of OSS projects matching this criteria

established projects that have a critical impact on infrastructure and user security

(Quote from Anthropic)

Is it an invitation for every bad actor to scrape pull requests of this single repo, extract email addresses and target those with every fishing/hacking/account takeover attack imaginable?

Is not that insane?

They are hungry for active projects, want to distill better datasets.

It’s free because they intend to steal your project if you’re dumb enough to let them scan it