GrapheneOS imposes a set of restrictions on how generative AI can be used in our project and community. It isn't forbidden but rather we imposed rules based on actual issues.

Posts made in our forum, chat rooms, issue tracker and the rest of our project must be written by humans. There's an exception from this rule for translation and accessibility features. That means commit messages, pull request messages, bug reports and feature requests are expected to be written by humans, not a machine.

Replying to @⁨GrapheneOS@grapheneos.social⁩

We largely disallow making posts entirely based on AI model output. For example, we get a lot of forum threads from people with concerns based on AI model output and will typically provide a quick answer followed by locking the thread and then eventually removing it.

It isn't permitted to submit pull requests or issue reports created by an AI model without fully understanding it. It's fully allowed to file issues found by an AI model if there's a human who understands it and writes the report.

Replying to @⁨GrapheneOS@grapheneos.social⁩

We do not want issue reports entirely generated by an AI model or which were written by a human and passed through an AI model to provide analysis. It's usually incredibly inaccurate and we would greatly prefer the user's explanation of what's wrong. We want a description of what happened, a crash traceback if one is available and optionally a subset of logs emailed to logs@grapheneos.org if people are comfortable doing that after triggering the crash or another issue. We sometimes request it.

Replying to @⁨GrapheneOS@grapheneos.social⁩

It can sometimes be useful to analyze an issue with an AI model but it usually isn't. It also definitely isn't useful to do it with a free or cheap model available without either paying for it or running it on very expensive hardware. In general, users do not have access to anything useful to analyze it. We don't want people to keep converting their issue reports into slop. We want what a user pasted into the AI model interface rather than what an AI model produced as output from that.

Replying to @⁨GrapheneOS@grapheneos.social⁩

Frontier models are very good at finding bugs but it requires an experienced developer to babysit it and extract the useful portion of the output. Important bugs are often found but mixed in with a bunch of low quality output. The details on the issue and especially proposals to fix it are often wrong. It's still very useful and finds many real issues. The volume of real vulnerabilities being found this way in Linux, Chromium, Firefox, Postfix and countless other projects is quite scary.

Replying to @⁨GrapheneOS@grapheneos.social⁩

@GrapheneOS You guys have made it very clear that you find frontier LLMs useful for this specific purpose, and that LLM contributions that are slop are not accepted. I trust you to know what is best for the future of the project I've followed since the Copperhead days.

However, my main concern regarding this is the legitimization of a very environmentally damaging service (not to mention the direct support of fascists) by endorsing its use.

Replying to @⁨theserpentguy236@mastodon.social⁩

@GrapheneOS
1. To what extent did you use LLMs in both the coding and review process? What are the lines you are not willing to cross?
2. How much did you pay for the models, and to which company/provider did you pay it to?
3. Are there any considerations on hiring more dedicated vulnerability researchers instead of spending on LLMs?
4. How do you justify the harms stated above?
5. Finally, how do you plan to mitigate the harms of using LLMs? Are there plans to use it locally, for instance?