GrapheneOS imposes a set of restrictions on how generative AI can be used in our project and community. It isn't forbidden but rather we imposed rules based on actual issues.

Posts made in our forum, chat rooms, issue tracker and the rest of our project must be written by humans. There's an exception from this rule for translation and accessibility features. That means commit messages, pull request messages, bug reports and feature requests are expected to be written by humans, not a machine.

@GrapheneOS for an OS project that refused porting to non-PIxel h/w for so long b/c other platforms didn't meet security reqs, this pro-LLM stance is ludicrous.

~all bugs found by LLMs can likely be found by other methods using the same-or-less compute resources, especially w/ the human effort "babysitting" the LLMs to ensure they don't go off the rails.

there just aren't billionaire fascists subsidizing that compute, be honest about it.

stop covering + doing PR-by-action for LLM fascists

@elle @GrapheneOS what's the relation of Google being Google and a language model?

@dasmatus

are you serious? what is the relation b/w GrapheneOS taking security so seriously that they wouldn't port their OS to h/w that didn't have Pixel's special sauce, and now being fine using LLM code + bug reports + dedicate human time to "babysit" the LLMs?

my intention is not to "bully" @GrapheneOS. we need folks like you in FOSS building rock-solid projects. I'm merely voicing criticism, and hoping the team explores other alternatives.

leave big (proprietary) LLMs in the bin

@elle @dasmatus GrapheneOS is based on Linux, AOSP, Chromium and hundreds of other upstream projects heavily using AI generated code.

Our OS development team hasn't been using any AI code generation but yet the OS is heavily using it because the vast majority of the code comes from upstream projects. Linux and Chromium are both massive projects.

Firefox is heavily using it too, so that isn't relevant to this. It's nowhere close to what we would need to use it as a replacement regardless.

@elle @dasmatus Our policy is only accepting high quality code. We've always had to do careful code review and avoid trusting submissions.

We aren't happy with the flood of slop which gets submitted to us via our issue tracker, security@ email and pull requests. However, a lot is reporting or fixing valid issues.

For pull requests fixing valid issues, we can either get them to improve it to meet our standards or do it ourselves. We often start with rounds of review but finish it up ourselves.

@elle @dasmatus In the past couple months, there has been a massive flood of valid vulnerabilities being discovered by frontier AI models. Nearly all of those are from human written code, often from many years ago.

Linux has heavily shifted away from adding new features and improving performance to fixing the bugs found by these models. They're being forced away from sticking with C and a monolithic kernel design. It's on ongoing crisis and people can regularly grab working exploits off GitHub.

@elle @dasmatus Cellebrite and other exploit development companies are having growing success exploiting all kinds of targets. They're openly talking about how they're using AI tools for reverse engineering, vulnerability discovery and exploit development.

It's a very large and rapidly growing short term advantage for attackers. The consensus among security researchers is that it's short term because vulnerabilities are being rapidly found by defenders too including before merging or release.

@elle @dasmatus We have a responsibility to protect our users. We're aware frontier AI models can be used to find many vulnerabilities. We made several accounts on cheap plans with a fixed amount of usage, applied for and received cybersecurity access and began testing it. We've found what others have which is that these tools find many actual vulnerabilities including subtle domain specific issues. It's largely beyond the scope of what traditional static analysis tools even attempt to do.

@elle @dasmatus As with anyone else, we make mistakes and there are many things we don't know about. There are undocumented behaviors of library and Linux kernel APIs. We have the wrong mental model about how some things work. We can review a pull request a dozen times with great care and still miss things quickly discovered with one these tools. We confirm it's a real issue and figure out how to fix it ourselves. Our fix can also be checked. We write small volumes of code so it's very cheap.

@elle @dasmatus Our OS development team and most of our other developers aren't using AI coding assistants in the form of an IDE. They'd need to be run in a VM as a secondary workspace with a separate GitHub account or a similar approach.

We're in a bubble where we haven't adopted it. Most people applying to work for us with the skills and experience we need use these tools as most companies are using it. The concerns we have about it aren't much different from supervising junior developers.

@GrapheneOS I disagree with you on this, without attacking you:

> The concerns we have about it aren't much different from supervising junior developers.

If you tell a junior developer that/where he or she made a mistake, he/she will learn and not do it again.

If you tell a LLM that/where it made a mistake, it will output "You're right, I shouldn't have done this" … and do it again. There is no learning from feedback.

With a human junior developer, you know what mistakes junior developers commonly make … what *you* struggled with at the beginning of *your* career … you know the common pitfalls to look out for. The mistakes of a human junior developer are obvious to more experienced developers.

What a human junior developer wouldn't do is write code that looks flawless to a senior engineer *and* to all the configured linters/tests but contains an extremely subtle bug that goes unnoticed despite careful code review. "AI" coding assistants do that, however.

Similiar to how AlphaGo plays Go in a way no human has ever played Go before, "AI" coding assistants write code in a way no human writes code, and "AI" coding assistants are thus much more likely to introduce bugs than humans.

Your concerns should be very different from supervising junior developers.