@laura_ge @GrapheneOS Hello again,

Sorry your post is a little difficult for me to read, I will repeat your question to make sure I am not misunderstanding you.

> You say more AI generated code helps you detect it more often:

I am unable to find the study so I cannot claim it to be false definitively, but I will disagree with the following reasons:

- For many projects, AI code sticks out like a sore thumb due to its weird tendencies and behaviors. If those tendencies become ingrained in the project, how would you determine AI from not?

- Since these are indeed probabilistic models, you will notice they code like a stack overflow post. It doesn't always "correctly" fit into the project the way one might expect. It gives this weird off feeling, however, that relies on the project having good coding standards to begin with. Something that can be eroded by continuous use of AI.

Otherwise, I agree with your post. AI is very much like a slot machine and I don't know what long term use looks like. This is why I am concerned that a project like Graphene is so quickly adopting it. I just don't think we have enough proof or data to show its a healthy or productive thing to include in a project intended to be this secure.

If I disregard the ethics concerns, I could give Graphene a pass for using AI to exclusively pen-test their platform (no code generation), but I think allowing any code-gen is my real big concern here.

@corigan01 @GrapheneOS It matters that Graphene OS isnt a vibe coded project, why does it matter if Graphene OS has marginal AI use if the software its based on already has major AI use in its recent months anyway? Graphene OS wont be able replace those dependencies. Graphene OS depends on Android AOSP, on Chromium, on Linux, by your logic Graphene OS should just close doors right now. Graphene OS doesnt function without the engineering resources of bigtech Google and all companies paying devs to contribute to Linux, and all those arent anti-AI and use AI in generous amounts, though for quality and reliability reasons they also arent vibe coded projects because every decent engineer knows it's not a way to make software for public consumption that works properly. It can work for some tech tools or backend software but even then it's meh. It certainly wont look good if you arent a team of sufficiently qualified engineering folks to begin with.

To stop AI, using or not in our very little privacy/free software projects wont do much, most companies etc. who have billions are already funding them all in, if we want to stop it, we must go and plug off the machines training the next models so they can't keep on with it. It's up to the people who live near enough to the places that train frontier models to do so, unfortunately where I live there are no frontier training datacenters so we can't locally do demonstrations or really oppose it meaningfully.

@laura_ge @GrapheneOS Hello again v2 lol,

I think that this is a valid concern and is something I am worried about too!

However, it is unfair to say that Graphene should contribute to the problem themselves just because everyone else is doing it. I think that while many dependencies are tainted, Graphene could still carefully manage it's own source.

We can bring up concerns with its dependencies's authors another time, however, we cannot begin without someone first taking a stance.

@corigan01 @laura_ge @GrapheneOS

Taking a stance in front of an avalanche is a really noble idea!

@corigan01 @laura_ge Linux, AOSP, Chromium and the many projects those use are not simply dependencies. That's the operating system we're building on. It has extensive use of AI regardless of what we do. Linux is also being quite lax about how it's used.

We're working on privacy and security where finding holes to break through the defenses is a key part of it. These AI models are extremely good at doing it and are heavily shifting the advantage to attackers unless that's countered.

@corigan01 @laura_ge It isn't important because everyone else is using it but rather because corporations and governments wanting to break into people's devices are heavily using it. It is our job to defend against them. People do not seem to realize how easy it's getting for attackers to the point you can go grab exploits on GitHub and take over non-GrapheneOS Android devices. It can get that way for us too if we don't stay on top of things despite how they're degrading.

@GrapheneOS @laura_ge Hello Graphene org, and thanks for taking the time to respond to me!

(NOTE: Sorry its getting a little late so my brain is a fried, but I will do my best lol.)

I understand the pressure is huge for a project like Graphene, and I too am worried about the current state of software security. Honestly, nothing could've prepared any FOSS project for this AI catastrophe. (I use Graphene myself and had trusted it to keep me safe, seriously thank you).

However, I also cannot dismiss what I've seen occur in other beloved FOSS projects, and I see this same pattern repeat over and over; its a heart breaking cycle of losing the software I once trusted. I think this is why many people on fedi get so upset when discussing AI, we've gone through this before and know whats on the other side.

I think overall I see the following things happen when projects start accepting AI:
- AI attracts the wrong crowd of developers.
- AI degrades standards over time.
- AI burns out talented contributors.
- AI leaves the project a shell of its previous state.

AI has destroyed our freedom, safety, and community. The companies festering these models have little incentive to create what is simply a "tool." These devices are cognitive hazards because they play with the very same addictive tendencies that keep people gambling, drinking, or doing drugs.

LLMs are optimized to scare people into thinking that without their magic new "tool", you will be left behind. They hook you, sing their siren song, and after all is done you're left without the very skills that made you great in the first place. You tell yourself you need to use the AI just to do the things you've confidently done for years. Its the same cycle we've all known before.

I don't believe people are upset about the pen-testing aspect of the AI use here. I think for security critical projects, one should use similar devices to those used by adversaries.

However, I think most of us are primarily concerned that allowing code contributions to be assisted with AI is ultimately harmful.

I think that if you only used AI in a very narrow way, effectively only to find bugs (like a fuzzer would) that is acceptable. We don't want to see AI replace fuzzing or other hand crafted tooling, but instead to serve as an aid to existing infrastructure due to the reasons you described.

Please, at least consider what I've said and invite more (healthy) discussion about AIs involvement before making your final choice.

@GrapheneOS @corigan01 @laura_ge it's hard for me to poopoo graphene using ai to find external threats when the Linux kernel and other OS's are burning ai programming into their systems.

@corigan01 @GrapheneOS I'm not saying that paper you are speaking is false but it's probably also not that simple, it's a strong support argument for anti-AI folks so that's why it's been mentioned alot but like all research just 1 paper isnt knowledge, we lack long term experience with those tools. Anyway I've seen this paper I know what you're talking about. I am happy with its conclusions if they are true it fits my political goals very nicely too, I find it good people are doing such research and we should do it more. I didnt say the more AI output the better necessarily, but that it's helpful to learn what those tools are like and how they work. I don't like having opinions on things I don't understand, it makes me feel like an impostor, how can you talk about something you have no knowledge of? It's much better in my opinion to make a very informed politcal critic of some tech, knowing very well how it works and not just spitting out inaccuracies and screaming out loud to progress a goal post. It makes it way more effective in my opinion if you can criticize it knowing exactly how it works and even better than many of its it's promoters if you can.

@laura_ge @GrapheneOS Hello Laurage,

You are overwhelming me with streams of thought, and I need a minute to catch up.

I am a little more tired than I was when I made the other three posts and my thought process is starting to degrade for the day, so I might take some time between my next couple responses.

I invite other nice people of fedi to come discuss in my place in the mean time!

@corigan01 @GrapheneOS I'm sorry yes I can be quite an intense writer, you don't need to answer, I'm making multiple toots to respond to 1 toot of yours because also my thoughts come up in chunks and I havent answered all points of your previous toots therefore I make another to finish as to not just ignore your side of the conversation.

@corigan01 @GrapheneOS "Since these are indeed probabilistic models, you will notice they code like a stack overflow post. It doesn't always "correctly" fit into the project the way one might expect. It gives this weird off feeling, however, that relies on the project having good coding standards to begin with. Something that can be eroded by continuous use of AI."

I don't think it's probabilistic nature causes the "off" feeling but rather if less capable models are used or if they werent given access to instructions to align their outputs with the project's code style, I was mentioning this because I think it has however impact on security use cases, that running the same input many times until saturation of substantially different outputs is relevant to pushing the limit of a model's capability for a task, specially security code review for a particular codebase, and also attention systems because of aggressive optimization that went into them for example models tend to reason more upon the tokens they were last fed and also post training instruction following bias (some models might be more trained and biased towards ignoring previous outputs more and specially ignoring all previous instructions and only follow the latest one skipping all previous instructions in context, it depends of models for that), so it can be beneficial to feed limited amount of code at a time for review so the model actually looks at all the code, etc.

I don't think Graphene OS wants AI use in any major capacity for code generation such that it would transform the project substancially in any way beyond it being human maintainable anymore as they said, and probably also only in areas where it doesnt matter for security like unprivileged apps where sometimes it can be needed to write alot of boring code etc.

Some anti-AI discourse seems to focus more on virtue of creating and using chemically AI pure software, while that's cool to keep writing code by hand for all the wonders and love of reading human code it can provide (I miss being able to go on git forges and happily read thousands of lines of code admiring and learning from it because it's written by people with good style or architecturing skills but I also do that less due to chronic headaches preventing me to feel good focusing too much on stuff). I don't believe such anti-AI virtue achieves much on a political basis to stop AI from existing or thriving while most of the people with alot of money are already largely funding it. Getting Graphene OS to be chemically AI pure software is not possible given it's dependencies and the AOSP project it's forked from already uses AI code generation.