@GrapheneOS
AI companies control what their bots do right? What if they tried to subtly place a vulnerability into graphene that your developers become too deskilled to notice? How do you know what is or isn't slop at that point? (We've seen nefarious PRs be accepted into the Linux kernel before AI, so humans cannot always be trusted to catch these types of vulnerabilities). I imagine the trust placed by the developers into these models only make this problem more of an issue.

- Who is the point of trust verifying slop from non-slop?
- Do you have extra security in place to test regressions imposed by AI models (that isn't an AI model itself)?
- What about the impending licencing issues? What if governments try to attack Graphene over illegally licenced code that was included from an AI model?

I think this is a bigger issue that needs more time to bake.

@corigan01 @GrapheneOS Ultimately, and EVEN if you want, it is not theoretically possible to prevent someone from disguising code generated by AI in ways that you would know it's written by AI, if someone wants to make you believe it's human authored it will. You will not be able to prove otherwise, there are clues and watermarking systems from major LLM providers but you can easily strip those. And even human-to-LLM communication, with sufficient guidance you can easily make an LLM generate ouput no human will ever doubt is AI generated, and with expert domain knowledge too. It's possible to do it, alot of people don't know how that's why it's easy to filter out vibe coded contributions from lower skilled engineering folks, but it's possible and it's not that difficult. Let's be serious, Graphene OS devs arent getting deskilled, Graphene OS isnt a vibe coded project. The risks of vulnerabilities being subtly introduced is an ongoing risk, AI or not, there's no way to prevent it, and no human will resist it all. The key is probably building durable trust in the community, but even then someone could be paid or corrupted to subtly introduce something in, it's just a possibility. But Graphene OS is designed in such a way also that there are areas of the code that is more sensitive than others, probably simplifying review of sensitive code.

@laura_ge @GrapheneOS Hello Laurage, and thanks for the response!

I think while not completely possible to detect all AI, I think the developers using it themselves in code creation brings up a bigger concern:

- The more one is exposed to AI generated code, the more that code becomes normalized within the project and the less likely you are to detect it in others.

- AI code removes the creative process of new novel solutions that might otherwise fix many issues.

- As far as I am aware, deskilling from AI happens quite quickly and its without ones own knowledge of it happening. I know the current developers are competent, but what's to say they will be long term? And what does that invite when we offload so much knowledge to the machine?

- I think AI invites too many edge cases that cause people to slip and invite the AI more and more until they are effectively not apart of the project themselves. Thinking you are immune to this problem only makes you more susceptible. AI is like a box of sirens.

@corigan01 @GrapheneOS I do agree it's awful overall however if you use it at least to understand what this technology is even about and how it works I do believe it also makes you better at understanding its patterns and recognizing AI-generated output or code and also understanding the why of its gaps and inaccuracies come from it's possible to identify them as a human and better understand them. For example, understanding that AI models are probabilist systems that don't always give the same answer to an identical input, they are inherently random, that context management and attention systems optimization generates inaccuracies because the model is optimized to "forget" its own working data to optimize memory bandwidth use. It's still helpful to study the machine learning concepts behind it to know exactly what to expect from those tools when they are forced on you and why, and also to criticize it and be taken seriously because you actually know what you're talking about. AIgen is certainly like a casino machine, it's addictive and the models are actively trained to make you believe they are useful by being overly confident etc. though they now really can be useful and do real work it's not just a "fluke" it would be dishonest to affirm that. I do agree it has detrimental effects on one's mental health, just like addictive mainstream social networks are and many things in tech, that's why I think tech is dead in many ways, even pre-AI to be frank. All it does to the masses is mass surveillance and control, and even mass manipulation now even leading to turning election results over.

@laura_ge @GrapheneOS Hello again,

Sorry your post is a little difficult for me to read, I will repeat your question to make sure I am not misunderstanding you.

> You say more AI generated code helps you detect it more often:

I am unable to find the study so I cannot claim it to be false definitively, but I will disagree with the following reasons:

- For many projects, AI code sticks out like a sore thumb due to its weird tendencies and behaviors. If those tendencies become ingrained in the project, how would you determine AI from not?

- Since these are indeed probabilistic models, you will notice they code like a stack overflow post. It doesn't always "correctly" fit into the project the way one might expect. It gives this weird off feeling, however, that relies on the project having good coding standards to begin with. Something that can be eroded by continuous use of AI.

Otherwise, I agree with your post. AI is very much like a slot machine and I don't know what long term use looks like. This is why I am concerned that a project like Graphene is so quickly adopting it. I just don't think we have enough proof or data to show its a healthy or productive thing to include in a project intended to be this secure.

If I disregard the ethics concerns, I could give Graphene a pass for using AI to exclusively pen-test their platform (no code generation), but I think allowing any code-gen is my real big concern here.

@corigan01 @GrapheneOS I'm not saying that paper you are speaking is false but it's probably also not that simple, it's a strong support argument for anti-AI folks so that's why it's been mentioned alot but like all research just 1 paper isnt knowledge, we lack long term experience with those tools. Anyway I've seen this paper I know what you're talking about. I am happy with its conclusions if they are true it fits my political goals very nicely too, I find it good people are doing such research and we should do it more. I didnt say the more AI output the better necessarily, but that it's helpful to learn what those tools are like and how they work. I don't like having opinions on things I don't understand, it makes me feel like an impostor, how can you talk about something you have no knowledge of? It's much better in my opinion to make a very informed politcal critic of some tech, knowing very well how it works and not just spitting out inaccuracies and screaming out loud to progress a goal post. It makes it way more effective in my opinion if you can criticize it knowing exactly how it works and even better than many of its it's promoters if you can.