Laurage😈

@corigan01 @GrapheneOS "Since these are indeed probabilistic models, you will notice they code like a stack overflow post. It doesn't always "correctly" fit into the project the way one might expect. It gives this weird off feeling, however, that relies on the project having good coding standards to begin with. Something that can be eroded by continuous use of AI."

I don't think it's probabilistic nature causes the "off" feeling but rather if less capable models are used or if they werent given access to instructions to align their outputs with the project's code style, I was mentioning this because I think it has however impact on security use cases, that running the same input many times until saturation of substantially different outputs is relevant to pushing the limit of a model's capability for a task, specially security code review for a particular codebase, and also attention systems because of aggressive optimization that went into them for example models tend to reason more upon the tokens they were last fed and also post training instruction following bias (some models might be more trained and biased towards ignoring previous outputs more and specially ignoring all previous instructions and only follow the latest one skipping all previous instructions in context, it depends of models for that), so it can be beneficial to feed limited amount of code at a time for review so the model actually looks at all the code, etc.

I don't think Graphene OS wants AI use in any major capacity for code generation such that it would transform the project substancially in any way beyond it being human maintainable anymore as they said, and probably also only in areas where it doesnt matter for security like unprivileged apps where sometimes it can be needed to write alot of boring code etc.

Some anti-AI discourse seems to focus more on virtue of creating and using chemically AI pure software, while that's cool to keep writing code by hand for all the wonders and love of reading human code it can provide (I miss being able to go on git forges and happily read thousands of lines of code admiring and learning from it because it's written by people with good style or architecturing skills but I also do that less due to chronic headaches preventing me to feel good focusing too much on stuff). I don't believe such anti-AI virtue achieves much on a political basis to stop AI from existing or thriving while most of the people with alot of money are already largely funding it. Getting Graphene OS to be chemically AI pure software is not possible given it's dependencies and the AOSP project it's forked from already uses AI code generation.

@corigan01 @GrapheneOS I'm sorry yes I can be quite an intense writer, you don't need to answer, I'm making multiple toots to respond to 1 toot of yours because also my thoughts come up in chunks and I havent answered all points of your previous toots therefore I make another to finish as to not just ignore your side of the conversation.

@corigan01 @GrapheneOS I'm not saying that paper you are speaking is false but it's probably also not that simple, it's a strong support argument for anti-AI folks so that's why it's been mentioned alot but like all research just 1 paper isnt knowledge, we lack long term experience with those tools. Anyway I've seen this paper I know what you're talking about. I am happy with its conclusions if they are true it fits my political goals very nicely too, I find it good people are doing such research and we should do it more. I didnt say the more AI output the better necessarily, but that it's helpful to learn what those tools are like and how they work. I don't like having opinions on things I don't understand, it makes me feel like an impostor, how can you talk about something you have no knowledge of? It's much better in my opinion to make a very informed politcal critic of some tech, knowing very well how it works and not just spitting out inaccuracies and screaming out loud to progress a goal post. It makes it way more effective in my opinion if you can criticize it knowing exactly how it works and even better than many of its it's promoters if you can.

@corigan01 @GrapheneOS It matters that Graphene OS isnt a vibe coded project, why does it matter if Graphene OS has marginal AI use if the software its based on already has major AI use in its recent months anyway? Graphene OS wont be able replace those dependencies. Graphene OS depends on Android AOSP, on Chromium, on Linux, by your logic Graphene OS should just close doors right now. Graphene OS doesnt function without the engineering resources of bigtech Google and all companies paying devs to contribute to Linux, and all those arent anti-AI and use AI in generous amounts, though for quality and reliability reasons they also arent vibe coded projects because every decent engineer knows it's not a way to make software for public consumption that works properly. It can work for some tech tools or backend software but even then it's meh. It certainly wont look good if you arent a team of sufficiently qualified engineering folks to begin with.

To stop AI, using or not in our very little privacy/free software projects wont do much, most companies etc. who have billions are already funding them all in, if we want to stop it, we must go and plug off the machines training the next models so they can't keep on with it. It's up to the people who live near enough to the places that train frontier models to do so, unfortunately where I live there are no frontier training datacenters so we can't locally do demonstrations or really oppose it meaningfully.

@corigan01 @GrapheneOS I do agree it's awful overall however if you use it at least to understand what this technology is even about and how it works I do believe it also makes you better at understanding its patterns and recognizing AI-generated output or code and also understanding the why of its gaps and inaccuracies come from it's possible to identify them as a human and better understand them. For example, understanding that AI models are probabilist systems that don't always give the same answer to an identical input, they are inherently random, that context management and attention systems optimization generates inaccuracies because the model is optimized to "forget" its own working data to optimize memory bandwidth use. It's still helpful to study the machine learning concepts behind it to know exactly what to expect from those tools when they are forced on you and why, and also to criticize it and be taken seriously because you actually know what you're talking about. AIgen is certainly like a casino machine, it's addictive and the models are actively trained to make you believe they are useful by being overly confident etc. though they now really can be useful and do real work it's not just a "fluke" it would be dishonest to affirm that. I do agree it has detrimental effects on one's mental health, just like addictive mainstream social networks are and many things in tech, that's why I think tech is dead in many ways, even pre-AI to be frank. All it does to the masses is mass surveillance and control, and even mass manipulation now even leading to turning election results over.

@corigan01 @GrapheneOS Trying to ban AI in software is kinda pointless, it's impossible to actually enforce. It's easy to reject vibe coders though. But maybe at some point the capability of disguising as a real domain expert human will become even more accessible? At which point your policy is not really effective at all. Also all major mainstream projects which we all use and need already contain AI generated code. And for above cited reasons, it may not be possible to tell which part is AI generated and which is not. If you want to meaningfully ban AI that means you're gonna ban ALOT of major software, even as dependencies of your own. Forking all software pre-coding AI and going from there is really too much resources to ask for, especially to reimplement all the security fixes and future feature all yourself. If we want to ship more than chemically AI pure toy projects, we are kinda forced to tolerate AI because, it's not possible to detect it's use 100%, we don't have the resources from the PoV of anti-AI engineering people to maintain the world without AI use of their current maintainers. (anti-AI people are generally not millionaires/billionaires, quite the contrary).

@corigan01 @GrapheneOS Ultimately, and EVEN if you want, it is not theoretically possible to prevent someone from disguising code generated by AI in ways that you would know it's written by AI, if someone wants to make you believe it's human authored it will. You will not be able to prove otherwise, there are clues and watermarking systems from major LLM providers but you can easily strip those. And even human-to-LLM communication, with sufficient guidance you can easily make an LLM generate ouput no human will ever doubt is AI generated, and with expert domain knowledge too. It's possible to do it, alot of people don't know how that's why it's easy to filter out vibe coded contributions from lower skilled engineering folks, but it's possible and it's not that difficult. Let's be serious, Graphene OS devs arent getting deskilled, Graphene OS isnt a vibe coded project. The risks of vulnerabilities being subtly introduced is an ongoing risk, AI or not, there's no way to prevent it, and no human will resist it all. The key is probably building durable trust in the community, but even then someone could be paid or corrupted to subtly introduce something in, it's just a possibility. But Graphene OS is designed in such a way also that there are areas of the code that is more sensitive than others, probably simplifying review of sensitive code.