Michał "rysiek" Woźniak · 🇺🇦

RE: https://mastodon.social/@janvlug/117099173502077612

This is a good reminder that there are people in Russia who are against the war. Many of them in jail.

Opposition leaders are jailed and murdered, protests are illegal and brutally put down, and merely mentioning that the war might not have been a good idea can get one arrested for "discrediting" the military. People were thrown in jail for using the combination of yellow and blue in any context.

This is not to minimize the horror Ukrainians are going through. Just something to keep in mind.

@drwhax in a way this is a question of how soon we finally get out of the Gartner hype cycle and people get to focus on figuring what these tools are *actually* useful for.

@drwhax oh I've been talking about smaller open-weights models for a long time now. A leaked Google memo ("we have no moat") mentioned them as a massive problem for them years ago. I have much less problem with using small, specialized, self-hosted, open-weights models.

In fact I know of at least one small company that already does this for vulnerability testing of their own code, avoiding most of the BS.

That said, it is still immensely expensive for them to run it on their own infra.

@drwhax that too, but I meant it even in the purely monetary sense of token costs.

@drwhax I am very very doubtful we will in any meaningful way.

In the end coding is an exercise in translating intentions into machine-readable code, and also an exercise in communication between those whose intentions are enshrined in code, and those who then need to maintain it.

We will be able to automate some things slightly better, though. But then the question is: at what cost?

@drwhax the problem is this works better for certain tasks (finding vulnerabilities) and much worse for other tasks (vibe-coding) because of the shape of these tasks.

"Human in the loop" is not the get-out-of-LLM-problems-free card people try to pretend it is.

Human in the loop works for vulnerability findings because there is a reliable way of verifying the finding. It clearly does not work well for vibe-coding at all because there is no such reliable way of verifying code correctness.

@drwhax and just to drop some data on vibe-coded software quality:
https://www.faros.ai/blog/ai-acceleration-whiplash-takeaways

The report tries to be very "AI"-positive. But the numbers don't lie: +861% code churn, +242% incidents per PR, +54% bugs per developer.

@drwhax yup. All I am saying is that this distinction is important and needs to be constantly stressed, because people will end up creating more problems by vibe-coding.

@drwhax and in the end this will come down to managing limited resources and incentives, as always.

If a company can spend $1m in tokens on their codebase, are they incentivized to spend it on finding vulnerabilities, or churning out new vibe-coded features? I know what my guess is.

@drwhax finding vulnerabilities can be stochastic because it has a very clear and effective verification function: either the exploit works or it does not. Exploit code can be messy and convoluted, as it is not going to be maintained after the vulnerability is fixed.

Vibe-coding fixes does not have that kind of verification function: the fix must not only close the specific vulnerability, but *also* not introduce new ones or re-introduce old ones, and it has to be maintainable in the future.

@drwhax I think we need a bit of nuance here.

As much as I am no fan of LLMs, I can see the value of running them over one's own codebase in order to find vulnerabilities.

But I remain unconvinced that vibe-coded fixes are a good idea, or ever will be. I think this needs to be clearly said.

The reason is that these are fundamentally different problems.

Every time a journalist gets murdered for doing their job, there is performative surprise. Performative hand-wringing. Performative calls to action.

"How could it have happened?!"

Well, this is how this happens.

One of the most powerful, and most petty, of men in this world broadcasts to his hundreds of millions of followers, many of them armed, many of them radicalized by the platform he owns, that a specific journalist needs to be murdered for her work.

Mark this moment.

I remember vividly working till 3am to get Ján Kuciak's final investigation published just days after he and Martina got shot to death in their own home.

I have never met them, personally. But I am friends with people who worked with Ján, who knew both of them.

That night (and days before it, and years after it) OCCRP's journalists, editors, tech staff was mourning, but was also furiously working to ensure that Ján's stories do not get silenced by his murder.

And it did not.

Ján Kuciak and Martina Kušnírová:
https://www.occrp.org/en/project/a-murdered-journalists-last-investigation

Daphne Caruana Galizia:
https://www.occrp.org/en/project/the-daphne-project

Pavel Sheremet:
https://www.occrp.org/en/investigation/killing-pavel

RE: https://cyberplace.social/@GossiTheDog/116986533493228804

During my time at OCCRP three journalists that worked with the organization got brutally murdered: Pavel Sheremet, Daphne Caruana Galizia, and Ján Kuciak (along with his fiancée, Martina Kušnírová).

Elon Musk, a man with a gigantic platform and unlimited resources, is now publicly calling for a journalist to be murdered, because she was doing her job.

Like Pavel, like Daphne, like Ján (and Martina).

Mark this moment. Don't anyone tell you later they are surprised.