@baldur Or likely both. But we can fix all that by letting another LLM review the code 🤡

@baldur I'm pretty sure it's both.

@baldur I still remember (in the early days of chatbots) my brother insisting he reviews 100% of the output himself. Nowadays he doesn't even bother pretending anymore.

@chapter1 @baldur I took a pair programming class a couple of weeks ago, and one of the exercises was for two humans to pair with copilot, which I hadn't used before. It was striking how quickly it felt that actually reviewing the code was an unnecessary bottleneck and I was holding things up by spending a couple of minutes reviewing what had been generated in a couple of seconds. I felt like I couldn't justify making the other person wait unless there was an obvious mistake.

@trantion Strong opinion: if review feels like the bottleneck, review is the only part still doing engineering. Speed up everything else, never that. I built AyCode around it: agents run in parallel, you read every diff.

@trantion @chapter1 @baldur

The most critical failures in any system - particularly in code, where there is more than one way to achieve an objective - are the NON-OBVIOUS mistakes!!

The obvious mistakes are normally trivial!

@baldur The lack of self-review is the thing which really really upsets me about this torrent of slop - if the operators of the tool were at least as proud of the *output* as they are of their *use* of the tool then we might not be quite so inundated with crud.

@kinnison @baldur A generation of kaleidoscope-owners convinced that it makes them the most gifted stained-glass artists known to the craft.

@kinnison @baldur also suffering from this. And I do not know solution. E. g. customer needs feature by tomorrow. I can do it by tomorrow with agents accepting “eagle eye review” and customer happy. Or I can deep dive, understand things, use agents, have clean code and no customer.

As a passionate engineer I HATE it. Only ~10% of tasks I can do maintaining decent output quality.

Sadly, it’s not even about engineering. It’s unavoidable outcome of capitalism+LLMs.

@baldur reviewing code is hard - it is likely that no one is capable of reviewing slop and revising it up to "written by a good engineer" quality.

@baldur The most perplexing ones are those who confidently state that the models now produce code better than humans. Either they are all bots, or there are a lot of devs who have no idea what good code even is. The last year has been the worst in my career for encountered code quality.

@danielleigh @baldur My bet is on most of the humans they've worked with wanting to keep their professional integrity and keeping their principles.

@danielleigh @baldur I'm guessing you haven't worked at a place where the majority of code is written by people who just graduated or are still in college, the bulk of whom get fired/promoted somewhere else and replaced after three-six months. But this is how software is written at scale these days. I'm not defending the practice, it just is what it is.

@baldur
i've been strucked by what was saying a Microsoft engineer in one article on vibe-coding, that they were so busy prompting they didn't review anymore, lack the time to do it..
(noted that detail but don't have the source and don't remember it)

@bituur_esztreym I think I came across that same article recently but don't have at hand either.

@baldur are they even given the time to review..... velocity of code seems to be the new token maxing

@bosconet That's basically what I've been hearing.

@baldur @bosconet
Heard that "The AI stuff revealed to us that for the triangle of Good-Fast-Cheap, CEOs everywhere would choose Fast twice"

@baldur I had someone who I had respected quite a bit tell me that AI-generated code no longer needed abstractions that makes the code readable and I found that to be dumb in the face.

@baldur the way I see it you can let an LLM generate code or review it but not both. As a human I'm better at writing it and wrong review suggestions are way easier to ignore.