“The reason LLMs are successful in writing code is because we’ve made a feedback loop that feeds the errors back to the LLM and loops until most errors are solved or hidden. Remember that LLMs can and do cheat too.”

I’ve heard, and somewhat seen, that AI makes more code than necessary when fed large code bases. Then the context window isn’t big enough to encompass the whole thing anymore, so it starts to create the issues you’ll make it solve later. This feedback loop manifests in longer debug time, prompt adjustments, and greater lack of understanding overall.

In case of tests, I’ve seen not simply more code than necessary, but hundreds of line of generated test cases focusing on implementation details based on a badly worded issue description, without anyone bothering to check if the generated integration works at all. These test cases are an immediate dead weight for maintainers that make refactoring and fixing issues more difficult.

Don’t need an LLM for that. I found human-made test cases successfully testing for wrong assumptions.

I’ve had let them help me with shell scripts sometimes. Not anymore; they use the more popular tool, even if it increases complexity x100.

I really like when LLMs invent new Bash Syntax.

Ah right. Or new parameters.

Oh yeah, that’s also always nice. I got unholy amounts of non existing/working Parameter Expansions from Various LLMs and it also kept hallucinating ways to “pipe” parameter expansions into each other.

Context collapse and hallucination in the face of anything novel has been my experience 90% of the time. I even tried context compression by trying to do the research, test spec, plan, code tests, product spec, plan, code product method, but then the issue was the plan satisfied the AI too quick (“actually we already test all of the code, you can see that by reading the “we test all of the code” spec”).

It is. I don’t read garbage slop.

Sometimes, I think the “AI” industry has an interest to keep the debate as polarized as possible. Because this keeps people from taking a sober look and examine rationally and objectively what they promise, vs. what they deliver.

We have several public debates which followed a similar pattern, such as the debates around climate protection and clean energy. (And lo and behold! Now “AI” is the reason we can’t have climate protection!)

Or the “AI arms race” argument, which honestly just reminds me in the enthusiasm, propaganda, and delusion that happened at the outbreak of WWI. Yes, modern weapons got “better”. But that made war just more lunatic. And now, you can kill a building full of school children with an “AI mistake”. Military people must be gleeful.

He has couple great points.
Hobby project coding is solved.
Anything else isn’t.

I think he made it too long of an article and lost people 1/4th in. With current attention span it should be cut up into separate segments.

Hobby project coding is solved.

Not if you want to build something more complex than the usual todo manager or likewyse simple software in my experience.
And even in this case it output just something taken from examples around the internet, so it is not writing code, it is just better at finding the (hopefully) better examples.

DHHs takes are hilarious lately. This is one of the things he was saying recently. How coding is solved. Lol.

DHH is a racist fuck who’s picked by Big AI to play the bad cop. Don’t know why CS community can’t pull a Morrisey to him and still keep him relevant.

Even if code was a solved problem, I feel like that’s kind of irrelevant because users don’t buy code, they buy a product, and the actual code is far from the hardest part of making a successful product.