monotremata

And indeed figuring out what problem you’re actually supposed to solve. People assume they can just give an LLM the same spec they’d give a programmer, and that doesn’t work, because the programmer will figure out all the hidden shit they omitted from the spec but which is actually absolutely critical to its function. They’ll also anticipate future refinements of specs and how to leave room to accommodate those changes. Etc.

I really feel like the author of this piece got this strawman in their head and didn’t bother looking into how people saying these things are actually using them, because most of the people I hear saying them are themselves programmers, and they’re not trying to say “my job is trivial.” They’re saying “the thing the LLM does fast is not my whole job. it leaves out all the important stuff.”

I mean, I don’t have the encyclopedia anymore, so I guess I can’t really prove it to you. But it was sort of my introduction to the idea that not everything in an encyclopedia is necessarily true, so it’s pretty memorable for me. I recognize the irony of using another encyclopedia here, but Wikipedia mentions in the History section of its Pain in Animals article, “Researchers remained unsure into the 1980s as to whether animals experience pain.” (en.wikipedia.org/wiki/Pain_in_animals#History ) I obviously don’t know the story about how the line ended up in World Book, but I know they employ fact checkers for every line in the book, so presumably in the 50’s the line reflected some kind of scientific consensus, as bizarre as that sounds now.

When I was a kid in the 80’s, my sister and I found an old World Book Encyclopedia from the 50’s that someone in our neighborhood was throwing out, so we brought it home. It had some interesting “facts” in it. E.g., did you know that guinea pigs are perfect as subjects for scientific research because they’re unable to feel pain? 1950’s worldbook did!

Earlier tonight it also wasn’t showing up for me in search results on Duck Duck Go. All the reddit links were just labeled “reddit,” and the text was “We would like to show you a description here but the site won’t allow us.” Weirdly it doesn’t seem to do it if “reddit” is one is the search terms.

If this is part of their strategy going forward, though, that’s gonna hurt their clickthrough too.