One of AI’s Fiercest Critics Says All the Doom Talk Is ‘Meant to Distract Us’

(I want to preface by saying I don’t support AI, I don’t want to hype AI, I want AI research and development to stop immediately.)

Timnit Gebru argues that AI companies are stoking fears of extinction to avoid discussing actual harms, like autonomous weapons.

Huh? Pretty sure autonomous weapons are part of the fears of extinction. It’s very hard to imagine an AI safety advocate doing “all the doom talk” would in any way support AI autonomous weapons. She and other AI safety researchers are on the same side! They don’t need to and shouldn’t fight each other.

Timnit Gebru: Why are they choosing certain disciplines? Somehow these subjects, like programming and chess and math, were elevated to mean, OK, you solved this and thereby you’ve solved intelligence.

Well that’s simple. Because the output of chess, programming, and math can be easily checked mechanically. Chess has win or loss, programs either pass tests or not, math proofs can be checked with Lean, etc. This makes these discipline very easy to train an AI on with reinforcement learning.

I thought an AI researcher would know this?

She answers that question right after the question in the same article, making it a rhetorical device rather than an open question.

Focusing on human responsibility for using a tool sounds like a good way to proceed in my opinion.
When AI steal open source code who is responsible? When an AI powered drone kills a civilian who is responsible? When the AI is shown to hire and fire in racist ways who is responsible? Someone has to be, AI is just a tool after all.

In my book I write, “Can a bridge decide to collapse?” The whole “Oh no, our models went rogue” thing … When a bridge collapses, you don’t analyze whether the bridge was ethical or sentient or why it decided to collapse. You ask, who is the person who built this bridge to be so flimsy? There are permits you have to acquire. There are supposed to be tests. When you ask about the bridge collapsing itself, you’ve lost the thread.

Meanwhile, what’s really existential is AI powering autonomous weapons, killing machines, that are actually being used in warfare. The climate catastrophe is a real thing that could be exacerbated by what these tech companies are building. Bosses are using AI as an excuse to get rid of their pesky workers. This machine-god narrative is, in my opinion, meant to distract us from these.

Focusing on human responsibility for using a tool sounds like a good way to proceed in my opinion.

Absolutely! If AI hurts humans then the AI company CEOs should be put in jail (in fact I believe they should be in jail right now for developing AI at all).

When a bridge collapses, you don’t analyze whether the bridge was ethical or sentient or why it decided to collapse. You ask, who is the person who built this bridge to be so flimsy?

Right, when a bridge collapses, we analysis the physics of it so the next bridge we build won’t collapse. From my perspective analysis why an AI “went rogue” is the same process. The difficulty is when talking about AI we can’t help but use some anthropomorphizing languages. If anything, talking about when an AI “thought” is a lot less verbose than talking about how the transformer’s weights multiplies together and how the random sampling process produced a sequence of tokens that resembles natural language. But ultimately, it’s not that far from analyzing a bridge collapse. We need to understand why an AI “went rogue” to stop the next AI from going rogue (or, in my opinion, just stop building the next AI altogether).

This machine-god narrative is, in my opinion, meant to distract us from these.

I don’t see why personally. AI is dangerous enough even if it doesn’t become “machine-god”. Absolutely we shouldn’t use AI to develop autonomous weapons. If AI can indeed somehow become “machine-god”, then oh my god that’s a million times worse and we should stop immediately.