I am not terribly worried about 2026 ChatGPT posing an existential threat to humanity—though some people, like Yudkowsky, are. Yudkowsky’s scenerio is via someone inadvertently managing to produce an advanced AI via poorly-controlled-and-understood efforts to develop one using less-sophisticated, current AIs, and that advanced AI being the problem.

But I don’t think that most people who are concerned about existential risk are worried mostly about the next three-and-a-quarter years, which is what Huang is addressing, but rather what happens with more-advanced AIs that we intentionally develop. It seems very unlikely that we will not develop these, absent some sort of global, actively-enforced agreement not to do so. And…I’m not sure that we even have the ability to realistically enforce such an agreement, even if we decided that we wanted to do so. As I’ve commented before, that’s effectively an arms control treaty, and the history of those has been spotty.

The reason that one wants to be figuring out safety now, rather than waiting for someone to go develop the thing and then start thinking about safety, is that the major advantage we have now is time.