We should be explicit about the path we’re on. We are engaged in a winner-takes-all fight for AI supremacy. The operative assumption is that whoever wins will rule the world, because that country will control a version of AI so powerful that it can improve itself to be invincible. Whoever wins will liquidate all opposition immediately, so there can be no challenge to its power. That’s the story for what’s at stake.
There are many consequences to this. To start with, no effective regulation of AI is possible, because it is impossible to say what constraints on AI development would get in the way of the fight to the death. In particular some of the most obviously risky issues—such as autonomous military decision making—are the most untouchable. This the worst kind of arms race, where it’s not just a matter of parity. Only victory is good enough. There are no effective limits to the amount of money it can take.
But even that is just the beginning. Once you’re in a fight-to-the-death, any form of international cooperation becomes extraneous. You can’t think too seriously about common interests when everyone is a mortal enemy. That mindset is all too easy in any case, as common interests are historically a hard sell (even for climate change). So we fall back on the world view we now get everyday—whoever has power deserves what he can get.
That this is acceptable without even much pushback is a matter of ultimate hubris. It’s all okay, because we’re going to win. We’re not interested in thinking too much about how international competition works or what it will really take to win. We’ve been on top for so long that we’re entitled. All the discussion of the “Thucydides trap” for competing powers is beside the point. God will make us win. That’s not an exaggeration for where we are.
There’s a lot that we don’t know about the end to this story. The future is never clear. But there is one thing that is definitely true—we’re stumbling our way into real danger. The risks are enormous. For technology we’re certainly not necessarily on top—even today China is a serious competitor—and we’re burning bridges to the future by killing research investments. Even for AI we’re a long way from the finish line. We’re in a wildly unstable arms race with many new technologies, proliferating nuclear weapons, and no good way even to know who is ahead.
The only workable way to forestall disaster is viable common interest. You don’t have war or countries laid waste if all parties recognize there’s lots to lose. (You can still have homicidal maniacs, but they’re less likely to gain power.) The world needs order, starting with recognition that interdependence, economic and otherwise, is not going away. So we had better fix it. Start with trade and the world economy, and then do something about war. A fight to the death ends with just that, and who knows how many millions would pay.