We should be explicit about the path we’re on. We are engaged in a winner-takes-all fight for AI supremacy. The operative assumption is that whoever wins will rule the world, because that country will control a version of AI so powerful that it can improve itself to be invincible. Whoever wins will liquidate all opposition immediately, so there can be no challenge to its power. That’s the story. We’ve got to win.
There are many consequences to this. To start with, no effective regulation of AI is possible, because it is impossible to say what constraints on AI development would get in the way of the fight to the death. In particular some of the most obviously risky issues—such as autonomous military decision making—are the most untouchable. This the worst kind of arms race, where it’s not just a matter of parity. Only victory is good enough. There are no effective limits to the amount of money it can take.
But even that is just the beginning. Once you’re in a fight-to-the-death, any form of international cooperation becomes extraneous. You can’t think too seriously about common interests when everyone is a mortal enemy. That mindset is all too easy in any case, since common interests are always a hard sell (even for climate change). So we fall back on the world view we now get everyday—whoever has power deserves what he can get.
That this is acceptable without even much pushback is a matter of ultimate hubris. It’s all okay, because we’re going to win. We’re not actually interested in thinking too much about how international competition works or what it will really take to win. But we’ve been on top for so long that we’re entitled. God will make us win. That’s not an exaggeration of where we are.
There’s a lot that we don’t know about the end to this story. The future is never clear. But there is one thing that is definitely true—we’re stumbling our way into real danger. The risks are enormous. For technology we’re certainly not necessarily on top—even today China is a serious competitor—and we’re burning bridges to the future by killing research investments. Even for AI we’re a long way from the finish line, and it’s unclear which of multiple paths will get there. Overall we’re in a wildly unstable arms race with many new technologies, proliferating nuclear weapons, and no good way even to know who is ahead. Yet we’ve declared that war is a perfectly natural element of national policy.
The only workable way to forestall conflict is viable common interest. You don’t have war or countries laid waste if all parties recognize there’s lots to lose. (You can still have homicidal maniacs, but they’re less likely to get power.) The world needs order, starting with recognition that interdependence, economic and otherwise, is not going away. So we had better fix it. Start with trade and the world economy, and then do something about war. A fight to the death ends with just that, and who knows how many millions would pay.