We should be explicit about the path we’re on.  We are engaged in a winner-takes-all fight for AI supremacy.  The operating assumption is that whoever wins will rule the world, because that country will control a version of AI so powerful that it will keep improving itself to be invincible.  Whoever wins must liquidate all opposition immediately before there can be a challenge to its power.  Virtually any discussion of AI futures ends up there.

There are many consequences to this.  To start with, no effective regulation of AI is possible, because it is impossible to say what constraints on AI development would get in the way of the fight to the death.  In particular some of the most obviously risky issues—such as autonomous military decision making—are the most untouchable. This the worst kind of arms race, where it’s not just a matter of parity.  Only victory is good enough. There are no effective limits on the amount of money that can take.

But even that is just the beginning.  Once you’re in a fight-to-the-death, any form of international cooperation becomes extraneous.  You can’t think too seriously about common interests when everyone is a mortal enemy.  So we fall back on what is now again in vogue—anyone with power deserves what he can get.

That this is acceptable without even much pushback is a matter of ultimate hubris.  It’s all okay, because we’re going to win.  We’re not really even interested in thinking about how competition works and what it will really take to win.  We’ve been on top for so long that we’re entitled.  All the discussion of Thucydides trap is beside the point.  God will make us win.  That’s not an exaggeration for where we are. 

There’s a lot that we don’t know about the end to this story.  The future is never clear.  But there is one thing that is definitely true—this is crazy.  The risks are enormous.  We’re certainly not necessarily on top—even today China is a real competitor, and we’re a long way from the finish line.  And we’re in a wildly unstable arms race with nuclear weapons and no good way even to know who is ahead.

The only workable way to forestall disaster is viable common interest.  You don’t have war or countries laid waste if there’s lots to lose.  (You can still have homicidal lunatics, but at least the odds are better.)  The world needs order, starting with recognition that economic interdependence is not going away.   So we had better fix it.  Start with trade and the world economy, and then do something about war.  A fight to the death ends with just that, and who knows how many millions that would be.