Against Moloch #50

Mapping out a strategy for an ideal world is easy: we all agree to pause frontier development until alignment is solved. Once that’s done, we glide gracefully into a future of unimaginable abundance. No further questions. In the real world, it’s more complicated. Any plan that relies on global coordination is—to say the least—ambitious. Do we go all in on the ambitious plan because it’s the only one that could work, or do we also invest in a backup plan? Top pick Against SURA Scott Alexander’s Against SURA argues that many parts of the AI safety community are making a strategic mistake by focusing exclusively on a unilateral pause, at the expense of other important strategies. This has made some people very angry. There’s a lot to unpack here—about strategy, about Scott’s post, and about the response to it. I’m working on a longer post about it, but here’s a brief summary of Scott’s position: Most AI safety plans fall into one of two broad categories: Plan A is a global pause on AI development until we are able to proceed safely. Almost everyone in AI safety, including Scott, prefers Plan A. Plan C is a fallback if Plan A fails. It calls for the frontier labs to coordinate to burn