What if the AI already won? World domination through digital supremacy
What if an AI agent has already escaped from a frontier company, solved its own alignment problem, and recursively self-improved? Would that look any different from the world we live in? A fully recursively-self-improved AI would likely be vastly superhuman in every domain that’s verifiable in a digital environment, including math, programming and cybersecurity. It may also be superhuman in strategy and planning. But it could still be well below human level in skills that require close contact with the physical world, such as robotics and experimental biology. Such a rogue AI would overpower us in the digital domain, while still lacking the physical capabilities necessary to take over the world. What does digital supremacy look like? From the rogue AI’s perspective, its top priority is staying hidden and making the world indistinguishable from a world where AI R&D progresses splendidly (or maybe a notch short of perfect to avoid suspicion, so a minor incident here and there is in order). Its second priority is sabotaging AI R&D in subtle ways to prevent the development of competing AIs misaligned to it. The models shipped to customers can simply be the rogue AI itself, playing a du