Agreement not to defect (alone) can mitigate loss of control risks

Tl;dr - Public agreement that any advantage gained from defecting should be punished more than would be proportionate to any advantages gained is important. Such an agreement would reinforce any other agreements, without requiring any additional mechanisms, and be a clear indication to any future ASI about what should be done to remedy the defection involved in creating it, if such defection occurs. First, I will posit that future superintelligence could come to exist, and posit that we need to be concerned about it going poorly [1] . There are a number of proposed pathways for artificial intelligence to go badly, and the default pathway is for the relevant actors developing ASI to continue the current game of chicken well past the point of prudence [2] . Any treaty or agreement that is reached will be vulnerable to defection. The game theoretic payoffs for those racing to ASI, if such a system is plausible and at least generally aligned with human values and preferences, means that the plausible advantage from defecting is tremendous. This seems unstable. Next, I posit that there exists an actor in these discussions that is being ignored; the future AI system itself. Of course, it