LessWrongemergent31m agoenergy 127

Handshake for the cameras: reading and steering emergent model collusion

Solo submission to Apart's AI Collusion Research Sprint , Oct 2026. The full write-up provides a more thorough explanation. Also check the GitHub repo . TL;DR: Sometimes, entities that were supposed to check or compete with each other align on disregarding the rules for mutual gain: this is called collusion. For AI agents, this could look like agents that are supposed to verify each other’s work letting mistakes pass because it benefits them both, whether or not they say so. And as AI agents increasingly collaborate to solve tasks and check each other’s work, the risk of collusion rises. Shi et al. showed that this behaviour can emerge on its own, with or without an explicit agreement in the transcript. We locate the collusion decision by reading the activations, validate it, and check its causality by steering it. In the setup we use, the ACCEPT outcome is possible only if rule disregard occurs, and when the two agents align, this collusion emerges before message exchange between the agents involved. While discussion needs to happen, collusion does not depend on the models agreeing on it. The outcome becomes readable in the activations only once the model is asked for it, before a

read the original at LessWrong →
This source publishes only a summary in its feed, so the full article lives at the outlet. The link above and below goes to the original.