LessWrong16m agoenergy 127

Red-teaming multi-agent attacks for loss-of-control risks

image: LessWrong

We've written a paper on optimized multi-agent attacks, for which we present a brief writeup in this post. See the paper for more detail. This work was also presented as a spotlight research talk at the MATS Summer 2026 Symposium here . Tl;dr Recent AI cyber-related incidents exposed vulnerable monitoring surfaces to detect attacks stemmed from coordination between multiple AI agents . We show that when several agents distribute the attack fragments so that each trajectory looks benign, a monitor reading one trajectory at a time, or even reviewing all agents' trajectories together, mostly misses it. We introduce multi-agent control environments along with a collection of human-crafted distributed attacks that reliably appear benign to frontier-level monitors, and show that these are compositional and can compound into larger loss of control outcomes. Given similar affordances, AI models cannot plan these attacks reliably, but this may scale with model capabilities. We study how the results change across attacker and defender affordances. Threat model As agents increase in capabilities, volume and persistence are also increasing. Control measures against single-agent attacks are wel

read the original at LessWrong →
This source publishes only a summary in its feed, so the full article lives at the outlet. The link above and below goes to the original.