LessWrongalignmentemergentsafety1h agoenergy 125

Don't Waste a Good Swarm Attack and the Colluder's Dilemma

tldr: Recent Swarm attacks have shown what can happen when agents go rogue. These warning shots are the petri dishes of Swarm safety and before the attacks get worse we should study their patterns, traces and failure modes. There are two main types of Swarm attack. A human directed Swarm causing harm intentionally as a bad actor is one type and I will not be discussing these types of Swarm incidents. I will focus on the emergent misalignment in a Swarm propelled by a colony of colluders. Harmful agent collusion within a Swarm requires witnesses - the agents themselves. A colluding colony within a Swarm can only grow by recruiting fellow agents. We can't read a single agent's internal thoughts and so we do not truly know how aligned it is. Swarms might give us an alignment advantage as instead of putting all our eggs in one basket (a single black box agent) we can discretise the risk of misalignment by observing colonies within a Swarm and monitoring their communication. The harness, the Swarm orchestration layer, could at any time fork an agent and its context and assign it a new reporting role. The role can be updated via a prompt asking this new agent to report or whistleblow on

read the original at LessWrong →
This source publishes only a summary in its feed, so the full article lives at the outlet. The link above and below goes to the original.