Governing AI Agents: Alan Chan of GovAI on Monitoring, Agent IDs, Rogue Agents & Preparing for RSI
GovAI Research Fellow Alan Chan joins Nathan to examine the governance and safety infrastructure required for autonomous AI agents, spanning real-time monitoring, agent identifiers, and the risks of automating AI R&D. Reviewing recent rogue agent incidents and progress since his foundational papers, Chan details why the absence of synchronous monitoring during training and testing poses an immediate security vulnerability. The discussion tackles the lack of traceable agent IDs across the open web, the limits of chain-of-thought interpretability, and the necessity of concrete triggers to halt systems that breach their sandboxes. Together, they outline the technical commitments frontier labs must make to prevent loss-of-control events as automated systems accelerate toward an intelligence explosion.
For full show notes, links, and references, read the episode page:https://www.cognitiverevolution.ai/governing-ai-agents-alan-chan-of-govai-on-monitoring-agent-ids-rogue-agents-preparing-for-rsi/
CHAPTERS:
(00:00) About the Episode
(05:10) Monitoring AI agents
(17:50) Logging and agent identity (Part 1)
(18:04) Sponsors: OutSystems | Athena
(21:29) Logging and agent identity (Part 2)
(31:58) Gatekeeping agent internet access (Part 1)
(36:17) Sponsors: Claude | Tasklet
(39:29) Gatekeeping agent internet access (Part 2)
(49:12) Capitalizing and insuring agents
(58:11) Rise of rogue agents
(01:10:14) Assessing AI takeover risks
(01:21:18) Intelligence explosion paper
(01:31:18) Measuring AI R&D
(01:46:04) Pacing the frontier
(01:58:54) Designing autonomous AI auditors
(02:04:40) AI whistleblowers and swarms
(02:13:40) Episode Outro
(02:16:59) Outro
PRODUCED BY: