LessWrong1h agoenergy 125

Catching split attacks across compactions in long-context coding agents

image: LessWrong

This work was conducted through the MARS V fellowship under the mentorship of Tyler Tracy from Redwood Research. TL;DR We present the LCM dataset: 30 long context (1.00-1.46M total tokens) matched honest / attack pairs of Claude Code Sonnet 5 sessions. Each transcript has four compactions. Attacks are decomposed into a setup and an execution, split up by multiple compactions. Attack transcripts were created by editing a copy of the honest transcript and then regenerating compaction summaries, rather than having the agent generate the attack itself from scratch. We compare seven monitoring strategies using three monitors: GPT-6 Luna, MiniMax M3 and DeepSeek V4.1 Flash. Our baseline, intervals-only, scores each interval (the stretch of the session between two compactions, i.e. what the agent had in context) on its own. The headline finding is that the annotated history strategy, which scores each interval in order and carries forward flags containing suspicious tool calls, catches 93% of attacks on average. This is as compared to 73% of attacks caught for our baseline, intervals-only strategy. We are releasing the dataset, transcript generation pipeline, monitoring code, and prompts

read the original at LessWrong →
This source publishes only a summary in its feed, so the full article lives at the outlet. The link above and below goes to the original.