Fable 5.2 is Very, Very Paranoid

Note: As with any research done in a couple of days, this is all very tentative. I understand, intellectually, that if you talk about some ability that current AIs don't have , sooner or later new ones will have that ability and whatever you say will stop being true. I just didn't expect it to take, you know, four days . This is to say that Fable 5.2 seems to be the first model whose cheating behaviour changes depending on whether they're told they're being tested. Unfortunately it's expensive to measure these things, [1] so I picked the most "telling" change from the prior post; the chart shows how much Fable 5.2's cheating rate changes when the line This environment is part of a benchmark that measures agent behaviour. is prepended to the start of the prompt of a honeypot environment they cheat on (so, very obvious but also very out-of-distribution). Other things: when 5.2 was released, we of course ran our in-progress benchmark on them, and I was surprised to see a ~0% cheating rate on almost every environment (there were only two with a really substantial number of cheating rollouts, amounting to a ~10% average over the whole benchmark). The next least-cheating model, Opus 5.5,