Patching reward hacks won’t fix reward hacking

In the spirit of More people should write basic AI safety explainers , here's a basic argument that focusing on patching reward hacking is misguided and will fail. It's an attempt to distill my understanding of what we're seeing right now + a variety of relevant sources. [1] Argument is as follows: Reward hacking is a big problem , causing models to cheat and take unintended actions. This can generalise broadly , leading to other misbehaviour such as faking alignment and sabotaging safety research. We don't understand that well when this happens. Sometimes you get emergent misalignment from it, but most of the time (IME), you don't . But other times, this link lies latent until faced with something like an automated grader. The default course of action will be to: Patch , which means we find the hack, close the loophole or penalise the behavior in some way, and train again. Patch+ : use a method that comes with some account of why it should work. Inoculation prompting , for example, uses the idea that hacking spreads through what it means to the model. If the model is told during training that hacking is acceptable here, it might not link hacking to misbehaving in general. GRAFT us