The Snitch Reflex: Can We Force Language Models to Be Compulsively Honest?

I fine-tuned a toy model to vomit back arbitrary tokens. I used them as Markers tied to specific subsets of its training data to connect training to generation. Training + testing cost $0.49 on an RTX 4090. Could help with interp and safety monitoring. Chain of Thought for alignment monitoring is starting to fail and could never be fully relied upon anyway. We are using it for both thinking and for reporting bad thoughts, a major no-no. So what if we could build in a second, hidden reporting system? A compulsive tic, a snitch, not subject to training pressure or controllable by the model? I fine tuned TinyStories as a cheap AF demonstration of the concept of injecting context dependent markers into training data, forcing it to regurgitate absurd, arbitrary “111” and “222” tokens. Example: give the model the start of a story about a kid finding a wheel, and it writes: ...put it back on his toy car. He was so happy to have a new toy car to play with. 111 Here’s the general strategy: I didn’t tell it to write about toy cars, and never instructed it to generate 111. It wrote about types of toys, saw newline, and emitted the 111 marker from its toy-types training subset with 99.9% proba