Behavior Theory-Building for Diverse Agents and Environments

The oversight problem consists of describing, eliciting, and understanding behavior. Prior work in black-box behavior investigation and interpretability seek to construct compact, faithful, and informative theories that explain model behavior. We propose training investigator agents and subject models to emit explicit theories, and using these theories as common currency for oversight across models. We hope to create a complementary mode of oversight to weight-space and model-specific approaches by studying the transfer properties of theories across model families. This post outlines our positioning and initial research agenda built on top of a few recent publications. Oversight as Theory-Building Transluce is working on oversight-as-inference, training oversight foundation models on experiments on a particular subject model: “ Any oversight question can be formalized as Bayesian inference over the outputs of a Pythonic world model. ” The models are posttrained to explain behaviors in natural language. Their method is promising for overseeing a system in depth: expressing interventions as a primitive set such as get_activations, patch, fine_tune, decode, and sample_model is a power