LessWrongllmpretrain*1h agoenergy 103

Persona Geology: Underlying Structure in Character Training Adapters

TLDR: Using the character training pipeline in Maiya et al. 2025 and Baines et al. 2026, we trained a model zoo of 134 personality LoRAs on Qwen3.5-4B. We find that the weights have structure . In fact, they sort into five factors (Warmth, Competence, Timidity, Arousal, Imagination) that loosely match the Big Five traits in humans. We find preliminary evidence that the resulting geometry is informative for predicting behaviour. We introduce CharIn, a TracIn-style score that forecasts how training data will shift a model’s persona before training. It tracks register well, and even beats the label used to generate the data, but generalises poorly out of distribution. We think this is evidence that models exhibit low-dimensional persona structure, not just in the activations, but also in the weights. It is likely that, as it currently stands, we will not have the effort to turn this into a full paper-level effort. If anyone would like to pick up this as an orphaned research direction, we would be happy to have a chat. Introduction One perspective on why AIs are human-like is the Persona Selection Model : pretraining teaches an LLM many possible characters to play; post-training select

read the original at LessWrong →
This source publishes only a summary in its feed, so the full article lives at the outlet. The link above and below goes to the original.