Exploring Unsupervised Methods for Recovering the Refusal Direction in LLMs

Introduction This is the first part of what we plan to be a research exploration into the geometries of LLMs. We start with building off existing work by Arditi et al focused on the refusal direction in LLMs. Background We recommend reading the Arditi et al. Lesswrong post and paper for more details. They explore the idea of the behavior categorized as Refusal being mediated by a single direction. Refusal is basically when an LLM refuses to do something, so examples like "I cannot fulfill..." "I cannot complete..." etc. A direction is defined by a vector. Here, it means a vector in an LLM's activation space. An LLM's activation space is the space where an LLM's internal vectors live. Think of the intermediary vectors that are used in the calculation of the final next-token-prediction process. Mediated means that if this direction were to be removed, then the behavior categorized as Refusal would stop. Arditi et al. performs directional ablation as their method of removal. They identify the refusal direction using difference-in-means . Methodology We apply Principal Component Analysis and Isomap alongside diff-in-means to an experimental setup of refusal. We test the Llama 8b 3.1 In