Confirming Claims of Superposition and Adversarial Examples in Vision Transformers

This is a replication of Adversarial Attacks Leverage Interference Between Features in Superposition , completed as part of the Second Look Summer Fellowship. tl;dr In a previous blog we replicated the toy models examples from Stevenson et al. In this work, we reproduce the vision transformer experiments from the paper: On every dataset, we find that increases in superposition lead to adversarial attacks that line up more closely with class-vector geometry. Likewise, trends across a variety of metrics we measure match the paper. Beyond the paper, we find that: The coupling between the input and classification head geometry is independent of adversarial attack methods. Adversarial training increased attack robustness but we couldn’t measure changes in geometry using the methods from the paper. Likewise, we retrained models of different widths but we didn’t measure a change in coupling with geometry and robustness. Our code is here , models are here , and we encourage readers to fork/run followup experiments! Background Elhage et al., 2022 defines superposition as the phenomenon where “models represent more features than they have dimensions.” Stevinson et al. , argues this interfere