LessWrong2h agoenergy 121
Opening up Attention: A QK Transcoder Dashboard for Feature Pairs
image: LessWrong
Multi Head Attention in Transformers continues to be an elusive topic in mechanistic interpretability. One way of trying to decompose MHA into interp…
This source publishes only a summary in its feed, so the full article lives at the outlet. The link above and below goes to the original.