What Did Astra See?

The Sparks of Weight Space Programming blog post describes some surprising capabilities Astra has for working directly with model weights. In one experiment, Astra is able to produce a diagram of the maze for a small Pacman game just by looking at the weights of a policy network trained to play it. This inspired us to look into what kind of visual reasoning Astra is able to do, and how that might help explain some of the impressive results found in the blog post. We found evidence that Astra can do quite a lot of encoding/decoding, visualization, and general computation without needing to use tools at all. We're concerned about the implications for AI monitorability : It suggests that models like Astra can use their forward passes to very effectively do a non-trivial amount of opaque and hard-to-monitor computation. Astra can see images without knowing their dimensions The default way a multimodal LLM sees an image is to have it fed in as dense vectors in the same space as its token embeddings. A reasonable assumption might be that the encoder that creates these dense vectors is the part of the system that can "see" images, but we found this is definitely not the case for Astra. We