Speaker
Description
Foundation models embed not only their training examples but the space those examples were drawn from. A transformer trained to write Lagrangians symbolically (such as BART-L) thus embeds the space of theories itself. In this work, we investigate the navigation of this learned theory space using activation steering, adapted from interpretability work on LLMs. Using only small contrastive datasets, we build steering vectors that take us from one theory domain to another. This hence reframes theory construction as a navigation problem, where reaching a theory of interest becomes a matter of finding and following directions, rather than an exhaustive scan of a landscape. We demonstrate navigations such as from anomalous to anomaly-free field content, from a base model to its supersymmetric and gauge-extended counterparts, and from charge-violating to charge-conserving Lagrangians. All of this requires no retraining or finetuning at all. The method also serves its original purpose of interpretability. By testing which operators can be recovered as vectors, we can ask whether the learned theory space is sufficient. We find directions corresponding to several Lorentz and gauge representation changes, including one that maps scalars onto fermions in the manner of a supersymmetry generator, as well as failure modes we associate with the limited size of our model and dataset. These results indicate that the model has, to some degree, embedded a navigable space of theories, and point toward a novel approach to theory search which is directed rather than exhaustive.