Conveners
🔀 Foundation Models
- Luigi Favaro (UCLouvain - CP3)
🔀 Foundation Models
- Tobias Golling (University of Geneva)
🔀 Foundation Models
- Luca Fallböhmer (Max-Planck-Institute for Nuclear Physics)
-
Matthias Vigl (TUM)25/08/2026, 11:00Foundation Models
Recent progress in machine learning has come as much from scale as from architecture, and high-energy physics is starting to follow: work on transformer scaling for flavour tagging in ATLAS [https://inspirehep.net/literature/3114069] has shown that tagger performance follows predictable power laws over orders of magnitude in compute on a multi-billion-jet dataset. This matters particularly in...
Go to contribution page -
Anna Hallin (anna.hallin@uni-hamburg.de)25/08/2026, 11:10Foundation Models
Scaling laws have become a central topic in modern machine learning, providing a quantitative understanding of how model performance improves with increasing model size, training data, and compute. They also offer insights into whether learning is approaching the information limits of a given dataset. In this contribution, we present the first study of neural scaling laws for generative jet...
Go to contribution page -
Joaquin Iturriza Ramirez (LPNHE - Sorbonne Université)25/08/2026, 11:20Foundation Models
Fast and precise evaluations of scattering amplitudes even in the case of precision calculations is essential for event generation tools at the HL-LHC. We explore the scaling behavior of the achievable precision of neural networks in this regression problem for multiple architectures, including a Lorentz symmetry aware multilayer perceptron and a fully Lorentz equivariant transformer using...
Go to contribution page -
Yong Sheng Koay (Uppsala University)25/08/2026, 11:40Foundation Models
Collider simulations simultaneously provide three sources of information: whether an event is signal or background, the physics parameters that generated each signal event, and the event kinematics from which signal regions are defined. Conventional pipelines use these sources in separate stages—training classifiers for signal discrimination, performing parameter inference for fixed analysis...
Go to contribution page -
Thanush Sivagnanalingam (University Heidelberg)25/08/2026, 11:50Foundation Models
Neural network training for LHC event generation should, ideally, benefit from common high-level patterns in different processes. We propose novel conditioning schemes for continuous parameters, process labels, and Feynman diagrams. We employ pre-trained LLMs as multi-modal foundation models to provide descriptive embeddings for an autoregressive transformer. With such high-level...
Go to contribution page -
Philipp Niedermayer (GSI GmbH)25/08/2026, 12:00Foundation Models
Large-scale accelerator facilities such as GSI/FAIR rely heavily on distributed expertise and fragmented documentation for daily operations, creating persistent challenges in knowledge retention, troubleshooting speed, and workforce continuity. To address these operational bottlenecks, we present an AI assistant designed to deliver context aware, expert level guidance to shift operators....
Go to contribution page -
Shreya Saha (Adelaide University)26/08/2026, 14:00Foundation Models
The integration of foundation models in particle physics is gaining pace rapidly and has expanded the search for new physics. This talk presents foundation models trained on low-level data from the first fully simulated dataset using Open Data Detector (ColliderML), to distinguish between Standard Model and Beyond Standard Model processes. We compare new physics discovery using only low level...
Go to contribution page -
Guillaume Letellier (GREYC - University of Caen Normandy, France)26/08/2026, 14:10Foundation Models
Jet tagging at the Large Hadron Collider (LHC) increasingly relies on deep learning models trained on massive simulated datasets, leading to high computational costs and limited robustness to detector mismodeling. We introduce JetParticle-JEPA (JP-JEPA), a self-supervised Joint-Embedding Predictive Architecture that learns physically meaningful jet representations directly from continuous...
Go to contribution page -
Desheng Yang (Sapienza Università di Roma and INFN sezione di Roma)26/08/2026, 14:20Foundation Models
Foundation models have recently emerged as powerful tools for learning general-purpose representations that can outperform task-specific models while being generalizable to multiple downstream tasks. In this project, we explore their application to particle physics by training a foundation model on simulated jet data from hadronic collisions. The goal is to build a model that can learn...
Go to contribution page -
Una Alberti (University of Bern)26/08/2026, 14:40Foundation Models
The high-energy physics pipeline already resembles a multi-modal foundation model: a general-purpose representation built once and reused across downstream tasks - but it lacks scale and differentiability. Already, we are seeing the gains from large-scale pre-training in individual pieces of the ATLAS pipeline, such as jet flavour identification. Flavour taggers, however, are normally trained...
Go to contribution page -
Giovanni Ottaviano (Sorbonne University)26/08/2026, 14:50Foundation Models
Tokenization plays a central role in modern foundation models, especially those relying on next-token prediction, and has been extensively studied in domains such as language and vision. In high-energy physics, however, the adaptation of tokenization to continuous jet data from collider experiments remains at an early stage, with few studies on tokenizer performance, comparable evaluation...
Go to contribution page -
Henning Rose (Uni Hamburg)26/08/2026, 15:00Foundation Models
Autoregressive transformers are increasingly applied outside the language domain that motivated them, but the tokenization step transfers poorly. Scientific data are already numerical, often partly discrete, and typically live in high-dimensional spaces where each token carries several features. The standard workaround, compressing feature vectors into a single token via a learned codebook,...
Go to contribution page -
Yannic Pietschke (Institute for Theoretical Physics, Heidelberg)27/08/2026, 14:00Foundation Models
Simulation-based inference (SBI) has become the default for intractable-likelihood problems across fundamental physics, but is limited by model misspecification: a density estimator trained on one simulator becomes inaccurate or miscalibrated when applied to data drawn from a different forward model or with unseen instrumental systematics. We study this in 21cm cosmology, where the...
Go to contribution page -
Johann Ioannou-Nikolaides (Niels Bohr Institute), Raphaël Bonnet-Guerrini (Computer Science Dep. University of Milan)27/08/2026, 14:10Foundation Models
Pretrained foundation models are emerging in neutrino telescopes, but what their internal representations actually contain, and whether the tasks built on top of them put it to use, is largely unknown. We study PolarBERT, a transformer pretrained on IceCube data and fine-tuned for direction reconstruction. Sparse autoencoders uncover a validated atlas of physics across the frozen backbone,...
Go to contribution page -
João A. Gonçalves (U. Bonn, B-it, Lamarr Institute)27/08/2026, 14:20Foundation Models
Jet quenching modifies the energy and substructure of jets in heavy-ion collisions, but the corresponding inverse problem remains largely unexplored at the constituent level: given a medium-modified jet, infer the vacuum jet associated with the same parton shower. We formulate this low-level inverse jet-quenching task using paired HYBRID simulations, in which each quenched shower is obtained...
Go to contribution page -
Thomas Gessey-Jones (PhysicsX)Foundation Models
Inferring a three-dimensional shape from sparse, indirect measurements is a severely ill-posed inverse problem, which arises throughout the physical and biological sciences. Doing so successfully with principled Bayesian uncertainties requires a tractable parametrisation of geometry, and a strong prior encoding domain knowledge of plausible shapes.
In this talk we show how a pretrained...
Go to contribution page