Speaker
Description
Maintaining data quality in real-time systems remains a fundamental requirement in nuclear and particle physics experiments. At the HADES collaboration, the Quality Assessment (QA) system generates plots that visualize the real-time performance of detector subsystems. Currently this requires that operators manually inspect these visualizations to identify potential anomalies.
This contribution presents an approach to automate this task using unsupervised machine learning. To this end, we first established a supervised Convolutional Neural Network (CNN) baseline for comparison, and then propose a fully automated anomaly detection algorithm. We modeled key detector plots by injecting realistic anomalies observed during beamtime. We then tested unsupervised models, choosing to combine a Variational Autoencoder (VAE) with HDBSCAN density-based clustering (VAE-HDBSCAN), where the VAE's CNN-encoded latent space serves as the feature representation for clustering. This approach eliminates the need for prelabeled data or constant human oversight.
As we demonstrate, VAE-HDBSCAN measurably outperforms non-latent clustering, achieving higher precision in detecting subtle per-bin shifts while maintaining a low false-positive rate. This unsupervised method achieved efficiencies comparable to supervised models on our controlled dataset, demonstrating the feasibility of training and deploying unsupervised machine learning during data-taking periods.
To facilitate adoption by operators, we are integrating this framework into Jefferson Lab's HYDRA QA system, a web-based data quality platform for online monitoring, labeling, and model training. JLab-HYDRA's accessible front end makes automated anomaly detection practical for operators without a machine learning background.