Speaker
Description
The petabyte-scale data generated by High Energy Physics (HEP) experiments presents a significant storage challenge. We present the Bytewise Online Autoregressive (BOA) Constrictor, a new pseudo-streaming lossless neural compressor built upon the Mamba state space model. BOA achieves competitive compression ratios across diverse structured HEP datasets, matching or exceeding LZMA, ZSTD and ZLIB at maximum compression, among other tested algorithms. BOA also demonstrates robust cross-file and cross-condition generalisation on CMS Open Data (NanoAOD format), where it obtains comparable or improved effective compression ratios (within 5%) with respect to the next-best traditional algorithm. Ablation studies show that transitioning to half-precision (FP16) weights reduces the model footprint without degrading predictive accuracy. The model has also been tested in other kinds of scientific data. BOA is supported by a deterministic reference C++ implementation which ensures bit-exact reproducibility across different CUDA architectures. In this proof-of-principle implementation, BOA delivers a decompression throughput that is not yet competitive with optimised algorithms such as ZSTD or LZMA, but still provides a first step towards data compression improvements for next-generation scientific data. This contribution summarises the work done by several summer students on inference acceleration, benchmarking, and environmental impact break-even point between equivalent carbon of inference/training vs embodied carbon for storage.