Abstract
In this paper, we present a method of embedding physics data manifolds with metric structure into lower dimensional spaces with simpler metrics, such as Euclidean and Hyperbolic spaces. We then demonstrate that it can be a powerful step in the data analysis pipeline for many applications. Using progressively more realistic simulated collisions at the Large Hadron Collider, we show that this embedding approach learns the underlying latent structure. With the notion of volume in Euclidean spaces, we provide for the first time a viable solution to quantifying the true search capability of model agnostic search algorithms in collider physics (i.e. anomaly detection). Finally, we discuss how the ideas presented in this paper can be employed to solve many practical challenges that require the extraction of physically meaningful representations from information in complex high dimensional datasets.
Article PDF
Similar content being viewed by others
Avoid common mistakes on your manuscript.
References
P.T. Komiske, E.M. Metodiev and J. Thaler, Metric space of collider events, Phys. Rev. Lett. 123 (2019) 041801 [arXiv:1902.02346] [INSPIRE].
T. Cai, J. Cheng, K. Craig and N. Craig, Which metric on the space of collider events?, Phys. Rev. D 105 (2022) 076003 [arXiv:2111.03670] [INSPIRE].
S. Kolouri et al., Generalized sliced Wasserstein distances, arXiv:1902.00434.
M. Crispim Romão et al., Use of a generalized energy Mover’s distance in the search for rare phenomena at colliders, Eur. Phys. J. C 81 (2021) 192 [arXiv:2004.09360] [INSPIRE].
S. Tsan et al., Particle graph autoencoders and differentiable, learned energy Mover’s distance, in the proceedings of the 35th conference on neural information processing systems, (2021) [arXiv:2111.12849] [INSPIRE].
K. Fraser et al., Challenges for unsupervised anomaly detection in particle physics, JHEP 03 (2022) 066 [arXiv:2110.06948] [INSPIRE].
J.H. Collins, An exploration of learnt representations of W jets, arXiv:2109.10919 [INSPIRE].
ATLAS collaboration, Measurements of multijet event isotropies using optimal transport with the ATLAS detector, ATLAS-CONF-2022-056, CERN, Geneva, Switzerland (2022).
E.M. Metodiev, B. Nachman and J. Thaler, Classification without labels: learning from mixed samples in high energy physics, JHEP 10 (2017) 174 [arXiv:1708.02949] [INSPIRE].
J.H. Collins, K. Howe and B. Nachman, Anomaly detection for resonant new physics with machine learning, Phys. Rev. Lett. 121 (2018) 241803 [arXiv:1805.02664] [INSPIRE].
J.H. Collins, K. Howe and B. Nachman, Extending the search for new resonances with machine learning, Phys. Rev. D 99 (2019) 014038 [arXiv:1902.02634] [INSPIRE].
B. Nachman and D. Shih, Anomaly detection with density estimation, Phys. Rev. D 101 (2020) 075042 [arXiv:2001.04990] [INSPIRE].
T. Heimel, G. Kasieczka, T. Plehn and J.M. Thompson, QCD or what?, SciPost Phys. 6 (2019) 030 [arXiv:1808.08979] [INSPIRE].
M. Farina, Y. Nakai and D. Shih, Searching for new physics with deep autoencoders, Phys. Rev. D 101 (2020) 075021 [arXiv:1808.08992] [INSPIRE].
O. Cerri et al., Variational autoencoders for new physics mining at the Large Hadron Collider, JHEP 05 (2019) 036 [arXiv:1811.10276] [INSPIRE].
M. Kuusela et al., Semi-supervised anomaly detection — towards model-independent searches of new physics, J. Phys. Conf. Ser. 368 (2012) 012032 [arXiv:1112.3329] [INSPIRE].
T.S. Roy and A.H. Vijay, A robust anomaly finder based on autoencoders, arXiv:1903.02032.
A. Blance, M. Spannowsky and P. Waite, Adversarially-trained autoencoders for robust unsupervised new physics searches, JHEP 10 (2019) 047 [arXiv:1905.10384] [INSPIRE].
J. Hajer, Y.-Y. Li, T. Liu and H. Wang, Novelty detection meets collider physics, Phys. Rev. D 101 (2020) 076015 [arXiv:1807.10261] [INSPIRE].
R.T. D’Agnolo, G. Grosso, M. Pierini, A. Wulzer and M. Zanetti, Learning multivariate new physics, arXiv:1912.12155.
R.T. D’Agnolo and A. Wulzer, Learning new physics from a machine, Phys. Rev. D 99 (2019) 015014 [arXiv:1806.02350] [INSPIRE].
M. Crispim Romão, N.F. Castro and R. Pedro, Finding new physics without learning about it: anomaly detection as a tool for searches at colliders, Eur. Phys. J. C 81 (2021) 27 [Erratum ibid. 81 (2021) 1020] [arXiv:2006.05432] [INSPIRE].
C. Fanelli, J. Giroux and Z. Papandreou, “Flux+Mutability”: a conditional generative approach to one-class classification and anomaly detection, Mach. Learn. Sci. Tech. 3 (2022) 045012 [arXiv:2204.08609] [INSPIRE].
B.M. Dillon, R. Mastandrea and B. Nachman, Self-supervised anomaly detection for new physics, Phys. Rev. D 106 (2022) 056005 [arXiv:2205.10380] [INSPIRE].
S. Alvi, C.W. Bauer and B. Nachman, Quantum anomaly detection for collider physics, JHEP 02 (2023) 220 [arXiv:2206.08391] [INSPIRE].
L. Bradshaw, S. Chang and B. Ostdiek, Creating simple, interpretable anomaly detectors for new physics in jet substructure, Phys. Rev. D 106 (2022) 035014 [arXiv:2203.01343] [INSPIRE].
V.S. Ngairangbam, M. Spannowsky and M. Takeuchi, Anomaly detection in high-energy physics using a quantum autoencoder, Phys. Rev. D 105 (2022) 095004 [arXiv:2112.04958] [INSPIRE].
S. Chekanov and W. Hopkins, Event-based anomaly detection for searches for new physics, Universe 8 (2022) 494 [arXiv:2111.12119] [INSPIRE].
V. Mikuni, B. Nachman and D. Shih, Online-compatible unsupervised nonresonant anomaly detection, Phys. Rev. D 105 (2022) 055006 [arXiv:2111.06417] [INSPIRE].
J.A. Aguilar-Saavedra, Anomaly detection from mass unspecific jet tagging, Eur. Phys. J. C 82 (2022) 130 [arXiv:2111.02647] [INSPIRE].
B. Ostdiek, Deep set auto encoders for anomaly detection in particle physics, SciPost Phys. 12 (2022) 045 [arXiv:2109.01695] [INSPIRE].
G. Kasieczka, B. Nachman and D. Shih, New methods and datasets for group anomaly detection from fundamental physics, in the proceedings of the Conference on knowledge discovery and data mining, (2021) [arXiv:2107.02821] [INSPIRE].
S. Caron, L. Hendriks and R. Verheyen, Rare and different: anomaly scores from a combination of likelihood and out-of-distribution models to detect new physics at the LHC, SciPost Phys. 12 (2022) 077 [arXiv:2106.10164] [INSPIRE].
T. Dorigo et al., RanBox: anomaly detection in the copula space, JHEP 01 (2023) 008 [arXiv:2106.05747] [INSPIRE].
O. Atkinson et al., Anomaly detection with convolutional graph neural networks, JHEP 08 (2021) 080 [arXiv:2105.07988] [INSPIRE].
T. Finke et al., Autoencoders for unsupervised anomaly detection in high energy physics, JHEP 06 (2021) 161 [arXiv:2104.09051] [INSPIRE].
L. van der Maaten and G. Hinton, Visualizing data using t-SNE, J. Mach. Learn. Res. 9 (2008) 2579.
L. McInnes, J. Healy and J. Melville, UMAP: Uniform Manifold Approximation and Projection for dimension reduction, arXiv:1802.03426.
G. Corso et al., Neural distance embeddings for biological sequences, Adv. Neural Inf. Process. Syst. 34 (2021) 18539 [arXiv:2109.09740].
A. Narayanan et al., graph2vec: learning distributed representations of graphs, arXiv:1707.05005.
B. Rozemberczki and R. Sarkar, Fast sequence-based embedding with diffusion graphs, in the proceedings of the International workshop on complex networks, (2018), p. 99.
F. Gong et al., SMR: medical knowledge graph embedding for safe medicine recommendation, Big Data Research 23 (2021) 100174.
N.K. Ahmed et al., Learning role-based graph embeddings, arXiv:1802.02896.
J. Pennington, R. Socher and C. Manning, Glove: global vectors for word representation, in the proceedings of the of the 2014 conference on Empirical Methods in Natural Language Processing (EMNLP), (2014) [https://doi.org/10.3115/v1/d14-1162].
C. Frogner, F. Mirzazadeh and J. Solomon, Learning embeddings into entropic Wasserstein spaces, in the proceedings of the International conference on learning representations, (2019).
A. Akbik, D. Blythe and R. Vollgraf, Contextual string embeddings for sequence labeling, in the proceedings of the of the 27th International conference on computational linguistics, Santa Fe, NM, U.S.A. (2018), p. 1638.
R. Bartusiak et al., WordNet2Vec: corpora agnostic word vectorization method, Neurocomputing 326-327 (2019) 141.
A. Sanakoyeu, V. Tschernezki, U. Buchler and B. Ommer, Divide and conquer the embedding space for metric learning, in the proceedings of the of the IEEE/CVF conference on Computer Vision and Pattern Recognition, (2019), p. 471.
D. Garcia-Gasulla et al., A visual embedding for the unsupervised extraction of abstract semantics, Cognitive Systems Research 42 (2017) 73.
B.M. Dillon et al., Symmetries, safety, and self-supervision, arXiv:2108.04253.
B.M. Dillon, R. Mastandrea and B. Nachman, Self-supervised anomaly detection for new physics, Phys. Rev. D 106 (2022) 056005 [arXiv:2205.10380] [INSPIRE].
A. Paszke et al., PyTorch: an imperative style, high-performance deep learning library, in Advances in neural information processing systems 32, Curran Associates Inc., U.S.A. (2019), p. 8024.
Y. LeCun and C. Cortes, MNIST handwritten digit database, http://yann.lecun.com/exdb/mnist/.
N. Courty, R. Flamary and M. Ducoffe, Learning Wasserstein embeddings, arXiv:1710.07457.
M. Cacciari, G.P. Salam and G. Soyez, The anti-kt jet clustering algorithm, JHEP 04 (2008) 063 [arXiv:0802.1189] [INSPIRE].
M. Cacciari, G.P. Salam and G. Soyez, FastJet user manual, Eur. Phys. J. C 72 (2012) 1896 [arXiv:1111.6097] [INSPIRE].
CMS collaboration, Measurement of the splitting function in pp and Pb-Pb collisions at \( \sqrt{s_{NN}} \) = 5.02 TeV, Phys. Rev. Lett. 120 (2018) 142302 [arXiv:1708.09429] [INSPIRE].
A Large Ion Collider Experiment and ALICE collaborations, Measurement of the groomed jet radius and momentum splitting fraction in pp and Pb-Pb collisions at \( \sqrt{s_{NN}} \) = 5.02 TeV, Phys. Rev. Lett. 128 (2022) 102001 [arXiv:2107.12984] [INSPIRE].
ATLAS collaboration, Properties of g → \( b\overline{b} \) at small opening angles in pp collisions with the ATLAS detector at \( \sqrt{s} \) = 13 TeV, Phys. Rev. D 99 (2019) 052004 [arXiv:1812.09283] [INSPIRE].
A. Vaswani et al., Attention is all you need, Adv. Neural Inf. Process. Syst. 30 (2017) [arXiv:1706.03762].
DELPHES collaboration, DELPHES 3, a modular framework for fast simulation of a generic collider experiment, JHEP 02 (2014) 057 [arXiv:1307.6346] [INSPIRE].
J. Alwall et al., The automated computation of tree-level and next-to-leading order differential cross sections, and their matching to parton shower simulations, JHEP 07 (2014) 079 [arXiv:1405.0301] [INSPIRE].
P. Skands, S. Carrazza and J. Rojo, Tuning PYTHIA 8.1: the Monash 2013 tune, Eur. Phys. J. C 74 (2014) 3024 [arXiv:1404.5630] [INSPIRE].
T. Sjöstrand et al., An introduction to PYTHIA 8.2, Comput. Phys. Commun. 191 (2015) 159 [arXiv:1410.3012] [INSPIRE].
M. Cacciari and G.P. Salam, Dispelling the N 3 myth for the kt jet-finder, Phys. Lett. B 641 (2006) 57 [hep-ph/0512210] [INSPIRE].
M. Cacciari, G.P. Salam and G. Soyez, FastJet user manual, Eur. Phys. J. C 72 (2012) 1896 [arXiv:1111.6097] [INSPIRE].
M. Nickel and D. Kiela, Poincaré embeddings for learning hierarchical representations, Adv. Neural Inf. Process. Syst. 30 (2017) [arXiv:1705.08039].
W. Peng et al., Hyperbolic deep neural networks: a survey, arXiv:2101.04562.
A. Klimovskaia, D. Lopez-Paz, L. Bottou and M. Nickel, Poincaré maps for analyzing complex hierarchies in single-cell data, Nature Commun. 11 (2020) 1.
L. Chennuru Vankadara and U. von Luxburg, Measures of distortion for machine learning, in the proceedings of the Advances in neural information processing systems 31, Curran Associates, Inc., U.S.A. (2018).
P.T. Komiske et al., Exploring the space of jets with CMS open data, Phys. Rev. D 101 (2020) 034009 [arXiv:1908.08542] [INSPIRE].
R. Flamary et al., POT: Python Optimal Transport, J. Mach. Learn. Res. 22 (2021) 1.
S. Ioffe and C. Szegedy, Batch normalization: accelerating deep network training by reducing internal covariate shift, arXiv:1502.03167 [INSPIRE].
N. Srivastava et al., Dropout: a simple way to prevent neural networks from overfitting, J. Mach. Learn. Res. 15 (2014) 1929.
Acknowledgments
We thank Jesse Thaler, Matthew Schwartz, and Javier Duarte for useful discussions and comments. Additionally, we thank the discussion group with Katherine Fraser, Samuel Homiller, Rashmish K. Mishra, and Patrick McCormack where the idea for this paper originated. P.H. acknowledges support by DOE grant de-sc0021943 and NSF CSSI award #1934700. SEP acknowledges support by DOE grant DE-SC0021225, and the Institute for Fundamental Interactions and Artificial Intelligence (NSF Award #2019786). We thank B. Wyslouch, J. Formaggio, and P. Fisher for providing office space on the 5th floor of MIT building 24.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
ArXiv ePrint: 2208.05484
Rights and permissions
Open Access . This article is distributed under the terms of the Creative Commons Attribution License (CC-BY 4.0), which permits any use, distribution and reproduction in any medium, provided the original author(s) and source are credited.
About this article
Cite this article
Park, S.E., Harris, P. & Ostdiek, B. Neural embedding: learning the embedding of the manifold of physics data. J. High Energ. Phys. 2023, 108 (2023). https://doi.org/10.1007/JHEP07(2023)108
Received:
Revised:
Accepted:
Published:
DOI: https://doi.org/10.1007/JHEP07(2023)108