Abstract

Telemetry data acquisition is becoming crucial for efficient detection and timely reaction in the case of network status changes, such as failures. Streaming telemetry data to many collectors might be hindered by scalability issues, causing delay in localization and detection procedures. Providing efficient mechanisms for managing the massive telemetry traffic coming from network devices can pave the way to novel procedures, speeding up failure detection and thus minimizing response time. This paper proposes a novel Kafka-based monitoring framework leveraging the telemetry service. The proposed framework exploits the built-in scalability and reliability of Kafka to go beyond traditional monitoring systems. The framework allows a continuous monitoring of optical system data and their distribution through simple compressed text messages to a large number of consumers. Moreover, the proposed framework keeps a limited history of the monitored data, easing, for example, root cause failure analysis. The implemented monitoring platform is experimentally validated, considering the disaggregated paradigm, in terms of functional assessment, scalability, resiliency, and end-to-end message latency. Obtained results show that the framework is highly scalable, supporting up to around 4000 messages per second (and potentially more) with low CPU load, and is capable of achieving an end-to-end (i.e., producer–consumer) latency of about 50 ms. Moreover, the considered architecture is capable of overcoming the failure of a monitoring framework core component without losing any message.

© 2021 Optical Society of America

Unified monitoring and telemetry platform supporting network intelligence in optical networks

Sen Shen, Jing Han, Klodian Bardhi, Haiyuan Li, Ruizhi Yang, Yiran Teng, Vaigai Yokar, Shuangyi Yan, and Dimitra Simeonidou
J. Opt. Commun. Netw. 17(2) 139-151 (2025)

Distributed intelligence for pervasive optical network telemetry

Luis Velasco, Pol González, and Marc Ruiz
J. Opt. Commun. Netw. 15(9) 676-686 (2023)

Machine-learning-based soft-failure localization with partial software-defined networking telemetry

Kayol S. Mayer, Jonathan A. Soares, Rossano P. Pinto, Christian E. Rothenberg, Dalton S. Arantes, and Darli A. A. Mello
J. Opt. Commun. Netw. 13(10) E122-E131 (2021)

Peer-to-peer disaggregated telemetry for autonomic machine-learning-driven transceiver operation

Francesco Paolucci, Andrea Sgambelluri, Moises Felipe Silva, Alessandro Pacini, Piero Castoldi, Luca Valcarenghi, and Filippo Cugini
J. Opt. Commun. Netw. 14(8) 606-620 (2022)

Extending P4 in-band telemetry to user equipment for latency- and localization-aware autonomous networking with AI forecasting

Davide Scano, Francesco Paolucci, Koteswararao Kondepu, Andrea Sgambelluri, Luca Valcarenghi, and Filippo Cugini
J. Opt. Commun. Netw. 13(9) D103-D114 (2021)