Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

A review of machine learning-based failure management in optical networks

  • Review
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Failure management plays a significant role in optical networks. It ensures secure operation, mitigates potential risks, and executes proactive protection. Machine learning (ML) is considered to be an extremely powerful technique for performing comprehensive data analysis and complex network management and is widely utilized for failure management in optical networks to revolutionize the conventional manual methods. In this study, the background of failure management is introduced, where typical failure tasks, physical objects, ML algorithms, data sources, and extracted information are illustrated in detail. An overview of the applications of ML in failure management is provided in terms of alarm analysis, failure prediction, failure detection, failure localization, and failure identification. Finally, the future directions on ML for failure management are discussed from the perspective of data, model, task, and emerging techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Musumeci F, Rottondi C, Corani G, et al. A tutorial on machine learning for failure management in optical networks. J Lightwave Technol, 2019, 37: 4125–4139

    Article  Google Scholar 

  2. Rafique D, Velasco L. Machine learning for network automation: overview, architecture, and applications. J Opt Commun Netw, 2018, 10: D126

    Article  Google Scholar 

  3. Musumeci F, Rottondi C, Nag A, et al. An overview on application of machine learning techniques in optical networks. IEEE Commun Surv Tut, 2019, 21: 1383–1408

    Article  Google Scholar 

  4. Khan F N, Fan Q, Lu C, et al. An optical communication’s perspective on machine learning and its applications. J Lightwave Technol, 2019, 37: 493–516

    Article  Google Scholar 

  5. Musumeci F. Machine learning for failure management in optical networks. In: Proceedings of Optical Fiber Communication Conference and Exhibition (OFC), San Francisco, 2021

  6. Wang D S, Wang D D, Zhang C Y, et al. Machine learning for optical layer failure management. In: Proceedings of the 26th Opto-Electronics and Communications Conference (OECC), Hong Kong, 2021

  7. Panayiotou T, Chatzis S P, Ellinas G. Leveraging statistical machine learning to address failure localization in optical networks. J Opt Commun Netw, 2018, 10: 162–173

    Article  Google Scholar 

  8. Barzegar S, Ruiz M, Sgambelluri A, et al. Soft-failure detection, localization, identification, and severity prediction by estimating QoT model input parameters. IEEE Trans Netw Serv Manage, 2021, 18: 2627–2640

    Article  Google Scholar 

  9. Musumeci F, Venkata V G, Hirota Y, et al. Domain adaptation and transfer learning for failure detection and failure-cause identification in optical networks across different lightpaths. J Opt Commun Netw, 2022, 14: 91–100

    Article  Google Scholar 

  10. Musumeci F, Venkata V G, Hirota Y, et al. Transfer learning across different lightpaths for failure-cause identification in optical networks. In: Proceedings of 2020 European Conference on Optical Communications (ECOC), Brussels, 2020. 1–4

  11. Abdelli K, Grießer H, Ehrle P, et al. Reflective fiber fault detection and characterization using long short-term memory. J Opt Commun Netw, 2021, 13: 32–41

    Article  Google Scholar 

  12. Shariati B, Ruiz M, Comellas J, et al. Learning from the optical spectrum: failure detection and identification. J Lightwave Technol, 2019, 37: 433–440

    Article  Google Scholar 

  13. Liu D M, Yang Y J, Tang Z F, et al. Implementation of optical module performance prediction and maintenance on data-driven. In: Proceedings of the 8th Symposium on Novel Photoelectronic Detection Technology and Applications, 2022. 12169: 3332–3336

    Google Scholar 

  14. Kruse L, Pachnicke S. EDFA soft-failure detection and lifetime prediction based on spectral data using 1-D convolutional neural network. In: Proceedings of the 22nd ITG Symposium, VDE, 2021. 1–6

  15. LeFevre B G, King W W, Hardee A G, et al. Failure analysis of connector-terminated optical fibers: two case studies. J Lightwave Technol, 1993, 11: 537–541

    Article  Google Scholar 

  16. Gu R T, Yang Z Y, Ji Y F. Machine learning for intelligent optical networks: a comprehensive survey. J Network Comput Appl, 2020, 157: 102576

    Article  Google Scholar 

  17. Wang D S, Zhang M. Artificial intelligence in optical communications: from machine learning to deep learning. Front Comms Net, 2021, 2: 656786

    Article  Google Scholar 

  18. Valcarenghi L, Pacini A, Sgambelluri A, et al. A scalable telemetry framework for zero touch optical network management. In: Proceedings of 2021 International Conference on Optical Network Design and Modeling (ONDM), Gothenburg, 2021. 1–6

  19. Sgambelluri A, Pacini A, Paolucci F, et al. Reliable and scalable Kafka-based framework for optical network telemetry. J Opt Commun Netw, 2021, 13: 42–52

    Article  Google Scholar 

  20. Stanic S, Subramaniam S, Sahin G, et al. Active monitoring and alarm management for fault localization in transparent all-optical networks. IEEE Trans Netw Serv Manage, 2010, 7: 118–131

    Article  Google Scholar 

  21. Wang D S, Zhang M, Zhang Z G, et al. Machine learning-based multifunctional optical spectrum analysis technique. IEEE Access, 2019, 7: 19726–19737

    Article  Google Scholar 

  22. Locatelli F, Christodoulopoulos K, Moreolo M S, et al. Spectral processing techniques for efficient monitoring in optical networks. J Opt Commun Netw, 2021, 13: 158–168

    Article  Google Scholar 

  23. Tanaka T, Inui T, Kawai S, et al. Monitoring and diagnostic technologies using deep neural networks for predictive optical network maintenance. J Opt Commun Netw, 2021, 13: 13–22

    Article  Google Scholar 

  24. Wang D S, Zhang M, Li J, et al. Intelligent constellation diagram analyzer using convolutional neural network-based deep learning. Opt Express, 2017, 25: 17150–17166

    Article  Google Scholar 

  25. Wang D, Zhang M, Li Z, et al. Modulation format recognition and OSNR estimation using CNN-based deep learning. IEEE Photon Technol Lett, 2017, 29: 1667–1670

    Article  Google Scholar 

  26. Sasai T, Nakamura M, Yamazaki E, et al. Digital longitudinal monitoring of optical fiber communication link. J Lightwave Technol, 2022, 40: 2390–2408

    Article  Google Scholar 

  27. Sasai T, Nakamura M, Yamazaki E, et al. Digital backpropagation for optical path monitoring: loss profile and passband narrowing estimation. In: Proceedings of 2020 European Conference on Optical Communications (ECOC), Brussels, 2020. 1–4

  28. Lun H Z, Wu Y W, Cai M, et al. ROADM-induced anomaly localization and evaluation for optical links based on receiver DSP and ML. J Lightwave Technol, 2021, 39: 2696–2703

    Article  Google Scholar 

  29. Dong Z H, Khan F N, Sui Q, et al. Optical performance monitoring: a review of current and future technologies. J Lightwave Technol, 2016, 34: 525–543

    Article  Google Scholar 

  30. Wang D W, Jiang H, Liang G W, et al. Optical performance monitoring of multiple parameters in future optical networks. J Lightwave Technol, 2021, 39: 3792–3800

    Article  Google Scholar 

  31. Shi Y, Wang Y Y, Zhao L, et al. An event recognition method for Φ-OTDR sensing system based on deep learning. Sensors, 2019, 19: 3421

    Article  Google Scholar 

  32. Wu H J, Liu X R, Xiao Y, et al. A dynamic time sequence recognition and knowledge mining method based on the hidden Markov models (HMMs) for pipeline safety monitoring with Φ-OTDR. J Lightwave Technol, 2019, 37: 4991–5000

    Article  Google Scholar 

  33. Zhao Y L, Yan B Y, Liu D M, et al. SOON: self-optimizing optical networks with machine learning. Opt Express, 2018, 26: 28713–28726

    Article  Google Scholar 

  34. Yan B Y, Zhao Y L, Rahman S, et al. Dirty-data-based alarm prediction in self-optimizing large-scale optical networks. Opt Express, 2019, 27: 10631–10643

    Article  Google Scholar 

  35. Zhang B, Zhao Y L, Li Y J, et al. Cognitive network management based on cross-layer AI interaction in ONOS-enabled self-optimizing optical networks. In: Proceedings of Asia Communications and Photonics Conference (ACP), Chengdu, 2019. 1–3

  36. Zhuang H T, Zhao Y L, Yu X S, et al. Machine-learning-based alarm prediction with GANs-based self-optimizing data augmentation in large-scale optical transport networks. In: Proceedings of International Conference on Computing, Networking and Communications (ICNC), Hawaii, 2020. 294–298

  37. Zhang B, Zhao Y L, Li Y J, et al. Transfer learning aided concurrent multi-alarm prediction in optical transport networks. In: Proceedings of Asia Communications and Photonics Conference, Beijing, 2020

  38. Zhao Y J, Yan B Y, Li Z T, et al. Coordination between control layer AI and on-board AI in optical transport networks. J Opt Commun Netw, 2020, 12: 49–57

    Article  Google Scholar 

  39. Liu T Y, Mei H Y, Sun Q, et al. Application of neural network in fault location of optical transport network. China Commun, 2019, 16: 214–225

    Article  Google Scholar 

  40. Zhao X D, Yang H, Guo H F, et al. Accurate fault location based on deep neural evolution network in optical networks for 5G and beyond. In: Proceedings of Optical Fiber Communication Conference and Exhibition (OFC), San Diego, 2019

  41. Li Z, Zhao Y, Li Y, et al. Demonstration of fault localization in optical networks based on knowledge graph and graph neural network. In: Proceedings of Optical Fiber Communication Conference and Exhibition (OFC), San Diego, 2020

  42. Li Z T, Zhao Y L, Li Y J, et al. Demonstration of alarm knowledge graph construction for fault localization on ONOS-based SDON platform. In: Proceedings of Optical Fiber Communications Conference and Exhibition (OFC), San Diego, 2020. 1–3

  43. Li Z T, Zhao Y L, Li Y J, et al. Fault localization based on knowledge graph in software-defined optical networks. J Lightwave Technol, 2021, 39: 4236–4246

    Article  Google Scholar 

  44. Lou L Q, Zhang M, Wang D S, et al. Alarm compression based on machine learning and association rules mining in optical networks. In: Proceedings of the 23rd Opto-Electronics and Communications Conference (OECC), Jeju Island, 2018. 1–2

  45. Wang D S, Lou L Q, Zhang M, et al. Dealing with alarms in optical networks using an intelligent system. IEEE Access, 2019, 7: 97760–97770

    Article  Google Scholar 

  46. Jia J W, Wang D S, Zhang C Y, et al. Transformer-based alarm context-vectorization representation for reliable alarm root cause identification in optical networks. In: Proceedings of European Conference on Optical Communication (ECOC), Bordeaux, 2021. 1–4

  47. Lu J N, Zhou G, Fan Q, et al. Performance comparisons between machine learning and analytical models for quality of transmission estimation in wavelength-division-multiplexed systems. J Opt Commun Netw, 2021, 13: B35

    Article  Google Scholar 

  48. Shariati B, Boitier F, Ruiz M, et al. Autonomic transmission through pre-FEC BER degradation prediction based on SOP monitoring. In: Proceedings of European Conference on Optical Communication (ECOC), Rome, 2018. 1–3

  49. Inuzuka F, Oda T, Tanaka T, et al. Demonstration of a novel framework for proactive maintenance using failure prediction and bit lossless protection with autonomous network diagnosis system. J Lightwave Technol, 2020, 38: 2695–2702

    Article  Google Scholar 

  50. Wang Z L, Zhang M, Wang D S, et al. Failure prediction using machine learning and time series in optical network. Opt Express, 2017, 25: 18553–18565

    Article  Google Scholar 

  51. Zhang C Y, Wang D S, Wang L L, et al. Temporal data-driven failure prognostics using BiGRU for optical networks. J Opt Commun Netw, 2020, 12: 277–287

    Article  Google Scholar 

  52. Zhang C Y, Wang D S, Jia J W, et al. Attention mechanism-driven potential fault cause identification in optical networks. In: Proceedings of 2021 Optical Fiber Communications Conference and Exhibition (OFC), San Francisco, 2021. 1–3

  53. Wang L L, Wang D S, Zhang C Y, et al. Uncertainty analysis for failure prediction in optical transport network using Bayesian neural network. In: Proceedings of Optical Fiber Communication Conference and Exhibition (OFC), San Francisco, 2021

  54. Smalakys L, Melninkaitis A. Predicting lifetime of optical components with Bayesian inference. Opt Express, 2021, 29: 903–915

    Article  Google Scholar 

  55. Abdelli K, Rafique D, Grießer H, et al. Lifetime prediction of 1550 nm DFB laser using machine learning techniques. In: Proceedings of Optical Fiber Communication Conference, San Diego, 2020

  56. Song H K, Li Y J, Liu M Z, et al. Experimental study of machine-learning-based detection and location of eavesdropping in end-to-end optical fiber communications. Optical Fiber Tech, 2022, 68: 102669

    Article  Google Scholar 

  57. Shu L, Yu Z M, Wan Z Q, et al. Dual-stage soft failure detection and identification for low-margin elastic optical network by exploiting digital spectrum information. J Lightwave Technol, 2019, 38: 2669–2679

    Article  Google Scholar 

  58. Shu L, Yu Z M, Wan Z Q, et al. Low-complexity dual-stage soft failure detection by exploiting digital spectrum information. In: Proceedings of 2019 European Conference on Optical Communication (ECOC), Dublin, 2019. 1–4

  59. Shu L, Yu Z M, Wan Z Q, et al. Low-complexity storage-reduced digital spectrum-based soft-failure management with Welch’s method. Opt Express, 2020, 28: 12529–12541

    Article  Google Scholar 

  60. Lun H Z, Liu X M, Cai M, et al. GAN based soft failure detection and identification for long-haul coherent transmission systems. In: Proceedings of 2021 Optical Fiber Communications Conference and Exhibition (OFC), San Francisco, 2021. 1–3

  61. Varughese S, Lippiatt D, Richter T, et al. Identification of soft failures in optical links using low complexity anomaly detection. In: Proceedings of Optical Fiber Communications Conference and Exhibition (OFC), San Diego, 2019

  62. Varughese S, Lippiatt D, Richter T, et al. Low complexity soft failure detection and identification in optical links using adaptive filter coefficients. In: Proceedings of 2020 Optical Fiber Communications Conference and Exhibition (OFC), San Diego, 2020. 1–3

  63. Vela A P, Ruiz M, Fresi F, et al. BER degradation detection and failure identification in elastic optical networks. J Lightwave Technol, 2017, 35: 4595–4604

    Article  Google Scholar 

  64. Shahkarami S, Musumeci F, Cugini F, et al. Machine-learning-based soft-failure detection and identification in optical networks. In: Proceedings of 2018 Optical Fiber Communications Conference and Exhibition (OFC), San Diego, 2018. 1–3

  65. Shariati B, Vela A P, Ruiz M, et al. Monitoring and data analytics: analyzing the optical spectrum for soft-failure detection and identification. In: Proceedings of 2018 International Conference on Optical Network Design and Modeling (ONDM), Dublin, 2018. 260–265

  66. Velasco L, Shariati B, Vela A P, et al. Learning from the optical spectrum: soft-failure identification and localization. In: Proceedings of 2018 Optical Fiber Communications Conference and Exhibition (OFC), San Diego, 2018. 1–3

  67. Boitier F, Lemaire V, Pesic J, et al. Proactive fiber damage detection in real-time coherent receiver. In: Proceedings of European Conference on Optical Communication (ECOC), Gothenburg, 2017. 1–3

  68. Liu S L, Wang D S, Zhang C Y, et al. Semi-supervised anomaly detection with imbalanced data for failure detection in optical networks. In: Proceedings of Optical Fiber Communication Conference and Exhibition (OFC), San Francisco, 2021

  69. Rafique D, Szyrkowiec T, Grieser H, et al. Cognitive assurance architecture for optical network fault management. J Lightwave Technol, 2017, 36: 1443–1450

    Article  Google Scholar 

  70. Abdelli K, Rafique D, Pachnicke S. Machine learning based laser failure mode detection. In: Proceedings of 2019 21st International Conference on Transparent Optical Networks (ICTON), Angers, 2019. 1–4

  71. Chen X L, Li B J, Proietti R, et al. Self-taught anomaly detection with hybrid unsupervised/supervised machine learning in optical networks. J Lightwave Technol, 2019, 37: 1742–1749

    Article  Google Scholar 

  72. Chen X L, Liu C Y, Proietti R, et al. On cooperative fault management in multi-domain optical networks using hybrid learning. IEEE J Sel Top Quantum Electron, 2022, 28: 1–9

    Google Scholar 

  73. Furdek M, Natalino C, Giglio A D, et al. Optical network security management: requirements, architecture, and efficient machine learning models for detection of evolving threats. J Opt Commun Netw, 2021, 13: A144

    Article  Google Scholar 

  74. Lun H Z, Liu X M, Cai M, et al. Anomaly localization in optical transmissions based on receiver DSP and artificial neural network. In: Proceedings of Optical Fiber Communication Conference and Exhibition (OFC), San Diego, 2020

  75. Mayer K S, Soares J A, Pinto R P, et al. Soft failure localization using machine learning with SDN-based network-wide telemetry. In: Proceedings of 2020 European Conference on Optical Communications (ECOC), Brussels, 2020. 1–4

  76. Mayer K S, Soares J A, Pinto R P, et al. Machine-learning-based soft-failure localization with partial software-defined networking telemetry. J Opt Commun Netw, 2021, 13: 122–131

    Article  Google Scholar 

  77. Barzegar S, Virgillito E, Ruiz M, et al. Soft-failure localization and device working parameters estimation in disaggregated scenarios. In: Proceedings of Optical Fiber Communication Conference and Exhibition (OFC), San Diego, 2020

  78. Gifre L, Izquierdo-Zaragoza J L, Shariati B, et al. Experimental demonstration of active and passive optical networks telemetry. In: Proceedings of Optical Fiber Communication Conference and Exhibition (OFC), San Diego, 2018

  79. Vela A P, Shariati B, Ruiz M, et al. Soft failure localization during commissioning testing and lightpath operation. J Opt Commun Netw, 2018, 10: A27

    Article  Google Scholar 

  80. Christodoulopoulos K, Sambo N, Varvarigos E. Exploiting network kriging for fault localization. In: Proceedings of Optical Fiber Communication Conference and Exhibition (OFC), 2016

  81. Sasai T, Nakamura M, Yamazaki E, et al. Physics-oriented learning of nonlinear Schrödinger equation: optical fiber loss and dispersion profile identification. 2021. ArXiv:2104.05890

  82. Lun H Z, Fu M F, Liu X M, et al. Soft failure identification for long-haul optical communication systems based on one-dimensional convolutional neural network. J Lightwave Technol, 2020, 38: 2992–2999

    Article  Google Scholar 

  83. Zhang C Y, Wang D S, Song C, et al. Interpretable learning algorithm based on XGboost for fault prediction in optical network. In: Proceedings of 2020 Optical Fiber Communications Conference and Exhibition (OFC), San Diego, 2020. 1–3

  84. Zhang C Y, Wang D S, Wang L L, et al. Cause-aware failure detection using an interpretable XGBoost for optical networks. Opt Express, 2021, 29: 31974–31992

    Article  Google Scholar 

  85. Zhang C Y, Wang D S, Jia J W, et al. Potential failure cause identification for optical networks using deep learning with an attention mechanism. J Opt Commun Netw, 2022, 14: A122

    Article  Google Scholar 

  86. Jiang X T, Wang D S, Fan Q R, et al. Solving the nonlinear Schrödinger equation in optical fibers using physics-informed neural network. In: Proceedings of Optical Fiber Communication Conference and Exhibition (OFC), San Francisco, 2021

  87. Jiang X T, Wang D S, Fan Q R, et al. Physics-informed neural network for nonlinear dynamics in fiber optics. Laser Photonics Rev, 2022. doi: https://doi.org/10.1002/lpor.202100483

  88. Fan Q R, Zhou G, Gui T, et al. Advancing theoretical understanding and practical performance of signal processing for nonlinear optical communications through machine learning. Nat Commun, 2020, 11: 3694

    Article  Google Scholar 

  89. Karandin O, Ayoub O, Musumeci F, et al. If not here, there. explaining machine learning models for fault localization in optical networks. In: Proceedings of International Conference on Optical Network Design and Modeling (ONDM), Warsaw, 2022. 1–3

Download references

Acknowledgements

This work was supported in part by National Key R&D Program of China (Grant No. 2019YFB1803502) and National Natural Science Foundation of China (Grant Nos. 61975020, 61871415, 62171053).

Author information

Authors and Affiliations

  1. State Key Laboratory of Information Photonics and Optical Communications, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Danshi Wang, Chunyu Zhang, Wenbin Chen, Hui Yang & Min Zhang

  2. Photonics Research Center, Department of Electrical Engineering, The Hong Kong Polytechnic University, Hong Kong, 999077, China

    Alan Pak Tao Lau

  3. The Hong Kong Polytechnic University Shenzhen Research Institute, Shenzhen, 518057, China

    Alan Pak Tao Lau

Authors
  1. Danshi Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chunyu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenbin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hui Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Min Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Alan Pak Tao Lau
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Min Zhang or Alan Pak Tao Lau.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, D., Zhang, C., Chen, W. et al. A review of machine learning-based failure management in optical networks. Sci. China Inf. Sci. 65, 211302 (2022). https://doi.org/10.1007/s11432-022-3557-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-022-3557-9

Keywords

Search

Navigation