Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

A robust training of dendritic neuron model neural network for time series prediction

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Many prediction methods proposed in the literature can be concerned under two main headings: probabilistic and non-probabilistic methods. In particular, as a kind of non-probabilistic model, artificial neural networks (ANNs), having different properties, have been commonly and effectively used in the literature. Some ANNs operate the additive aggregation function in the structure of their neuron models, while others employ the multiplicative aggregation function. Recently proposed dendritic neural networks also have both additional and multiplicative neuron models. The prediction performance of such an artificial neural network will inevitably be negatively affected by the outliers that the time series of interest may contain due to the neuron model in its structure. This study, for the training of a dendritic neural network, presents a robust learning algorithm. The presented robust algorithm is the first for the training of DNM in the literature as far as is known and uses Huber's loss function as the fitness function. The iterative process of the robust learning algorithm is carried out by particle swarm optimization. The productivity and efficiency of the suggested learning algorithm were evaluated by analysing different real-life time series. All analyses were performed with original and contaminated data sets under different scenarios. The R-DNM has the best performance for the original data sets with a value of 2.95% in the ABC time series, while the FTSE showed the best performance in approximately 27% and the second best in 33% of all analyses. The proposed R-DNM has been the least affected by outliers in almost all scenarios for contaminated ABC data sets. Moreover, it has been the least affected model by outliers in approximately 71% of the 90 analyses performed for the contaminated FTSE time series. The obtained results show that the dendritic artificial neural network trained by the proposed robust learning algorithm produces the satisfactory predictive results in the analysis of time series with and without outliers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

FTSE data that support the findings of this study can be obtained from https://www.investing.com/indices/uk-100 or all data are available from the authors upon reasonable request.

References

  1. McCulloch WSPW (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5:115–133

    Article  MATH  MathSciNet  Google Scholar 

  2. Rumelhart DE, Hinton GEWR (1986) Learning representations by backpropagating errors. Nature 323(6188):533–536

    Article  MATH  Google Scholar 

  3. Hill M (1999) the learning behavior of single neuron classifiers on linearly separable. 1259–1264

  4. Basu M, Ho T-K (1999) The learning behavior of single neuron classifiers on linearly separable or nonseparable input. In: Proceedings of the 1999 international joint conference on neural networks

  5. Labib R (1999) New single neuron structure for solving nonlinear problems. In: Proceedings of the international joint conference on neural networks vol 1, p 617–620. https://doi.org/10.1109/ijcnn.1999.831569

  6. Plate T (2000) Randomly connected sigma-pi neurons can form associator networks. Netw Comput Neural Syst 11:321–332. https://doi.org/10.1088/0954-898x/11/4/305

    Article  MATH  Google Scholar 

  7. Zhang CN, Zhao M, Wang M (2000) Logic operations based on single neuron rational model. IEEE Trans Neural Netw 11:739–747. https://doi.org/10.1109/72.846745

    Article  Google Scholar 

  8. Yadav RN, Kumar N, Kalra PK, John J (2006) Learning with generalized-mean neuron model. Neurocomputing. https://doi.org/10.1016/j.neucom.2005.10.006

    Article  Google Scholar 

  9. Shiblee M, Chandra B, Kalra PK (2010) Learning of geometric mean neuron model using resilient propagation algorithm. Expert Syst Appl. https://doi.org/10.1016/j.eswa.2010.04.018

    Article  Google Scholar 

  10. Yadav RN, Kalra PK, John J (2007) Time series prediction with single multiplicative neuron model. Appl Soft Comput J. https://doi.org/10.1016/j.asoc.2006.01.003

    Article  Google Scholar 

  11. Legenstein R, Maass W (2011) Branch-specific plasticity enables self-organization of nonlinear computation in single neurons. J Neurosci 31:10787–10802. https://doi.org/10.1523/JNEUROSCI.5684-10.2011

    Article  Google Scholar 

  12. Ghosh J, Shin Y (1992) Efficient higher-order neural networks for classification and function approximation. Int J Neural Syst 3:323–350

    Article  Google Scholar 

  13. Gurney KN (1992) Training nets of hardware realizable sigma-pi units. Neural Netw 5(2):289–303

    Article  Google Scholar 

  14. Weber C, Wermter S (2007) A self-organizing map of sigma-pi units. Neurocomputing 70:2552–2560. https://doi.org/10.1016/j.neucom.2006.05.014

    Article  Google Scholar 

  15. Yilmaz A, Yolcu U (2022) Dendritic neuron model neural network trained by modified particle swarm optimization for time-series forecasting. J Forecast 41:793–809. https://doi.org/10.1002/for.2833

    Article  MathSciNet  Google Scholar 

  16. Koch C, Poggio T, Torre V (1983) Nonlinear interactions in a dendritic tree: localization, timing, and role in information processing. Proc Nat Acad Sci USA 80(9):2799–2802

    Article  Google Scholar 

  17. Attia, MA, Sallam, EA, Fahmy MM (2012) A proposed generalized mean single multiplicative neuron model. In: Proceedings—2012 IEEE 8th international conference on intelligent computer communication and processing, ICCP 2012, art no 6356163 73–78

  18. Gao S, Zhou M, Wang Y et al (2019) Dendritic neuron model with effective learning algorithms for classification, approximation, and prediction. IEEE Trans Neural Netw Learn Syst 30:601–614. https://doi.org/10.1109/TNNLS.2018.2846646

    Article  Google Scholar 

  19. Zhou T, Gao S, Wang J et al (2016) Financial time series prediction using a dendritic neuron model. Knowl Based Syst. https://doi.org/10.1016/j.knosys.2016.05.031

    Article  Google Scholar 

  20. Bas E, Uslu VR, Egrioglu E (2016) Robust learning algorithm for multiplicative neuron model artificial neural networks. Expert Syst Appl. https://doi.org/10.1016/j.eswa.2016.02.051

    Article  Google Scholar 

  21. Chen DS, Jain RC (1994) A robust back propagation learning algorithm for function approximation. IEEE Trans Neural Netw. https://doi.org/10.1109/72.286917

    Article  Google Scholar 

  22. Rusiecki A (2012) Robust learning algorithm based on iterative least median of squares. Neural Process Lett 36(2):145–160

    Article  Google Scholar 

  23. Ham FM, McDowall TM (1997) Robust learning in a partial least-squares neural network. Nonlinear Anal Theory Methods Appl 30:2903–2914. https://doi.org/10.1016/S0362-546X(96)00223-4

    Article  MATH  MathSciNet  Google Scholar 

  24. Neubauer A (1995) Robust learning algorithms for multi-layer perceptrons with discretized synaptic weights. In: IEEE international conference on neural networks - conference proceedings

  25. Yolcu U, Bas E, Egrioglu E, Aladag CH (2015) A new multilayer feedforward network based on trimmed mean neuron model. Neural Netw World 25:587–602. https://doi.org/10.14311/NNW.2015.25.029

    Article  Google Scholar 

  26. Egrioglu E, Yolcu U, Bas E, Dalar AZ (2019) Median-Pi artificial neural network for forecasting. Neural Comput Appl 31:307–316. https://doi.org/10.1007/s00521-017-3002-z

    Article  Google Scholar 

  27. Aladag CH, Egrioglu E, Yolcu U (2014) Robust multilayer neural network based on median neuron model. Neural Comput Appl. https://doi.org/10.1007/s00521-012-1315-5

    Article  MATH  Google Scholar 

  28. Harter HL (1993) The chronological annotated bibliography of order statistics. Eur J Oper Res. https://doi.org/10.1016/0377-2217(93)90102-s

    Article  MATH  Google Scholar 

  29. Lei H, Zhang BL, Huang Q (1998) Robust interval regression analysis using neural networks. Fuzzy Sets Syst 97:337–347. https://doi.org/10.1016/s0165-0114(96)00325-9

    Article  Google Scholar 

  30. Hsiao C (2012) A panel data approach for program evaluation: measuring the benefits of political and economic integration of Hong kong with mainland China. J Appl Econ 27(5):705–740

    Article  MathSciNet  Google Scholar 

  31. El-melegy MT, Essai MH, Ali AA (2009) Robust training of artificial feedforward neural networks. Stud Comput Intell 201:217–242. https://doi.org/10.1007/978-3-642-01082-8_9

    Article  Google Scholar 

  32. Thomas P, Bloch G, Sirou F, Eustache V (1999) Neural modeling of an induction furnac using robust learning criteria. Integr Comput Aided Eng. https://doi.org/10.3233/ica-1999-6103

    Article  Google Scholar 

  33. Connor JT, Martin RD, Atlas LE (1994) Recurrent Neural Networks and Robust Time Series Prediction. IEEE Trans Neural Netw doi. https://doi.org/10.1109/72.279188

    Article  Google Scholar 

  34. Sánchez AVD (1995) Robustization of a learning method for RBF networks. Neurocomputing 9:85–94. https://doi.org/10.1016/0925-2312(95)00000-V

    Article  Google Scholar 

  35. Wang Z, Huang B, Unbehauen H (1999) Robust reliable control for a class of uncertain nonlinear state-delayed systems. Automatica 35:955–963. https://doi.org/10.1016/S0005-1098(98)00233-7

    Article  MATH  MathSciNet  Google Scholar 

  36. Mili L, Coakley CW (1996) Robust estimation in structured linear regression. Ann Stat 24:2593–2607. https://doi.org/10.1214/aos/1032181171

    Article  MATH  MathSciNet  Google Scholar 

  37. Fan J, Chow TWS (2020) Exactly robust kernel principal component analysis. IEEE Trans Neural Netw Learn Syst 31:749–761. https://doi.org/10.1109/TNNLS.2019.2909686

    Article  MathSciNet  Google Scholar 

  38. Fan J, Chow TWS (1976) Component analysis. Acta Psychiatr Scand 53:298–302. https://doi.org/10.1111/j.1600-0447.1976.tb02339.x

    Article  Google Scholar 

  39. Shi P, Shue S-P (1999) Robust H/sub /spl infin// control for linear discrete-time systems with norm-bounded nonlinear uncertainties. IEEE Trans Automat Contr 44:108–111. https://doi.org/10.1109/9.739084

    Article  MATH  Google Scholar 

  40. Sinha S, Wiens DP (2002) Robust sequential designs for nonlinear regression. Can J Stat 30:601–618. https://doi.org/10.2307/3316099

    Article  MATH  MathSciNet  Google Scholar 

  41. Allende H, Moraga C, Salas R (2002) Robust estimator for the learning process in neural networks applied in time series. In: Dorronsoro JR (ed) Artificial neural networks—ICANN 2002. Springer, Berlin, pp 1080–1086

    Chapter  MATH  Google Scholar 

  42. Pernía-Espinoza AV, Ordieres-Meré JB, Martínez-De-Pisón FJ, González-Marcos A (2005) TAO-robust backpropagation learning algorithm. Neural Netw 18:191–204. https://doi.org/10.1016/j.neunet.2004.11.007

    Article  Google Scholar 

  43. Hans A, Udluft S (2010) Ensembles of neural networks for robust reinforcement learning. In: Proceedings—9th international conference on machine learning and applications, ICMLA 2010 401–406. https://doi.org/10.1109/ICMLA.2010.66

  44. Song C, Zhao H, Jing W, Zhu H (2012) Robust video stabilization based on particle filtering with weighted feature points. IEEE Trans Consum Electron 58:570–577. https://doi.org/10.1109/TCE.2012.6227462

    Article  Google Scholar 

  45. Liu B, Malon C, Xue L, Kruus E (2022) Improving neural network robustness through neighborhood preserving layers. Image Vis Comput. https://doi.org/10.1016/J.IMAVIS.2022.104469

    Article  Google Scholar 

  46. Nakamura T, Fukagata K (2022) Robust training approach of neural networks for fluid flow state estimations. Int J Heat Fluid Flow. https://doi.org/10.1016/J.IJHEATFLUIDFLOW.2022.108997

    Article  Google Scholar 

  47. Lei Y, Li J (2022) Robust adaptive neural networks control for a class of time-delay parabolic systems with nonlinear periodic time-varying parameter. J Franklin Inst 359:8014–8043. https://doi.org/10.1016/J.JFRANKLIN.2022.07.055

    Article  MATH  MathSciNet  Google Scholar 

  48. Shi X, Peng Y, Chen Q et al (2022) Robust convolutional neural networks against adversarial attacks on medical images. Pattern Recognit. https://doi.org/10.1016/J.PATCOG.2022.108923

    Article  Google Scholar 

  49. Yolcu U, Egrioglu E, Aladag CH (2013) A new linear & nonlinear artificial neural network model for time series forecasting. Decis Support Syst 54:1340–1347. https://doi.org/10.1016/j.dss.2012.12.006

    Article  Google Scholar 

  50. Egrioglu E, Yolcu U, Bas E, Dalar AZ (2019) Median-Pi artificial neural network for forecasting. Neural Comput Appl. https://doi.org/10.1007/s00521-017-3002-z

    Article  Google Scholar 

  51. Broomhead D (1988) Multivariable functional interpolation and adaptive networks. Complex Syst 2:321–355

    MATH  MathSciNet  Google Scholar 

Download references

Funding

This research received no specific grant from any funding agency.

Author information

Authors and Affiliations

  1. Department of Statistics, Ondokuz Mayıs Univesity, Samsun, Turkey

    Ayşe Yilmaz

  2. Department of Statistics, Marmara University, Istanbul, Turkey

    Ufuk Yolcu

Authors
  1. Ayşe Yilmaz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ufuk Yolcu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ufuk Yolcu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Consent for publication

All authors consented to the publication of the research.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 190 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yilmaz, A., Yolcu, U. A robust training of dendritic neuron model neural network for time series prediction. Neural Comput & Applic 35, 10387–10406 (2023). https://doi.org/10.1007/s00521-023-08240-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-023-08240-6

Keywords

Search

Navigation