Abstract
We consider an integrated approach to design the classification rule. Here qualities of statistical and neural net approaches are merged together. Instead of using the multivariate models and statistical methods directly to design the classifier, we use them in order to whiten the data and then to train the perceptron. A special attention is paid to magnitudes of the weights and to optimization of the training procedure. We study an influence of all characteristics of the cost function (target values, conventional regularization parameters), parameters of the optimization method (learning step, starting weights, a noise injection to original training vectors, to targets, and to the weights) on a result. Some of the discussed methods to control complexity are almost not discussed in the literature yet.
Chapter PDF
Similar content being viewed by others
Keywords
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.
References
Amari, S.: A theory of adaptive pattern classifiers. — IEEE Trans. Electron. Computers, EC-16 (1967) 623–625
An, G.: The effects of adding noise during backpropagation training on generalization performance, Neural Computation, 8 (1996) 643–674
Bishop, C..M.: Neural Networks for Pattern Recognition. Oxford Univ. Press (1995)
Devijver, P.A., Kittler, J. Pattern Recognition. A statistical approach, Precentice-Hall International, Inc., London (1982).
Duin, R.P.W.: Small sample size generalization. Proc. 9th Scandinavian Conference on Image Analysis, June 6–9, 1995, Uppsala, Sweden (1995)
Kanal L., Chandrasekaran, B.: On dimensionality and sample size in statistical pattern recognition. Pattern Recognition, 3 (1971) 238–255
Mao, J. Jain, A.: Regularization techniques in artificial neural networks. In: Proc. World Congress on Neural Networks, Portland, July 1993.
Raudys, S. On the problems of sample size in pattern recognition. In: Proc., 2nd All-Union Conf. Statist. Methods in Control Theory; Moscow, Nauka, 64–67 (1970)
Raudys S.: Evolution and generalization of a single neurone. I. SLP as seven statistical classifiers. Neural Networks, 11 (1998) 283–296.
Raudys S.: Statistical and Neural Classification Algorithms. An Integrated Approach. Springer, London (2001)
Raudys S, Skurikhina, M., Cibas, T., Gallinari, P.: Ridge estimates of the covariance matrix and regularization of artificial neural network classifier. Pattern Recognition and Image Processing, Int. J. of Russian Academy of Sciences, Moscow, 1995, N4 633–650
Reed R.: Pruning Algorithms-A Survey. IEEE Transactions on Neural Networks, 4, (1993) 740–747
Reed R., Marks II, R.J., Oh, S.: Similarities of error regularization, sigmoid gain scaling, target smoothing, and training with jitter, IEEE Transactions on Neural Networks, 6 (1995) 529–538.
Skurichina M., Raudys, S.. Duin, R.P.W.: K-nearest neighbors directed noise injection in multilayer perceptron training, IEEE Trans. on Neural Networks, 11 (2000) 504–511
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, Berlin (1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2000 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Raudys, Š. (2000). Classifier’s Complexity Control while Training Multilayer Perceptrons. In: Ferri, F.J., Iñesta, J.M., Amin, A., Pudil, P. (eds) Advances in Pattern Recognition. SSPR /SPR 2000. Lecture Notes in Computer Science, vol 1876. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-44522-6_4
Download citation
DOI: https://doi.org/10.1007/3-540-44522-6_4
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-67946-2
Online ISBN: 978-3-540-44522-7
eBook Packages: Springer Book Archive