research-article

A Weight Initialization Method Associated with Samples for Deep Feedforward Neural Network

Authors:

Yanli Yang,

Yichuan HeAuthors Info & Claims

ICCDE '20: Proceedings of 2020 6th International Conference on Computing and Data Engineering

Pages 54 - 59

https://doi.org/10.1145/3379247.3379253

Published: 07 March 2020 Publication History

Get Access

Abstract

Artificial neural network is an important force to promote the development of artificial intelligence, but it usually needs to be trained before use. The initial weight given randomly is the most widely used method when training neural networks. However, randomly given initial weights are independent of the samples. A weight initialization method associated with samples for deep feedforward neural network (DFFNN) is proposed. The initial weights set by this method are a combination of the original weights given randomly and the weights obtained by the first epoch training. The initial weights not only have random characteristics, but are also closely related to the samples to be trained. The proposed method is tested with the bearing data provided by the Case Western Reserve University (CWRU) Bearing Data Center. The testing results show that the proposed method can accelerate the training of DFFNN to some extent.

References

[1]

Hinton G. E. and Salakhutdinov R. R. 2006. Reducing the dimensionality of data with neural networks, Science 313, 504--507.

Crossref

Google Scholar

[2]

Wang L. N., Yang Y., Min R. Q., and Chakradhar S. 2017. Accelerating deep neural network training with inconsistent stochastic gradient descent, Neural Networks 93, 219--229.

Crossref

Google Scholar

[3]

Nur A. S., Radzi N. H. M., and Ibrahim A. O. 2014. Artificial neural network weight optimization: A review, TELKOMNIKA Indonesian Journal of Electrical Engineering 12, 6897--6902.

Crossref

Google Scholar

[4]

Combes R. T. D., Pezeshki M., Shabanian S., Courville A., and Bengio Yoshua, 2018. On the learning dynamics of deep neural networks, arXiv: 1809.06848v1.

Google Scholar

[5]

Ramos E. Z., Nakakuni M., and Yfantis E. 2017. Quantitative measures to evaluate neural network weight initialization strategies, in IEEE 7th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, USA, 2017.

Google Scholar

[6]

Kumar S. K., On weight initialization in deep neural networks, arXiv:1704.08863v2, 2017.

Google Scholar

[7]

Daniely A., Frostig R., and Singer Y. 2016. Toward deeper understanding of neural networks: The power of initialization and a dual view on expressivity, in 30th Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain, 2016, pp. 1--9.

Google Scholar

[8]

Lee A., Geem Z. W., and Suh K. D., 2016. Determination of optimal initial weights of an artificial neural network by using the harmony search algorithm: Application to breakwater armor stones, Applied Sciences 6, 1--17.

Crossref

Google Scholar

[9]

Schmidhuber J. 2015. Deep learning in neural networks: An overview, Neural Networks 61, 85--117.

Digital Library

Google Scholar

[10]

Case Western Reserve University Bearing Data Center Website. <http://csegroups.case.edu/bearingdatacenter/home>.

Google Scholar

[11]

DeepLearnToolbox website. <https://github.com/rasmusbergpalm/DeepLearnToolbox>.

Google Scholar

[12]

Yang Y. L., Fu P. Y., and He Y. C. 2018. Bearing faults automatic classification based on deep learning, IEEE Access 6, 71540--71554.

Crossref

Google Scholar

[13]

Yang Y. L. and Fu P. Y. 2018. Rolling-element bearing fault data automatic clustering based on wavelet and deep neural network, Shock and Vibration 2018, Article ID 3047830, 11 pages.

Google Scholar

Cited By

View all

Yang YLi C(2021)Quantitative analysis of the generalization ability of deep feedforward neural networksJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-20167940:3(4867-4876)Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.3233/JIFS-201679
Priya VNatesan PVenu KGothai E(2021)Improving convergence speed of the neural network model using Meta heuristic algorithms for weight initialization2021 International Conference on Computer Communication and Informatics (ICCCI)10.1109/ICCCI50826.2021.9402415(1-6)Online publication date: 27-Jan-2021
https://doi.org/10.1109/ICCCI50826.2021.9402415
Biba MVajjhala NRakshit S(2021)A State-of-the-Art Survey on Deep Learning Methods and Applications in BioinformaticsProceedings of International Conference on Advanced Computing Applications10.1007/978-981-16-5207-3_62(765-771)Online publication date: 24-Nov-2021
https://doi.org/10.1007/978-981-16-5207-3_62

Index Terms

A Weight Initialization Method Associated with Samples for Deep Feedforward Neural Network
1. General and reference
  1. Cross-computing tools and techniques
    1. Empirical studies
  2. Document types
    1. General conference proceedings

Recommendations

A review on weight initialization strategies for neural networks
Abstract
Over the past few years, neural networks have exhibited remarkable results for various applications in machine learning and computer vision. Weight initialization is a significant step employed before training any neural network. The weights of a ...
Improved weight initialization for deep and narrow feedforward neural network
Abstract
Appropriate weight initialization settings, along with the ReLU activation function, have become cornerstones of modern deep learning, enabling the training and deployment of highly effective and efficient neural network models across diverse ...
Mutual information based weight initialization method for sigmoidal feedforward neural networks

When a sigmoidal feedforward neural network (SFNN) is trained by the gradient-based algorithms, the quality of the overall learning process strongly depends on the initial weights. To improve the algorithm stability and avoid local minima, a Mutual ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

ICCDE '20: Proceedings of 2020 6th International Conference on Computing and Data Engineering

January 2020

279 pages

ISBN:9781450376730

DOI:10.1145/3379247

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 March 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICCDE 2020

ICCDE 2020: 2020 The 6th International Conference on Computing and Data Engineering

January 4 - 6, 2020

Sanya, China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
115
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Yang YLi C(2021)Quantitative analysis of the generalization ability of deep feedforward neural networksJournal of Intelligent & Fuzzy Systems: Applications in Engineering and Technology10.3233/JIFS-20167940:3(4867-4876)Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.3233/JIFS-201679
Priya VNatesan PVenu KGothai E(2021)Improving convergence speed of the neural network model using Meta heuristic algorithms for weight initialization2021 International Conference on Computer Communication and Informatics (ICCCI)10.1109/ICCCI50826.2021.9402415(1-6)Online publication date: 27-Jan-2021
https://doi.org/10.1109/ICCCI50826.2021.9402415
Biba MVajjhala NRakshit S(2021)A State-of-the-Art Survey on Deep Learning Methods and Applications in BioinformaticsProceedings of International Conference on Advanced Computing Applications10.1007/978-981-16-5207-3_62(765-771)Online publication date: 24-Nov-2021
https://doi.org/10.1007/978-981-16-5207-3_62

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

A review on weight initialization strategies for neural networks

Improved weight initialization for deep and narrow feedforward neural network

Mutual information based weight initialization method for sigmoidal feedforward neural networks