Abstract
This article examines and analyzes the use of the word2vec method for solving semantic coding problems. The task of semantic coding has acquired particular importance with the development of search system. The relevance of such technologies is associated primarily with the ability to search in large-volume databases. Based on the obtained practical results, a prototype of a search system based on the use of selected semantic information for the implementation of relevant search in the database of documents has been developed. Proposed two main scenarios for the implementation of such a search. The training set has been prepared on the basis of documents in the English version of Wikipedia, which includes more than 100,000 original articles. The resulting set was used in the experimental part of the work to test the effectiveness of the developed prototype search system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Mikolov T, Chen K, Corrado G, Dean J (2019) Efficient estimation of word representations in vector space. https://arxiv.org/abs/1301.3781. Accessed 14 Apr 2019
Mikolov T, Sutskever I, Chen K, Corrado GS, Dean J (2019) Distributed representations of words and phrases and their compositionality. https://arxiv.org/abs/1310.4546v1. Accessed 14 Apr 2019
Pennington J, Socher R, Manning C (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543
Lin J, Kolcz A (2012) Large-scale machine learning at twitter. In: Proceedings of the ACM SIGMOD international conference on management of data, Scottsdale, Arizona, USA, pp 793–804
Smola A, Narayanamurthy S (2010) An architecture for parallel topic models. In: Proceedings of the 36th international conference on very large data bases, Singapore, pp 703–710
Ng A et al (2006) Map-reduce for machine learning on multicore. In: Proceedings of advances in neural information processing systems, Vancouver, Canada, pp 281–288
Panda B, Herbach J, Basu S, Bayardo R (2012) MapReduce and its application to massively parallel learning of decision tree ensembles, in scaling up machine learning: parallel and distributed approaches. Cambridge University Press, Cambridge
Crego E, Munoz G, Islam F (2019) Big data and deep learning: big deals or big delusions? http://www.hufngtonpost.com/george-munoz-frank-islamand-ed-crego/big-data-and-deep-learnin_b_3325352.html. Accessed 14 Apr 2019
Bengio Y, Bengio S (2000) Modeling high-dimensional discrete data with multi-layer neural networks. In: Proceedings of advances in neural information processing systems, Vancouver, Canada, vol 12, pp 400–406
Ranzato MA, Boureau YL, LeCun Y (2007) Sparse feature learning for deep belief networks. In: Proceedings of advances in neural information processing systems, Vancouver, Canada, vol 20, pp 1185–1192
Hinton GE, Osindero ES, Teh Y (2006) A fast learning algorithm for deep belief nets. Neural Comput 18:1527–1554
Hinton G, Salakhutdinov R (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507
Hinton GE (2010) A practical guide to training restricted Boltzmann machines. Machine Learning Group, University of Toronto, Technical report 2010-000
LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444
Bengio Y, Lamblin P, Popovici D, Larochelle H (2007) Greedy layer-wise training of deep networks. In: Scholkopf B, Platt JC, Hoffman T (eds) Advances in neural information processing systems, vol 11. MIT Press, Cambridge, pp 153–160
Bengio Y (2009) Learning deep architectures for AI. Found Trends Mach Learn 2(1):1–127
Bengio Y et al (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35(8):1798–1828
Erhan D, Bengio Y, Courville A, Manzagol P-A, Vincent P, Bengio S (2010) Why does unsupervised pre-training help deep learning? J Mach Learn Res 11:625–660
Deng L, Yu D (2014) Deep learning: methods and applications. Found Trends Signal Process 7(3–4):197–387
Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61(1):85–117
Fischer A, Igel C (2014) Training restricted Boltzmann machines: an introduction. Pattern Recogn 47(1):25–39
Golovko V, Kroschanka A (2016) The nature of unsupervised learning in deep neural networks: a new understanding and novel approach. Opt Mem Neural Netw 3:127–141
Golovko V (2017) Deep learning: an overview and main paradigms. Opt Mem Neural Netw 1:1–17
Hinton G et al (2012) Deep neural network for acoustic modeling in speech recognition. IEEE Signal Process Mag 29:82–97
Golovko V, Egor M, Brich A, Sachenko A (2017) A shallow convolutional neural network for accurate handwritten digits classification. In: Krasnoproshin V, Ablameyko S (eds) Pattern recognition and information processing, communications in computer and information science, vol 673. Springer, Cham, pp 77–85
Krizhevsky A et al (2012) Imagenet classification with deep convolutional neural networks. Proc Adv Neural Inf Process Syst 25:1090–1098
Dahl GE, Yu D, Deng L, Acero A (2012) Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Trans Audio Speech Lang Process 20(1):30–41
Cirean D, Meler U, Cambardella L, Schmidhuber J (2010) Deep, big, simple neural nets for handwritten digit recognition. Neural Comput 22(12):3207–3220
Zeiler M, Taylor G, Fergus R (2011) Adaptive deconvolutional networks for mid and high level feature learning. Proc IEEE Int Conf Comput Vis 1(2):2018–2025
Dorosh V, Komar M, Sachenko A, Golovko V (2018) Parallel deep neural network for detecting computer attacks in information telecommunication systems. In: Proceedings of the 38th IEEE international conference on electronics and nanotechnology. TUU “Kyiv Polytechnic Institute”, Kyiv, pp 675–679
Komar M, Sachenko A, Golovko V, Dorosh V (2018) Compression of network traffic parameters for detecting cyber attacks based on deep learning. In: Proceedings of the 9th IEEE international conference on dependable systems, services and technologies, Kyiv, Ukraine, pp 44–48
Komar M, Dorosh V, Sachenko A, Hladiy G (2018) Deep neural network for detection of cyber attacks. In: Proceedings of the IEEE first international conference on system analysis and intelligent computing, Kyiv, Ukraine, pp 186–189
Komar M, Golovko V, Sachenko A, Dorosh V, Yakobchuk P (2018) Deep neural network for image recognition based on the caffe framework. In: Proceedings of the IEEE second international conference on data stream mining and processing, Lviv, Ukraine, pp 102–106
Golovko V, Bezobrazov S, Kroshchanka A, Sachenko A, Komar M, Karachka A (2017) Convolutional neural network based solar photovoltaic panel detection in satellite photos. In: Proceedings of the 9th IEEE international conference on intelligent data acquisition and advanced computing systems: technology and applications, Bucharest, Romania, pp 14–19
Golovko V, Kroshchanka A, Bezobrazov S, Komar M, Sachenko A, Novosad O (2018) Development of solar panels detector. In: Proceedings of the IEEE international scientific-practical conference “problems of infocommunications, science and technology”, Kharkiv, Ukraine, pp 761–764
Salakhutdinov R, Mnih A, Hinton G (2007) Restricted Boltzmann machines for collaborative filtering. In: Proceedings 24th international conference on machine learning, Corvalis, USA, pp 791–798
Efrati A (2019) How ‘deep learning’ works at Apple, beyond. https://www.theinformation.com/How-Deep-Learning-Works-at-Apple-Beyond. Accessed 14 Apr 2019
Jones N (2014) Computer science: the learning machines. Nature 505(7482):146–148
Wang Y, Yu D, Ju Y, Acero A (2011) Voice search. In: Tur G, De Mori R (eds) Language understanding: systems for extracting semantic information from speech, chap 5. Wiley, New York
Kirk J (2019) Universities, IBM join forces to build a brain-like computer. http://www.pcworld.com/article/2051501/universities-join-ibm-in-cognitive-computing-researchproject.html. Accessed 14 Apr 2019
Kroshchenko A, Golovko V, Bezobrazov S, Mikhno E, Rubanov V, Krivulets I (2017) The organization of semantic coding of words and search engine on the basis of neural networks. Vesnyk Brest State Tech Univ 5(107):9–12 (in Russian)
Van der Maaten L, Hinton GE (2008) Visualizing high-dimensional data using t-SNE. J Mach Learn Res 9:2579–2605
Pelevina M, Arefyev N, Biemann C, Panchenko A (2019) Making sense of word embeddings. https://arxiv.org/pdf/1708.03390.pdf. Accessed 14 Apr 2019
Xiong S, Wang X, Duan P, Yu Z, Dahou A (2017) Deep knowledge representation based on compositional semantics for Chinese geography. In: Proceedings of the 9th international conference on agents and artificial intelligence, pp 17–23
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Golovko, V., Kroshchanka, A., Komar, M., Sachenko, A. (2020). Neural Network Approach for Semantic Coding of Words. In: Lytvynenko, V., Babichev, S., Wójcik, W., Vynokurova, O., Vyshemyrskaya, S., Radetskaya, S. (eds) Lecture Notes in Computational Intelligence and Decision Making. ISDMCI 2019. Advances in Intelligent Systems and Computing, vol 1020. Springer, Cham. https://doi.org/10.1007/978-3-030-26474-1_45
Download citation
DOI: https://doi.org/10.1007/978-3-030-26474-1_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-26473-4
Online ISBN: 978-3-030-26474-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)