Abstract
The aim of this paper is to provide a systematic route of information retrieval from a knowledge-based database (or domain knowledge) through a dialog system of natural language interaction. The application is about a comprehensive building at a university, with classrooms, laboratory rooms, meeting rooms, research rooms and offices, and is to present related information the user asks for. First, the domain knowledge is expressed with predicate expressions based on the ontology structure; then the vocabulary is presented distributedly with word embedding enhanced with the domain knowledge; queries from the user are then converted into the intent (general) and slot elements (specific) with the help of trained recurrent neural network (RNN). The system works smoothly. The key point is integrating the two methods of knowledge-based and data-driven natural language processing into one system, and the domain knowledge is in the central part which is incorporated into the word embedding to make it specifically fit the natural language in this application.
Article PDF
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
References
J. Eisenstein, Natural Language Processing, MIT Press, Georgia Tech, USA, 2018, pp. 20–22.
T. Young, et al., Recent trends in deep learning based natural language processing, IEEE Comput. Intell. Mag. 13 (2018), 55–75.
S. Lai, et al., How to generate a good word embedding, IEEE Intell. Syst. 31 (2016), 5–14.
Z.S. Harris, Distributional structure, Word. 10 (1954), 146–162.
Q. Liu, et al., Learning semantic word embeddings based on ordinal knowledge constraints, in Proceedings of the 53th Annual Meeting of the Association for Computational Linguistics, Beijing, China, 2015.
M.C. Yang, et al., Knowledge-based question answering using the semantic embedding space, Expert Syst. Appl. 42 (2015), 9086–9104.
J. Bian, B. Gao, T. Liu, Knowledge-powered deep learning for word embedding, in: T. Calders, F. Esposito, E. Hüllermeier, R. Meo (Eds.), Machine Learning and Knowledge Discovery in Databases, Springer, Berlin, Germany, 2014, pp. 132–148.
B. Hu, et al., A novel word embedding learning model using the dissociation between nouns and verbs, Neurocomputing. 171 (2016), 1108–1117.
M. Liu, et al., Measuring similarity of academic articles with semantic profile and joint word embedding, Tsinghua Sci. Technol. 22 (2017), 619–632.
M. Faruqui, et al., Retrofitting word vectors to semantic lexicons, in Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Denver, CO, USA, 2015.
M. Faruqui, et al., Sparse overcomplete word vector representations, in Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China, 2015.
A. Bordes, et al., Learning structured embeddings of knowledge bases, in Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence, 2011.
Q. Wang, et al., Knowledge graph embedding: a survey of approaches and applications, IEEE Trans. Knowl. Data Eng. 29 (2017), 2724–2743.
Z. Wang, et al., Knowledge graph and text jointly embedding, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, 2014, pp. 1591–1601.
K. Toutanova, et al., Representing text for joint embedding of text and knowledge bases, in Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Portugal, 2015, pp. 1499–1509.
Z. Zhang, et al., ERNIE: enhanced language representation with informative entities, in Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 2019.
G. Tur, R. De Mori, Spoken Language Understanding: Systems for Extracting Semantic Information from Speech, John Wiley & Sons, Hoboken, NJ, USA, 2011.
M. McTear, Z. Callejas, D. Griol, Spoken language understanding, in: M. McTear, Z. Callejas, D. Griol (Eds.), The Conversational Interface, Springer, Berlin, Germany, 2016, pp. 161–185.
D. Gunning, S. Mark, C. Jaesik, et al., Explainable Artificial Intelligence (XAI), Science Robotics, 4 (2019), y7120.
M.S. Yakoub, S. Selouani, R. Nkambou, Mobile spoken dialogue system using parser dependencies and ontology, Int. J. Speech Technol. 18 (2015), 449–457.
D. Chen, C.D. Manning, A fast and accurate dependency parser using neural networks, in Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Qatar, 2014, pp. 740–750.
R. Cai, et al., An encoder-decoder framework translating natural language to database queries, in Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, 2017.
T. Rocktäschel, et al., Low-dimensional embeddings of logic, in Proceedings of the ACL 2014 Workshop on Semantic Parsing, Baltimore, MD, USA, 2014, pp. 45–49.
B. Chandrasekaran, J.R. Josephson, V.R. Benjamins, What are ontologies, and why do we need them? IEEE Intell. Syst. Appl. 14 (1999), 20–26.
H. Wang, J. Ren, A semantic map for indoor robot navigation based on predicate logic, Int. J. Knowl. Syst. Sci. 11 (2020), 1–21.
G. Sriharee, Indoor navigation using semantic symbolic information, in Proceedings of the 2013 IEEE 16th International Conference on Computational Science and Engineering, Sydney, NSW, Australia, 2013, pp. 1167–1173.
K. Lee, J. Lee, M.P. Kwan, Location-based service using ontology-based semantic queries: a study with a focus on indoor activities in a university context, Comput. Environ. Urban Syst. 62 (2017), 41–52.
A. Ameen, K.U.R. Khan, B.P. Rani, Construction of university ontology, in 2012 World Congress on Information and Communication Technologies, Trivandrum, India, 2013, pp. 39–44.
J. Scholz, S. Schabus, An indoor navigation ontology for production assets in a production environment, in International Conference on Geographic Information Science, Vienna, Austria, 2014, pp. 204–220.
W. Ertel, Logic programming with PROLOG, in: W. Ertel (Ed.), Introduction to Artificial Intelligence, Springer, Berlin, Germany, 2017, pp. 75–90.
T. Mikolov, et al., Distributed representations of words and phrases and their compositionality, in Proceedings of the 26th International Conference on Neural Information Processing Systems, 2013, pp. 3111–3119.
T. Mikolov, W.T. Yih, G. Zweig, Linguistic regularities in continuous space word representations, in Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Atlanta, GA, USA, 2013.
T. Mikolov, et al., Efficient estimation of word representations in vector space, in Proceedings of the International Conference on Learning Representations, 2013.
L.V.D. Maaten, G. Hinton, Visualizing data using t-SNE, J. Mach. Learn. Res. 9 (2008), 2579–2605.
B. Liu, I. Lane, Attention-based recurrent neural network models for joint intent detection and slot filling, in Proceeding of Interspeech 2016, 2016.
D. Bahdanau, K. Cho, Y. Bengio, Neural machine translation by jointly learning to align and translate, arXiv preprint arXiv:1409.0473, 2014.
F.A. Gers, J.U.R. Schmidhuber, F. Cummins, Learning to forget: continual prediction with LSTM, in Ninth International Conference on Artificial Neural Networks (ICANN), Edinburgh, UK, 1999.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
This is an open access article distributed under the CC BY-NC 4.0 license (https://doi.org/creativecommons.org/licenses/by-nc/4.0/).
About this article
Cite this article
Ren, J., Wang, H. & Liu, T. Information Retrieval Based on Knowledge-Enhanced Word Embedding Through Dialog: A Case Study. Int J Comput Intell Syst 13, 275–290 (2020). https://doi.org/10.2991/ijcis.d.200310.002
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.2991/ijcis.d.200310.002