Abstract
Natural Language Processing deals with the understanding and generation of texts through computer programs. There are many different functionalities used in this area, but among them there are some functions that are the support of the remaining ones. These methods are related to the core processing of the morphology of the language (such as lemmatization) and automatic identification of the part-of-speech tag. Thereby, this paper describes the implementation of a basic NLP toolkit for a new language, focusing in the features mentioned before, and testing them in an own corpus built for the occasion. The obtained results exceeded the expected results and could be used for more complex tasks such as machine translation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Available in: chana.inf.pucp.edu.pe/chanot.
- 2.
Available in: chana.inf.pucp.edu.pe/resources.
References
Acosta, S., Natalia, K., Huamancayo Curi, E., Mori Clement, M., Carbajal Solis, V.: Documento nacional de lenguas originarias del Perú (2013)
Daelemans, W., Groenewald, H.J., Van Huyssteen, G.B.: Prototype-Based Active Learning for Lemmatization (2009)
Ekbal, A., Bandyopadhyay, S.: Part of speech tagging in Bengali using support vector machine. In: International Conference on Information Technology 2008, ICIT 2008, pp. 106–111. IEEE (2008)
Gupta, V., Joshi, N., Mathur, I.: Design and development of a rule-based Urdu lemmatizer. In: Proceedings of International Conference on ICT for Sustainable Development, pp. 161–169. Springer (2016)
Jurafsky, D., Martin, J.H.: Speech and Language Processing, vol. 3. Pearson, London (2014)
Juršic, M., Mozetic, I., Erjavec, T., Lavrac, N.: Lemmagen: multilingual lemmatisation with induced ripple-down rules. J. Univers. Comput. Sci. 16(9), 1190–1214 (2010)
Khaltar, B.O., Fujii, A.: A lemmatization method for mongolian and its application to indexing for information retrieval. Inf. Process. Manag. 45(4), 438–451 (2009)
Ministerio de Educación del Perú: Minedu oficializa alfabetos de 24 lenguas originarias a ser utilizados por todas las entidades públicas. http://www.minedu.gob.pe/n/noticia.php?id=33082. Accessed 31 Mar 2016
Nivre, J., de Marneffe, M.C., Ginter, F., Goldberg, Y., Hajic, J., Manning, C.D., McDonald, R., Petrov, S., Pyysalo, S., Silveira, N., et al.: Universal dependencies v1: A multilingual treebank collection. In: Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016), pp. 1659–1666 (2016)
Paul, A., Purkayastha, B.S., Sarkar, S.: Hidden Markov model based part of speech tagging for Nepali language. In: 2015 International Symposium on Advanced Computing and Communication (ISACC), pp. 149–156. IEEE (2015)
Singh, S., Jha, G.N.: Statistical tagger for Bhojpuri (employing support vector machine). In: 2015 International Conference on Advances in Computing, Communications and Informatics (ICACCI), pp. 1524–1529. IEEE (2015)
Valenzuela, P.: Transitivity in Shipibo-Konibo grammar. Ph.D. thesis, University of Oregon (2003)
Acknowledgments
For this study, the authors appreciate the linguistic team effort that made possible the corpus annotation, and also acknowledge the support of the “Consejo Nacional de Ciencia, Tecnología e Innovación Tecnológica” (CONCYTEC Perú) under the contract 225-2015-FONDECYT.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Pereira-Noriega, J., Mercado-Gonzales, R., Melgar, A., Sobrevilla-Cabezudo, M., Oncevay-Marcos, A. (2017). Ship-LemmaTagger: Building an NLP Toolkit for a Peruvian Native Language. In: Ekštein, K., Matoušek, V. (eds) Text, Speech, and Dialogue. TSD 2017. Lecture Notes in Computer Science(), vol 10415. Springer, Cham. https://doi.org/10.1007/978-3-319-64206-2_53
Download citation
DOI: https://doi.org/10.1007/978-3-319-64206-2_53
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-64205-5
Online ISBN: 978-3-319-64206-2
eBook Packages: Computer ScienceComputer Science (R0)