Abstract
Effective methods of the detection of multiword expressions are important for many technologies related to Natural Language Processing. Most contemporary methods are based on the sequence labeling scheme, while traditional methods use statistical measures. In our approach, we want to integrate the concepts of those two approaches. In this paper, we present a novel weakly supervised multiword expressions extraction method which focuses on their behaviour in various contexts. Our method uses a lexicon of Polish multiword units as the reference knowledge base and leverages neural language modelling with deep learning architectures. In our approach, we do not need a corpus annotated specifically for the task. The only required components are: a lexicon of multiword units, a large corpus, and a general contextual embeddings model. Compared to the method based on non-contextual embeddings, we obtain gains of 15% points of the macro F1-score for both classes and 30% points of the F1-score for the incorrect multiword expressions. The proposed method can be quite easily applied to other languages.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abadi, M., Agarwal, A., Barham, P., et al.: TensorFlow: large-scale machine learning on heterogeneous systems (2015). https://www.tensorflow.org/
Agrawal, S., Sanyal, R., Sanyal, S.: Hybrid method for automatic extraction of multiword expressions. Int. J. Eng. Technol. 7, 33 (2018)
Anke, L.E., Schockaert, S., Wanner, L.: Collocation classification with unsupervised relation vectors. In: Proceedings of the 57th Annual Meeting of the ACL (2019)
Berk, G., Erden, B., Güngör, T.: Deep-BGT at PARSEME shared task 2018: bidirectional LSTM-CRF model for verbal multiword expression identification. In: Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018), pp. 248–253. ACL (2018)
Bojanowski, P., Grave, E., Joulin, A., et al.: Enriching word vectors with subword information. Trans. ACL 5, 135–146 (2017)
Boros, T., Burtica, R.: GBD-NER at PARSEME shared task 2018: Multi-word expression detection using bidirectional long-short-term memory networks and graph-based decoding. In: Proceedings of the Joint Work. on Linguistic Annotation, Multiword Expressions and Constructions (LAW-MWE-CxG-2018), pp. 254–260. ACL (2018)
Broda, B., Derwojedowa, M., Piasecki, M.: Recognition of structured collocations in an inflective language. Syst. Sci. 34(4), 27–36 (2008)
Buljan, M., Šnajder, J.: Combining linguistic features for the detection of Croatian multiword expressions. In: Proc. of the 13th Workshop on Multiword Expressions (MWE 2017), pp. 194–199. ACL (2017)
Chakraborty, S., Cougias, D., Piliero, S.: Identification of multiword expressions using transformers (2020)
Chawla, N.V., Bowyer, K.W., Hall, L.O., et al.: Smote: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)
Dziob, A., Piasecki, M., Rudnicka, E.K.: plwordnet 4.1 - a linguistically motivated, corpus-based bilingual resource. In: Proceedings of the Tenth Global Wordnet Conference: 23–27 July 2019, Wrocław (Poland), pp. 353–362 (2019)
Evert, S.: The Statistics of Word Cooccurrences: Word Pairs and Collocations. Ph.D. thesis, Institut für maschinelle Sprachverarbeitung, Univ. of Stuttgart (2004)
Fu, R., Guo, J., et al.: Learning semantic hierarchies via word embeddings. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pp. 1199–1209. Baltimore, Maryland (2014)
Green, S., de Marneffe, M.C., Manning, C.D.: Parsing models for identifying multiword expressions. Comput. Linguist. 39(1), 195–227 (2013)
Han, H., Wang, W.Y., Mao, B.H.: Borderline-smote: a new over-sampling method in imbalanced data sets learning. In: International Conference on Intelligent Computing, pp. 878–887 (2005)
Handler, A., Denny, M., Wallach, H., et al.: Bag of what? simple noun phrase extraction for text analysis. In: NLP+CSS@EMNLP (2016)
He, H., Bai, Y., Garcia, E.A., Li, S.: Adasyn: adaptive synthetic sampling approach for imbalanced learning. In: 2008 IEEE International Joint Conference on Neural Networks, pp. 1322–1328. IEEE (2008)
Hosseini, M.J., Smith, N.A., Lee, S.I.: UW-CSE at SemEval-2016 task 10: detecting multiword expressions and supersenses using double-chained conditional random fields. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp. 931–936. ACL (2016)
Klyueva, N., Doucet, A., Straka, M.: Neural networks for multi-word expression detection. In: Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pp. 60–65. ACL (2017)
Kocoń, J., Gawor, M.: Evaluating kgr10 polish word embeddings in the recognition of temporal expressions using bilstm-crf. Schedae Informaticae 27 (2018)
Kurfalı, M.: TRAVIS at PARSEME shared task 2020: how good is (m)BERT at seeing the unseen? In: Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons, pp. 136–141. ACL (2020)
Levy, O., Remus, S., et al.: Do supervised distributional methods really learn lexical inference relations? In: Proceedings of the 2015 Conference of the North American Chapter of ACL: Human Language Technologies, pp. 970–976 (2015)
Maziarz, M., Szpakowicz, S., Piasecki, M.: A procedural definition of multi-word lexical units. In: Mitkov, R., Angelova, G., Boncheva, K. (eds.) Proceedings of the International Conference Recent Advances in Natural Language Processing - RANLP’2015, pp. 427–435. INCOMA Ltd. (2015)
Mikolov, T., Chen, K., Corrado, G., et al.: Efficient estimation of word representations in vector space. In: Bengio, Y., LeCun, Y. (eds.) 1st International Conference on Learning Representations, ICLR 2013, pp. 2–4. Arizona, USA, May, Scottsdale (2013)
Mroczkowski, R., Rybak, P., Wróblewska, A., et al.: HerBERT: efficiently pretrained transformer-based language model for Polish. In: Proceedings of the 8th Workshop on Balto-Slavic Natural Language Processing, pp. 1–10. ACL (2021)
Pedregosa, F., Varoquaux, G., Gramfort, A., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Pečina, P.: Lexical association measures and collocation extraction. Lang. Res. Eval. 44, 137–158 (2010)
Piasecki, M., Wendelberger, M., Maziarz, M.: Extraction of the multi-word lexical units in the perspective of the wordnet expansion. In: Proceedings of the International Conference Recent Advances in Natural Language Processing (2015)
Ramisch, C.: Multiword Expressions Acquisition. TANLP, Springer, Cham (2015). https://doi.org/10.1007/978-3-319-09207-2
Ramisch, C., et al.: Edition 1.1 of the PARSEME shared task on automatic identification of verbal multiword expressions. In: Proceedings of the Joint Workshop on Linguistic Annotation, Multiword Expressions and Constructions (2018)
Ramisch, C., Savary, A., Guillaume, B., et al.: Edition 1.2 of the PARSEME shared task on semi-supervised identification of verbal multiword expressions. In: Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons (2020)
Ramshaw, L., Marcus, M.: Text chunking using transformation-based learning. In: Third Workshop on Very Large Corpora (1995)
Rohanian, O., Taslimipoor, S., Kouchaki, S., et al.: Bridging the gap: attending to discontinuity in identification of multiword expressions. In: Proceedings of the 2019 Conference of the North American Chapter of the ACL: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 2692–2698. ACL (2019)
Saied, H.A., Candito, M., Constant, M.: Comparing linear and neural models for competitive MWE identification. In: Proceedings of the 22nd Nordic Conference on Computational Linguistics, pp. 86–96. Linköping University Electronic Press (2019)
Scholivet, M., Ramisch, C.: Identification of ambiguous multiword expressions using sequence models and lexical resources. In: Proceedings of the 13th Workshop on Multiword Expressions (MWE 2017), pp. 167–175 (2017)
Seretan, V.: Syntax-Based Collocation Extraction, Text, Speech and Language Technology, vol. 44. Springer, Netherlands (2011). https://doi.org/10.1007/978-94-007-0134-2
Spasić, I., Owen, D., Knight, D., et al.: Unsupervised multi-word term recognition in Welsh. In: Proceedings of the Celtic Language Technology Workshop, pp. 1–6 (2019)
Taslimipoor, S., Bahaadini, S., Kochmar, E.: MTLB-STRUCT @Parseme 2020: capturing unseen multiword expressions using multi-task learning and pre-trained masked language models. In: Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons, pp. 142–148. ACL (2020)
Yirmibeşoğlu, Z., Güngör, T.: ERMI at PARSEME shared task 2020: embedding-rich multiword expression identification. In: Proceedings of the Joint Workshop on Multiword Expressions and Electronic Lexicons, pp. 130–135. ACL (2020)
Acknowledgements
This work was financed by the National Science Centre, Poland, project no. 2019/33/B/HS2/02814.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Piasecki, M., Kanclerz, K. (2022). Is Context All You Need? Non-contextual vs Contextual Multiword Expressions Detection. In: Groen, D., de Mulatier, C., Paszynski, M., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2022. ICCS 2022. Lecture Notes in Computer Science, vol 13350. Springer, Cham. https://doi.org/10.1007/978-3-031-08751-6_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-08751-6_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08750-9
Online ISBN: 978-3-031-08751-6
eBook Packages: Computer ScienceComputer Science (R0)