Abstract
Pretrained embeddings have proven effective in legal problems in English. Even so, working well in one language does not guarantee that these models have an advantage in other languages. Understanding the characteristics of these models in a particular language helps us to make more accurate decisions when choosing technology for problems in that language. This paper provides an analytical perspective on pretrained embeddings in the legal field in Japanese. These models are measured on quantitative numbers as well as visualized in terms of their ability to represent Japanese legal terms. With such contributions, this paper may be useful to researchers and engineers who are building Japanese legal embeddings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
japaneselawtranslation.go.jp.
- 2.
- 3.
- 4.
References
Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, pp. 1532–1543. Association for Computational Linguistics (2014). https://doi.org/10.3115/v1/D14-1162
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)
Devlin, J., Chang, M.-W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I.: Language models are unsupervised multitask learners (2019)
Brown, T.B., et al.: Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)
Lewis, M., et al.: Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019)
Rabelo, J., Kim, M.-Y., Goebel, R., Yoshioka, M., Kano, Y., Satoh, K.: COLIEE 2020: Methods for Legal Document Retrieval and Entailment (2020)
Hannes, W., Jaromir, S., Karim, B.: Paragraph similarity scoring and fine-tuned bert for legal information retrieval and entailment. In: COLIEE 2020 (2020)
Nguyen, H.-T., et al.: Jnlp team: deep learning for legal processing in COLIEE 2020. arXiv preprint arXiv:2011.08071 (2020)
Hsuan-Lei, S., Yi-Chia, C., Sieh-Chuen, H.: Bert-based ensemble model for the statute law retrieval and legal information entailment. In: COLIEE 2020 (2020)
Nguyen, H.-T., et al.: Paralaw nets-cross-lingual sentence-level pretraining for legal text processing. arXiv preprint arXiv:2106.13403 (2021)
Nguyen, H.-T., et al.: Transformer-based approaches for legal text processing. Rev. Socionetwork Strat. 16, 1–21 (2022). https://doi.org/10.1007/s12626-022-00102-2
Thanh, N.H., Binh, D.T., Quan, B.M., Le Minh, N.: Evaluate and visualize legal embeddings for explanation purpose. In: 2021 13th International Conference on Knowledge and Systems Engineering (KSE), pp. 1–6. IEEE (2021)
Mikolov, T., Grave, E., Bojanowski, P., Puhrsch, C., Joulin, A.: Advances in pre-training distributed word representations. In: Proceedings of the International Conference on Language Resources and Evaluation (LREC 2018) (2018)
Chalkidis, I., Kampas, D.: Deep learning in law: early adaptation and legal word embeddings trained on large corpora. Artif. Intell. Law 27(2), 171–198 (2018). https://doi.org/10.1007/s10506-018-9238-9
Chalkidis, I., Fergadiotis, M., Malakasiotis, P., Aletras, N., Androutsopoulos, I.: Legal-bert: the muppets straight out of law school. arXiv preprint arXiv:2010.02559 (2020)
Conneau, A., et al.: Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116 (2019)
Ackowledgement
This work was supported by JSPS Kakenhi Grant Number 20H04295, 20K20406, and 20K20625.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Nguyen, HT., Nguyen, LM., Satoh, K. (2022). A Survey of Pretrained Embeddings for Japanese Legal Representation. In: Fujita, H., Fournier-Viger, P., Ali, M., Wang, Y. (eds) Advances and Trends in Artificial Intelligence. Theory and Practices in Artificial Intelligence. IEA/AIE 2022. Lecture Notes in Computer Science(), vol 13343. Springer, Cham. https://doi.org/10.1007/978-3-031-08530-7_30
Download citation
DOI: https://doi.org/10.1007/978-3-031-08530-7_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08529-1
Online ISBN: 978-3-031-08530-7
eBook Packages: Computer ScienceComputer Science (R0)