research-article

Natural language processing in law: : Prediction of outcomes in the higher courts of Turkey

Authors:

Emre Mumcuoğlu,

Ceyhun E. Öztürk,

Haldun M. Ozaktas,

Aykut KoçAuthors Info & Claims

Volume 58, Issue 5

https://doi.org/10.1016/j.ipm.2021.102684

Published: 01 September 2021 Publication History

Abstract

Natural language processing (NLP) based approaches have recently received attention for legal systems of several countries. It is of interest to study the wide variety of legal systems that have so far not received any attention. In particular, for the legal system of the Republic of Turkey, codified in Turkish, no works have been published. We first review the state-of-the-art of NLP in law, and then study the problem of predicting verdicts for several different courts, using several different algorithms. This study is much broader than earlier studies in the number of different courts and the variety of algorithms it includes. Therefore it provides a reference point and baseline for further studies in this area. We further hope the scope and systematic nature of this study can set a framework that can be applied to the study of other legal systems. We present novel results on predicting the rulings of the Turkish Constitutional Court and Courts of Appeal, using only fact descriptions, and without seeing the actual rulings. The methods that are utilized are based on Decision Trees (DTs), Random Forests (RFs), Support Vector Machines (SVMs) and state-of-the-art deep learning (DL) methods; specifically Gated Recurrent Units (GRUs), Long Short-Term Memory networks (LSTMs) and bidirectional LSTMs (BiLSTMs), with the integration of an attention mechanism for each model. The prediction results for all algorithms are given in a comparative and detailed manner. We demonstrate that outcomes of the courts of Turkish legal system can be predicted with high accuracy, especially with deep learning based methods. The presented results exhibit similar performance to earlier work in the literature for other languages and legal systems.

References

[1]

Akın A.A., Akın M.D., Zemberek, an open source NLP framework for Turkic Languages, 2013, GitHub Repository https://github.com/ahmetaa/zemberek-nlp.

[2]

Akkaya E.K., Can B., Transfer learning for Turkish named entity recognition on noisy text, Natural Language Engineering 27 (1) (2020) 35–64,.

[3]

Aletras N., Tsarapatsanis D., Preotiuc-Pietro D., Lampos V., Predicting judicial decisions of the European Court of Human Rights: A natural language processing perspective, PeerJ Computer Science 2 (2016),.

[4]

Aleven V., Using background knowledge in case-based legal reasoning: A computational model and an intelligent learning environment, Artificial Intelligence 150 (2003) 183–237,.

Digital Library

[5]

Ansay T., Wallace D., Introduction to Turkish law, Kluwer Law International, 2005,.

[6]

de Araujo P.H.L., de Campos T.E., de Oliveira R.R.R., Stauffer M., Couto S., Bermejo P., LeNER-Br: A dataset for named entity recognition in Brazilian legal text, in: International conference on the computational processing of Portuguese, Springer, 2018, pp. 313–323,.

Digital Library

[7]

Ashley K.D., Modelling legal argument: Reasoning with cases and hypotheticals, (Ph.D. thesis) University of Massachusetts, USA, 1988, Order No: GAX88-13198.

[8]

Ashley K.D., Toward a computational theory of arguing with precedents, in: Proceedings of the 2nd international conference on artificial intelligence and law, Association for Computing Machinery, New York, NY, USA, 1989, pp. 93–102,.

Digital Library

[9]

Ashley K.D., Reasoning with cases and hypotheticals in HYPO, International Journal of Man-Machine Studies 34 (6) (1991) 753–796.

[10]

Ashley K.D., Case-based reasoning and its implications for legal expert systems, Artificial Intelligence and Law 1 (1992) 113–208,.

Digital Library

[11]

Ashley K.D., Brüninghaus S., Automatically classifying case texts and predicting outcomes, Artificial Intelligence and Law 17 (2) (2009) 125–165,.

Digital Library

[12]

Ashley K.D., Rissland E.L., A case-based approach to modeling legal expertise, IEEE Expert: Intelligent Systems and their Applications 3 (3) (1988) 70–77,.

Digital Library

[13]

Bahdanau, D., Cho, K., & Bengio, Y. (2015). Neural machine translation by jointly learning to align and translate. In Y. Bengio & Y. LeCun (Eds.), 3rd international conference on learning representations.

[14]

Bench-Capon T., Araszkiewicz A.M., Ashley A.K., Atkinson K., Bex F., Borges F., Bourcier D., Bourgine P., Conrad J.G., Francesconi E., Gordon T.F., Governatori G., Leidner J.L., Lewis D.D., Loui R.P., McCarty L.T., Prakken H., Schilder F., Schweighofer E., …., Wyner A.Z., A history of AI and Law in 50 papers: 25 years of the international conference on AI and Law, Artificial Intelligence and Law 20 (2012) 215–319,.

Digital Library

[15]

Branting K.L., Yeh A., Weiss B., Merkhofer E., Brown B., Inducing predictive models for decision support in administrative adjudication, in: Pagallo U., Palmirani M., Casanovas P., Sartor G., Villata S. (Eds.), AI approaches to the complexity of legal systems, Springer International Publishing, 2018, pp. 465–477,.

[16]

Breiman L., Random forests, Machine Learning 45 (2001) 5–32,.

Digital Library

[17]

Buchanan B.G., Headrick T.E., Some speculation about artificial intelligence and legal reasoning, Stanford Law Review 23 (1970) 40–62,.

[18]

Cardellino C., Teruel M., Alemany L.A., Villata S., A low-cost, high-coverage legal named entity recognizer, classifier and linker, in: Proceedings of the 16th edition of the international conference on articial intelligence and law, Association for Computing Machinery, New York, NY, USA, 2017, pp. 9–18,.

Digital Library

[19]

Chalkidis I., Androutsopoulos I., A deep learning approach to contract element extraction, in: Wyner A.Z., Casini G. (Eds.), Legal knowledge and information systems, in: Frontiers in artificial intelligence and applications, vol. 302, IOS Press, 2017, pp. 155–164,.

[20]

Chalkidis I., Androutsopoulos I., Michos A., Extracting contract elements, in: Proceedings of the 16th edition of the international conference on articial intelligence and law, Association for Computing Machinery, New York, NY, USA, 2017, pp. 19–28,.

Digital Library

[21]

Chalkidis I., Androutsopoulos I., Michos A., Obligation and prohibition extraction using hierarchical RNNs, in: Proceedings of the 56th annual meeting of the association for computational linguistics (volume 2: Short papers), Association for Computational Linguistics, Melbourne, Australia, 2018, pp. 254–259,.

[22]

Chalkidis I., Kampas D., Deep learning in law: Early adaptation and legal word embeddings trained on large corpora, Artificial Intelligence and Law 27 (2018) 1–28,.

Digital Library

[23]

Cho K., van Merriënboer B., Gulcehre C., Bahdanau D., Bougares F., Schwenk H., Bengio Y., Learning phrase representations using RNN encoder–decoder for statistical machine translation, in: Proceedings of the 2014 conference on empirical methods in natural language processing, Association for Computational Linguistics, Doha, Qatar, 2014, pp. 1724–1734,.

[24]

Cortes C., Vapnik V.N., Support vector networks, Machine Learning 20 (1995) 273–297,.

[25]

Do P., Nguyen H., Tran C., Nguyen M., Nguyen M., Legal question answering using ranking SVM and deep convolutional neural network, 2017,. arXiv preprint arXiv:1703.05320 [Cs.IT].

[26]

Dozier C., Kondadadi R., Light M., Vachher A., Veeramachaneni S., Wudali R., Named entity recognition and resolution in legal text, in: Semantic processing of legal texts: Where the language of law meets the law of language, Springer-Verlag, Berlin, Heidelberg, 2010, pp. 27–43,.

[27]

Elnagar A., Al-Debsi R., Einea O., Arabic text classification using deep learning models, Information Processing & Management 57 (1) (2020),.

Digital Library

[28]

Elnaggar A., Otto R., Matthes F., Deep learning for named-entity linking with transfer learning for legal documents, in: Proceedings of the 2018 artificial intelligence and cloud computing conference, Association for Computing Machinery, New York, NY, USA, 2018, pp. 23–28,.

Digital Library

[29]

Follett L., Geletta S., Laugerman M., Quantifying risk associated with clinical trial termination: A text mining approach, Information Processing & Management 56 (3) (2019) 516–525,.

Digital Library

[30]

Goodfellow I., Bengio Y., Courville A., Deep learning, MIT Press, 2016.

Digital Library

[31]

Güneş A., Tantuğ A.C., Turkish named entity recognition with deep learning, in: 2018 26th IEEE signal processing and communications applications conference, 2018, pp. 1–4,.

[32]

Güngör O., Güngör T., Üsküdarlı S., The effect of morphology in named entity recognition with sequence tagging, Natural Language Engineering 25 (2019) 147–169,.

[33]

Hafner C.D., Berman D.H., The role of context in case-based legal reasoning: Teleological, temporal, and procedural, Artificial Intelligence and Law 10 (1–3) (2002) 19–64,.

Digital Library

[34]

Haneczok J., Piskorski J., Shallow and deep learning for event relatedness classification, Information Processing & Management 57 (6) (2020),.

[35]

Hochreiter S., Schmidhuber J., Long short-term memory, Neural Computation 9 (8) (1997) 1735–1780,.

Digital Library

[36]

Ikram A.Y., Chakir L., Arabic text classification in the legal domain, in: 2019 3rd international conference on intelligent computing in data sciences, 2019, pp. 1–6,.

[37]

Ji D., Gao J., Fei H., Teng C., Ren Y., A deep neural network model for speakers coreference resolution in legal texts, Information Processing & Management 57 (6) (2020),.

[38]

Ji D., Tao P., Fei H., Ren Y., An end-to-end joint model for evidence information extraction from court record document, Information Processing & Management 57 (6) (2020),.

[39]

Junqué de Fortuny E., De Smedt T., Martens D., Daelemans W., Evaluating and understanding text-based stock price prediction models, Information Processing & Management 50 (2) (2014) 426–441,.

Digital Library

[40]

Katz D.M., Bommarito II M.J., Blackman J., A general approach for predicting the behavior of the Supreme Court of the United States, PLoS One 12 (4) (2017) 1–18,.

[41]

Kaufhold M.-A., Bayer M., Reuter C., Rapid relevance classification of social media posts in disasters and emergencies: A system and evaluation featuring active, incremental and online learning, Information Processing & Management 57 (1) (2020),.

Digital Library

[42]

Kim M.-Y., Xu Y., Goebel R., Applying a convolutional neural network to legal question answering, in: Otake M., Kurahashi S., Ota Y., Satoh K., Bekki D. (Eds.), New frontiers in artificial intelligence, Springer International Publishing, 2017, pp. 282–294,.

[43]

Köksal A., Turkish pre-trained Word2Vec model, 2018, GitHub Repository https://github.com/akoksal/Turkish-Word2Vec.

[44]

Kowsrihawat K., Vateekul P., Boonkwan P., Predicting judicial decisions of criminal cases from Thai Supreme Court using bi-directional GRU with attention mechanism, in: 2018 5th Asian conference on defense technology, 2018, pp. 50–55,.

[45]

Kumar A., Srinivasan K., Cheng W.-H., Zomaya A.Y., Hybrid context enriched deep learning model for fine-grained sentiment analysis in textual and visual semiotic modality social data, Information Processing & Management 57 (1) (2020),.

Digital Library

[46]

Leitner E., Rehm G., Moreno-Schneider J., Fine-grained named entity recognition in legal documents, in: Acosta M., Cudré-Mauroux P., Maleshkova M., Pellegrini T., Sack H., Sure-Vetter Y. (Eds.), Semantic systems. The power of AI and knowledge graphs, Springer International Publishing, 2019, pp. 272–287,.

[47]

Li X., Wu P., Wang W., Incorporating stock prices and news sentiments for stock market prediction: A case of Hong Kong, Information Processing & Management 57 (5) (2020),.

[48]

Long S., Tu C., Liu Z., Sun M., Automatic judgment prediction via legal reading comprehension, in: Sun M., Huang X., Ji H., Liu Z., Liu Y. (Eds.), Chinese computational linguistics, Springer International Publishing, Cham, 2019, pp. 558–572,.

Digital Library

[49]

Martin A.D., Quinn K.M., Ruger T.W., Kim P.T., Competing approaches to predicting supreme court decision making, Perspectives on Politics 2 (4) (2004) 761–767,.

[50]

Mikolov T., Chen K., Corrado G., Dean J., Efficient estimation of word representations in vector space, 2013, arXiv:1301.3781.

[51]

Mikolov T., Sutskever I., Chen K., Corrado G., Dean J., Distributed representations of words and phrases and their compositionality, in: Proceedings of the 26th international conference on neural information processing systems - volume 2, Curran Associates Inc, Red Hook, NY, USA, 2013, pp. 3111–3119.

[52]

Morimoto A., Kubo D., Sato M., Shindo H., Matsumoto Y., Legal question answering system using neural attention, in: Satoh K., Kim M., Kano Y., Goebel R., Oliveira T. (Eds.), 4th competition on legal information extraction and entailment, held in conjunction with the 16th international conference on artificial intelligence and law in king’s college London, UK, in: EPiC series in computing, vol. 47, EasyChair, 2017, pp. 79–89,.

[53]

Nanda R., John A.K., Caro L.D., Boella G., Robaldo L., Legal information retrieval using topic clustering and neural networks, in: Satoh K., Kim M.-Y., Kano Y., Goebel R., Oliveira T. (Eds.), 4th competition on legal information extraction and entailment, in: EPiC series in computing, vol. 47, EasyChair, 2017, pp. 68–78,.

[54]

Nejadgholi I., Bougueng R., Witherspoon S., A semi-supervised training method for semantic search of legal facts in Canadian immigration cases, in: Wyner A.Z., Casini G. (Eds.), Legal knowledge and information systems, in: Frontiers in artificial intelligence and applications, vol. 302, IOS Press, 2017, pp. 125–134,.

[55]

Nguyen T.-S., Nguyen L.-M., Tojo S., Satoh K., Shimazu A., Recurrent neural network-based models for recognizing requisite and effectuation parts in legal texts, Artificial Intelligence and Law 26 (2) (2018) 169–199,.

Digital Library

[56]

O’Neill J., Buitelaar P., Robin C., O’Brien L., Classifying sentential modality in legal language: A use case in financial regulations, acts and directives, in: Proceedings of the 16th edition of the international conference on artificial intelligence and law, Association for Computing Machinery, New York, NY, USA, 2017, pp. 159–168,.

Digital Library

[57]

Pennington J., Socher R., Manning C., GloVe: Global vectors for word representation, in: Proceedings of the 2014 conference on empirical methods in natural language processing, Association for Computational Linguistics, Doha, Qatar, 2014, pp. 1532–1543,.

[58]

Qian Y., Deng X., Ye Q., Ma B., Yuan H., On detecting business event from the headlines and leads of massive online news articles, Information Processing & Management 56 (6) (2019),.

Digital Library

[59]

Rokach L., Maimon O., Decision trees, in: Maimon O., Rokach L. (Eds.), Data mining and knowledge discovery handbook, Springer US, Boston, MA, 2005, pp. 165–192,.

[60]

Ruger T., Kim P., Martin A., Quinn K., The supreme court forecasting project: Legal and political science approaches to predicting supreme court decisionmaking, Columbia Law Review 104 (2004) 1150–1210,.

[61]

Sangeetha D., Kavyashri R., Swetha S., Vignesh S., Information retrieval system for laws, in: 2016 8th international conference on advanced computing, IEEE, 2017, pp. 212–217,.

[62]

Sartor G., A simple computational model for nonmonotonic and adversarial legal reasoning, in: Proceedings of the 4th international conference on artificial intelligence and law, Association for Computing Machinery, New York, NY, USA, 1993, pp. 192–201,.

Digital Library

[63]

Schumaker R.P., Chen H., A quantitative stock prediction system based on financial news, Information Processing & Management 45 (5) (2009) 571–583,.

Digital Library

[64]

Shulayeva O., Siddharthan A., Wyner A., Recognizing cited facts and principles in legal judgements, Artificial Intelligence and Law 25 (1) (2017) 107–126,. Open access via Springer Compact Agreement.

Digital Library

[65]

Sleimi A., Sannier N., Sabetzadeh M., Briand L., Dann J., Automated extraction of semantic legal metadata using natural language processing, in: 2018 IEEE 26th international requirements engineering conference, IEEE, 2018, pp. 124–135,.

[66]

Şulea O., Zampieri M., Malmasi S., Vela M., Dinu L.P., van Genabith J., Exploring the use of text classification in the legal domain, in: Ashley K.D., Atkinson K., Branting L.K., Francesconi E., Grabmair M., Lauritsen M., Walker V.R., Wyner A.Z. (Eds.), Proceedings of the second workshop on automated semantic analysis of information in legal texts co-located with the 16th international conference on artificial intelligence and law, in: CEUR workshop proceedings, vol. 2143, CEUR-WS.org, 2017.

[67]

Şulea O.-M., Zampieri M., Vela M., van Genabith J., Predicting the law area and decisions of french supreme court cases, in: Proceedings of the international conference recent advances in natural language processing, INCOMA Ltd., Varna, Bulgaria, 2017, pp. 716–722,.

[68]

Tan P.-N., Steinbach M., Kumar V., Introduction to data mining, 1st ed., Addison-Wesley Longman Publishing Co., Inc, USA, 2005.

[69]

Tang G., Guo H., Guo Z., Xu S., Matching law cases and reference law provision with a neural attention model, IBM China Research, Beijing, 2016.

[70]

Tuke J., Nguyen A., Nasim M., Mellor D., Wickramasinghe A., Bean N., Mitchell L., Pachinko prediction: A Bayesian method for event prediction from social media data, Information Processing & Management 57 (2) (2020),.

Digital Library

[71]

Tür G., Hakkani-Tür D., Oflazer K., A statistical information extraction system for Turkish, Natural Language Engineering 9 (2) (2003) 181–210,.

Digital Library

[72]

Turney P.D., Pantel P., From frequency to meaning: Vector space models of semantics, Journal of Artificial Intelligence Research 37 (1) (2010) 141–188,.

[73]

Tursun E., Ganguly D., Osman T., Yang Y.-T., Abdukerim G., Zhou J.-L., Liu Q., A semisupervised tag-transition-based Markovian model for Uyghur morphology analysis, ACM Transactions on Asian and Low-Resource Language Information Processing 16 (2) (2016) 8:1–23,.

Digital Library

[74]

Virtucio M.B.L., Aborot J.A., Abonita J.K.C., Aviñante R.S., Copino R.J.B., Neverida M.P., Osiana V.O., Peramo E.C., Syjuco J.G., Tan G.B.A., Predicting decisions of the Philippine supreme court using natural language processing and machine learning, in: 2018 IEEE 42nd annual computer software and applications conference, IEEE, 2018, pp. 130–135,.

[75]

Wyner A., An ontology in OWL for legal case-based reasoning, Artificial Intelligence and Law 16 (361) (2008),.

Digital Library

Cited By

Bozdag MSevim NKoç A(2023)Measuring and Mitigating Gender Bias in Legal Contextualized Language ModelsACM Transactions on Knowledge Discovery from Data10.1145/362860218:4(1-26)Online publication date: 18-Oct-2023
https://dl.acm.org/doi/10.1145/3628602
Tyagi NBhushan B(2023)Demystifying the Role of Natural Language Processing (NLP) in Smart City Applications: Background, Motivation, Recent Advances, and Future Research DirectionsWireless Personal Communications: An International Journal10.1007/s11277-023-10312-8130:2(857-908)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1007/s11277-023-10312-8
Bhattacharya PGhosh KPal AGhosh S(2022)Legal case document similarityInformation Processing and Management: an International Journal10.1016/j.ipm.2022.10306959:6Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1016/j.ipm.2022.103069
Show More Cited By

Index Terms

Natural language processing in law: Prediction of outcomes in the higher courts of Turkey

Index terms have been assigned to the content through auto-classification.

Recommendations

Extracting value from Brazilian Court decisions
Abstract
We propose a methodology to extract value from Brazilian Court decisions to support judges and lawyers in their decision-making. We instantiate our methodology in one information system we have developed. Such system (i) extracts ...
Highlights
- The Brazilian legal system has seen an increase of the case law in the last years.
Lessons in Copyright Activism: K-12 Education and the DMCA 1201 Exemption Rulemaking Process

Digital learning is being transformed by changes in copyright law. This article discusses the author's personal journey as a copyright education activist through two rounds of rulemaking proceedings before the Copyright Office concerning the anti-...
The Study of the Interrelation between Law Programs and Digital Forensics in UAE Academia
InfoSecCD '13: Proceedings of the 2013 on InfoSecCD '13: Information Security Curriculum Development Conference

The field of digital forensics is growing in the Middle East which is shown by the establishment of technical digital forensic programs in various universities. Even though these programs are important for the development and advancement of the field ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Information Processing and Management: an International Journal

Information Processing and Management: an International Journal Volume 58, Issue 5

Sep 2021

986 pages

ISSN:0306-4573

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Pergamon Press, Inc.

United States

Publication History

Published: 01 September 2021

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 24 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Bozdag MSevim NKoç A(2023)Measuring and Mitigating Gender Bias in Legal Contextualized Language ModelsACM Transactions on Knowledge Discovery from Data10.1145/362860218:4(1-26)Online publication date: 18-Oct-2023
https://dl.acm.org/doi/10.1145/3628602
Tyagi NBhushan B(2023)Demystifying the Role of Natural Language Processing (NLP) in Smart City Applications: Background, Motivation, Recent Advances, and Future Research DirectionsWireless Personal Communications: An International Journal10.1007/s11277-023-10312-8130:2(857-908)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1007/s11277-023-10312-8
Bhattacharya PGhosh KPal AGhosh S(2022)Legal case document similarityInformation Processing and Management: an International Journal10.1016/j.ipm.2022.10306959:6Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1016/j.ipm.2022.103069
Şenel LŞahinuç FYücesoy VSchütze HÇukur TKoç A(2022)Learning interpretable word embeddings via bidirectional alignment of dimensions with semantic conceptsInformation Processing and Management: an International Journal10.1016/j.ipm.2022.10292559:3Online publication date: 1-May-2022
https://dl.acm.org/doi/10.1016/j.ipm.2022.102925

View Options

View options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents