short-paper

A new conceptual framework for enhancing legal information retrieval at the Brazilian Superior Court of Justice

Authors:

Marcelo LadeiraAuthors Info & Claims

MEDES '20: Proceedings of the 12th International Conference on Management of Digital EcoSystems

Pages 26 - 29

https://doi.org/10.1145/3415958.3433087

Published: 27 November 2020 Publication History

Abstract

Effective retrieval of jurisprudence (case-law) is imperative to achieve consistency and predictability for any legal system. In this work, we propose and proceed to an empirical evaluation of a framework for jurisprudence retrieval of the Brazilian Superior Court of Justice in order to ease the task of retrieval of other decisions with the same legal opinion. The experimental results shown that our approach based on text similarity performs better than the legacy system of the Court based on Boolean queries. The building of complex Boolean queries is very specialized and we aim to offer a tool able to use free text as queries without any operator. With the legacy system as baseline, we compare the TF-IDF traditional retrieval model, the BM25 probabilistic model and the Word2Vec model. Our results indicate that the Word2Vec Skip-Gram model, trained on a specialized legal corpus and BM25 yield similar performance and surpasses the legacy system. Combining BM25 model with embedding models improved the performance up to 19%.

References

[1]

Bruna Armonas Colombo, Pedro Buck, and Vinicius Miana Bezerra. 2017. Challenges When Using Jurimetrics in Brazil---A Survey of Courts. Future Internet 9, 4 (2017), 68.

[2]

Rhuan Barros, André Peres, Fabiana Lorenzi, Leandro Krug Wives, and Etiene Hubert da Silva Jaccottet. 2018. Case Law Analysis with Machine Learning in Brazilian Court. In International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems. Springer, 857--868.

[3]

Yen-Liang Chen, Yi-Hung Liu, and Wu-Liang Ho. 2013. A text mining approach to assist the general public in the retrieval of legal documents. Journal of the American Society for Information Science and Technology 64, 2 (2013), 280--290.

Digital Library

[4]

Pedro Henrique Luz de Araujo, Teófilo E de Campos, Renato RR de Oliveira, Matheus Stauffer, Samuel Couto, and Paulo Bermejo. 2018. LeNER-Br: A Dataset for Named Entity Recognition in Brazilian Legal Text. In International Conference on Computational Processing of the Portuguese Language. Springer, 313--323.

[5]

Diego de Vargas Feijó and Viviane Pereira Moreira. 2018. RulingBR: A Summarization Dataset for Legal Texts. In International Conference on Computational Processing of the Portuguese Language. Springer, 255--264.

[6]

Wael H Gomaa, Aly A Fahmy, et al. 2013. A survey of text similarity approaches. International Journal of Computer Applications 68, 13 (2013), 13--18.

[7]

Zellig S Harris. 1954. Distributional structure. Word 10, 2-3 (1954), 146--162.

[8]

Nathan Hartmann, Erick Fonseca, Christopher Shulby, Marcos Treviso, Jessica Rodrigues, and Sandra Aluisio. 2017. Portuguese word embeddings: Evaluating on word analogies and natural language tasks. arXiv preprint arXiv:1708.06025 (2017).

[9]

Kalervo Järvelin and Jaana Kekäläinen. 2017. IR evaluation methods for retrieving highly relevant documents. In ACM SIGIR Forum, Vol. 51. ACM New York, NY, USA, 243--250.

Digital Library

[10]

Karen Sparck Jones. 1972. A statistical interpretation of term specificity and its application in retrieval. Journal of documentation (1972).

[11]

Sushanta Kumar. 2014. Similarity Analysis of Legal Judgments and applying 'Paragraph-link'to Find Similar Legal Judgments. Ph.D. Dissertation. PhD thesis, International Institute of Information Technology Hyderabad.

[12]

Sushanta Kumar, P Krishna Reddy, V Balakista Reddy, and Aditya Singh. 2011. Similarity analysis of legal judgments. In Proceedings of the Fourth Annual ACM Bangalore Conference. ACM, 17.

Digital Library

[13]

Daniel Locke and Guido Zuccon. 2018. A test collection for evaluating legal case law search. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. ACM, 1261--1264.

Digital Library

[14]

Daniel Locke, Guido Zuccon, and Harrisen Scells. 2017. Automatic Query Generation from Legal Texts for Case Law Retrieval. In Asia Information Retrieval Symposium. Springer, 181--193.

Digital Library

[15]

Arpan Mandal, Raktim Chaki, Sarbajit Saha, Kripabandhu Ghosh, Arindam Pal, and Saptarshi Ghosh. 2017. Measuring similarity among legal court case documents. In Proceedings of the 10th Annual ACM India Compute Conference. ACM, 1--9.

Digital Library

[16]

Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 3111--3119.

Digital Library

[17]

Radim Řehůřek and Petr Sojka. 2010. Software Framework for Topic Modelling with Large Corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks. ELRA, Valletta, Malta, 45--50. http://is.muni.cz/publication/884893/en.

[18]

Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Now Publishers Inc.

Digital Library

[19]

Tony Russell-Rose, Jon Chamberlain, and Leif Azzopardi. 2018. Information retrieval in the workplace: A comparison of professional search practices. Information Processing& Management 54, 6 (2018), 1042--1057.

[20]

Luis Sanchez, Jiyin He, Jarana Manotumruksa, Dyaa Albakour, Miguel Martinez, and Aldo Lipani. 2020. Easing Legal News Monitoring with Learning to Rank and BERT. In Advances in Information Retrieval, Joemon M.Jose, Emine Yilmaz, João Magalhaes, Pablo Castells, Nicola Ferro, Mário J. Silva, and Flávio Martins (Eds.). Springer International Publishing, Cham, 336--343.

[21]

Milagro Teruel, Cristian Cardellino, Fernando Cardellino, Laura Alonso Alemany, and Serena Villata. 2018. Legal text processing within the MIREL project. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018) (Miyazaki, Japan, 7-12), Georg Rehm, Víctor Rodríguez-Doncel, and Julián Moreno-Schneider (Eds.). European Language Resources Association (ELRA), Paris, France.

[22]

Edwin Thuma and Nkwebi Peace Motlogelwa. 2017. On the importance of Legal Catchphrases in Precedence Retrieval. In FIRE (Working Notes). 92--94.

[23]

Alberto Tonon, Gianluca Demartini, and Philippe Cudré-Mauroux. 2015. Pooling-based continuous evaluation of information retrieval systems. Information Retrieval Journal 18, 5 (2015), 445--472.

Digital Library

[24]

Marc Van Opijnen and Cristiana Santos. 2017. On the concept of relevance in legal information retrieval. Artificial Intelligence and Law 25, 1 (2017), 65--87.

Digital Library

Cited By

Vitório DSouza EMartins Lda Silva Nde Carvalho AOliveira Ade Andrade F(2024)Building a relevance feedback corpus for legal information retrieval in the real-case scenario of the Brazilian Chamber of DeputiesLanguage Resources and Evaluation10.1007/s10579-024-09767-3Online publication date: 18-Aug-2024
https://doi.org/10.1007/s10579-024-09767-3
dos Santos JSouza EFilho CAlbuquerque HVitório Dde Lucena DSilva Nde Carvalho A(2024)HIRS: A Hybrid Information Retrieval System for Legislative DocumentsProgress in Artificial Intelligence10.1007/978-3-031-73497-7_26(320-331)Online publication date: 16-Nov-2024
https://doi.org/10.1007/978-3-031-73497-7_26
Sakiyama KNogueira RRomero R(2023)Automated Keyphrase Generation for Brazilian Legal Information Retrieval2023 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN54540.2023.10191598(1-8)Online publication date: 18-Jun-2023
https://doi.org/10.1109/IJCNN54540.2023.10191598
Show More Cited By

Index Terms

A new conceptual framework for enhancing legal information retrieval at the Brazilian Superior Court of Justice
1. Applied computing
  1. Law, social and behavioral sciences
    1. Law
2. Information systems
  1. Information retrieval
    1. Retrieval models and ranking

Recommendations

Evaluation of information retrieval for E-discovery

The effectiveness of information retrieval technology in electronic discovery (E-discovery) has become the subject of judicial rulings and practitioner controversy. The scale and nature of E-discovery tasks, however, has pushed traditional information ...
Legal content fusion for legal information retrieval
ICAIL '17: Proceedings of the 16th edition of the International Conference on Articial Intelligence and Law

With recent increasing attention to legal information processing, legal information retrieval (IR) has become one of the active research fields. However, there are still many hindrances obtaining rigorous results in legal IR applications in comparison ...
A quantitative approach to ranking corporate law precedents in the Brazilian Superior Court of Justice
Abstract
This paper aims to contribute to the goal of finding influential legal precedents by quantitative methods. A lot of work has been made in this direction worldwide, especially in the context of common law jurisdictions. However, this type of work ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

MEDES '20: Proceedings of the 12th International Conference on Management of Digital EcoSystems

November 2020

170 pages

ISBN:9781450381154

DOI:10.1145/3415958

General Chairs:
Richard Chbeir
University Pau & Pays de l'Adour, France
,
Yannis Manolopoulos
Open University of Cyprus, Cyprus
,
Ernesto Damiani
Khalifa University of Science and Technology, UAE
,
Djamal Benslimane
Lyon 1 University, France
,
Program Chairs:
Ladjel Bellatreche
LIAS/ISAE-ENSMA, France
,
Tadeusz Morzy
Poznan University of Technology, Poland

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGAPP: ACM Special Interest Group on Applied Computing

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 November 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper
Research
Refereed limited

Conference

MEDES '20

MEDES '20: 12th International Conference on Management of Digital EcoSystems

November 2 - 4, 2020

Virtual Event, United Arab Emirates

Acceptance Rates

MEDES '20 Paper Acceptance Rate 19 of 27 submissions, 70%;

Overall Acceptance Rate 267 of 682 submissions, 39%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
120
Total Downloads

Downloads (Last 12 months)28
Downloads (Last 6 weeks)3

Reflects downloads up to 16 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Vitório DSouza EMartins Lda Silva Nde Carvalho AOliveira Ade Andrade F(2024)Building a relevance feedback corpus for legal information retrieval in the real-case scenario of the Brazilian Chamber of DeputiesLanguage Resources and Evaluation10.1007/s10579-024-09767-3Online publication date: 18-Aug-2024
https://doi.org/10.1007/s10579-024-09767-3
dos Santos JSouza EFilho CAlbuquerque HVitório Dde Lucena DSilva Nde Carvalho A(2024)HIRS: A Hybrid Information Retrieval System for Legislative DocumentsProgress in Artificial Intelligence10.1007/978-3-031-73497-7_26(320-331)Online publication date: 16-Nov-2024
https://doi.org/10.1007/978-3-031-73497-7_26
Sakiyama KNogueira RRomero R(2023)Automated Keyphrase Generation for Brazilian Legal Information Retrieval2023 International Joint Conference on Neural Networks (IJCNN)10.1109/IJCNN54540.2023.10191598(1-8)Online publication date: 18-Jun-2023
https://doi.org/10.1109/IJCNN54540.2023.10191598
Vitório DSouza EMartins Lda Silva Nde Leon Ferreira de Carvalho AOliveira A(2022)Ulysses-RFSQ: A Novel Method to Improve Legal Information Retrieval Based on Relevance FeedbackIntelligent Systems10.1007/978-3-031-21686-2_6(77-91)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.1007/978-3-031-21686-2_6
Dias Canedo EAymoré Martins VCoelho Ribeiro Vdos Reis VCarvalho Chaves LMachado Gravina RAlberto Moreira Dias FLopes de Mendonça FOrozco ABalaniuk Rde Sousa R(2021)Development and Evaluation of an Intelligence and Learning System in Jurisprudence Text Mining in the Field of Competition DefenseApplied Sciences10.3390/app11231136511:23(11365)Online publication date: 1-Dec-2021
https://doi.org/10.3390/app112311365

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents