demonstration

Semi-Supervised Methods for Explainable Legal Prediction

Authors:

A. Chakraborty,

A. YehAuthors Info & Claims

ICAIL '19: Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law

Pages 22 - 31

https://doi.org/10.1145/3322640.3326723

Published: 17 June 2019 Publication History

Abstract

Legal decision-support systems have the potential to improve access to justice, administrative efficiency, and judicial consistency, but broad adoption of such systems is contingent on development of technologies with low knowledge-engineering, validation, and maintenance costs. This paper describes two approaches to an important form of legal decision support---explainable outcome prediction---that obviate both annotation of an entire decision corpus and manual processing of new cases. The first approach, which uses an Attention Network for prediction and attention weights to highlight salient case text, was shown to be capable of predicting decisions, but attention-weight-based text highlighting did not demonstrably improve human decision speed or accuracy in an evaluation with 61 human subjects. The second approach, termed SCALE (Semi-supervised Case Annotation for Legal Explanations), exploits structural and semantic regularities in case corpora to identify textual patterns that have both predictable relationships to case decisions and explanatory value.

References

[1]

Al-Abdulkarim, L., Atkinson, K., Bench-Capon, T.J.M., Whittle, S., Williams, R., Wolfenden, C.: Noise induced hearing loss: An application of the angelic methodology. In: Legal Knowledge and Information Systems - JURIX 2017: The Thirtieth Annual Conference, Luxembourg, 13-15 December 2017. pp. 79--88 (2017)

[2]

Alarie, B., Niblett, A., Yoon, A.: Using Machine Learning to Predict Outcomes in Tax Law (December 15 2017), available at SSRN: https://ssrn.com/abstract=2855977 or

[3]

Aletras, N., Tsarapatsanis, D., Preotiuc-Pietro, D., Lampos, V.: Predicting judicial decisions of the European Court of Human Rights: a natural language processing perspective. PeerJ CompSci (October 24 2016), https://peerj.com/articles/cs-93/

[4]

Aleven, V., Ashley, K.: Evaluating a learning environment for case-based argumentation skills. In: Proceedings of the Sixth International Conference on Artificial Intelligence and Law. pp. 170--179. ACM Press, University of Melbourne, Melbourne, Australia (June 30-July 3, 1997)

Digital Library

[5]

Aleven, V.A.W.M.M.: Teaching Case-based Argumentation Through a Model and Examples. Ph.D. thesis, University of Pittsburgh, Pittsburgh, PA, USA (1997), aAI9821228

Digital Library

[6]

Ashley, K.D., Brüninghaus, S.: Automatically classifying case texts and predicting outcomes. Artif. Intell. Law 17(2), 125--165 (Jun 2009)

Digital Library

[7]

Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)

[8]

Bench-Capon, T.J.M., Dunne, P.E.: Argumentation in artificial intelligence. Artif. Intell. 171(10-15), 619--641 (Jul 2007)

Digital Library

[9]

Berger, A.L., Pietra, V.J.D., Pietra, S.A.D.: A maximum entropy approach to natural language processing. Computational Linguistics 22(1), 39--71 (Mar 1996)

Digital Library

[10]

Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information. CoRR abs/1607.04606 (2016), http://arxiv.org/abs/1607.04606

[11]

Boles, D.B., Adair, L.P.: The multiple resources questionnaire (mrq). Proceedings of the Human Factors and Ergonomics Society Annual Meeting 45(25), 1790--1794 (2001)

[12]

Branting, L.K.: Reasoning with Rules and Precedents: A Computational Model of Legal Analysis. Kluwer Academic Publishers, Dordrect/Boston/London (2000)

[13]

Branting, L.K., Yeh, A., Weiss, B., Merkhofer, E.M., Brown, B.: Inducing predictive models for decision support in administrative adjudication. In: AI Approaches to the Complexity of Legal Systems - AICOL International Workshops 2015-2017, Revised Selected Papers. Lecture Notes in Computer Science, vol. 10791, pp. 465--477. Springer (2017)

[14]

Brooke, J.: SUS--a quick and dirty usability scale. Usability evaluation in industry 189(194), 4--7 (1996)

[15]

Brüninghaus, S., Ashley, K.D.: Toward adding knowledge to learning algorithms for indexing legal cases. In: Proceedings of the 7th International Conference on Artificial Intelligence and Law. pp. 9--17. ICAIL '99, ACM, New York, NY, USA (1999).

Digital Library

[16]

Bruninghaus, S., Ashley, K.D.: Predicting outcomes of case based legal arguments. In: Proceedings of the 9th International Conference on Artificial Intelligence and Law. pp. 233--242. ICAIL '03, ACM, New York, NY, USA (2003)

Digital Library

[17]

Chen, T., Guestrin, C.: Xgboost: A scalable tree boosting system. In: Proceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. pp. 785--794. KDD '16, ACM, New York, NY, USA (2016)

Digital Library

[18]

Gunning, D.: Defense advanced research projects agency (darpa) program information: Explainable artificial intelligence (xai) (2018), https://www.darpa.mil/program/explainable-artificial-intelligence, last visited December 26, 2018

[19]

Hadfield, G.: Rules for a Flat World: Why Humans Invented Law and How to Reinvent It for a Complex Global Economy. Oxford University Press (2016)

[20]

Katz, D.M., II, M.J.B., Blackman, J.: A general approach for predicting the behavior of the supreme court of the united states. PLoSOne 12(4) (2017)

[21]

Lippi, M., Torroni, P.: Argumentation mining: State of the art and emerging trends. ACM Trans. Internet Technol. 16(2), 10:1--10:25 (Mar 2016)

Digital Library

[22]

Maxwell, K.T., Oberlander, J., Lavrenko, V.: Evaluation of semantic events for legal case retrieval. In: Proceedings of the WSDM '09 Workshop on Exploiting Semantic Annotations in Information Retrieval. pp. 39--41. ESAIR '09, ACM, New York, NY, USA (2009).

Digital Library

[23]

McCarty, L.T.: Research Handbook on the Law of Artificial Intelligence, chap. Finding the right balance in artificial intelligence and law. Edward Elgar Publishing (2018)

[24]

Mikolov, T., Yih, S.W.t., Zweig, G.: Linguistic regularities in continuous space word representations. In: Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT-2013). Association for Computational Linguistics (May 2013)

[25]

Peterson, M., Waterman, D.: Rule-based models of legal expertise. In: Walters, C. (ed.) Computing Power and Legal Reasoning, pp. 627--659. West Publishing Company, Minneapolis, Minnesota (1985)

[26]

Rissland, E., Skalak, D., Friedman, T.: Supporting legal arguments through heuristic retrieval. Artificial Intelligence and Law 4(1), 1--71 (1996)

Digital Library

[27]

Rissland, E.L., Ashley, K.D., Branting, L.K.: Case-based reasoning and law. The Knowledge Engineering Review 20(3), 293--298 (2005)

Digital Library

[28]

Rush, A.M., Chopra, S., Weston, J.: A neural attention model for abstractive sentence summarization. CoRR abs/1509.00685 (2015), http://arxiv.org/abs/1509.00685

[29]

Sauro, J., Dumas, J.S.: Comparison of three one-question, post-task usability questionnaires. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 1599--1608. CHI '09 (2009)

Digital Library

[30]

Sergot, M.J., Sadri, F., Kowalski, R.A., Kriwaczek, F., Hammond, P., Cory, H.T.: The British Nationality Act as a logic program. Commun. ACM 29(5), 370--386 (May 1986).

Digital Library

[31]

Shulayeva, O., Siddharthan, A., Wyner, A.: Recognizing cited facts and principles in legal judgements. Artificial Intelligence and Law 25(1), 107--126 (Mar 2017)

Digital Library

[32]

Sulea, O., Zampieri, M., Vela, M., van Genabith, J.: Predicting the law area and decisions of french supreme court cases. In: RANLP. pp. 716--722. INCOMA Ltd. (2017)

[33]

Surdeanu, M., Nallapati, R., Gregory, G., Walker, J., Manning, C.: Risk analysis for intellectual property litigation. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Law. ACM, Pittsburgh, PA (June 6-10 2011)

Digital Library

[34]

Wyner, A.Z., Peters, W.: Lexical semantics and expert legal knowledge towards the identification of legal case factors. In: JURIX. Frontiers in Artificial Intelligence and Applications, vol. 223, pp. 127--136. IOS Press (2010)

Digital Library

[35]

The EXplainable AI in Law (XAILA) 2018 workshop. http://xaila.geist.re (December 12-14 2018), Groningen, The Netherlands

[36]

Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of NAACL-HLT. pp. 1480--1489 (2016)

Cited By

Jakab R(2024)National Report on Automation in Decision-Making in Public Administration in SlovakiaAUC IURIDICA10.14712/23366478.2024.2870:2(147-157)Online publication date: 23-May-2024
https://doi.org/10.14712/23366478.2024.28
Savelka JAshley K(2023)The unreasonable effectiveness of large language models in zero-shot semantic annotation of legal textsFrontiers in Artificial Intelligence10.3389/frai.2023.12797946Online publication date: 17-Nov-2023
https://doi.org/10.3389/frai.2023.1279794
Mozer RMiratrix LRelyea JKim J(2023)Combining Human and Automated Scoring Methods in Experimental Assessments of Writing: A Case Study TutorialJournal of Educational and Behavioral Statistics10.3102/10769986231207886Online publication date: 8-Nov-2023
https://doi.org/10.3102/10769986231207886
Show More Cited By

Index Terms

Semi-Supervised Methods for Explainable Legal Prediction
1. Applied computing
  1. Law, social and behavioral sciences
    1. Law
2. Computing methodologies
  1. Artificial intelligence
    1. Natural language processing

Recommendations

Scalable and explainable legal prediction
Abstract
Legal decision-support systems have the potential to improve access to justice, administrative efficiency, and judicial consistency, but broad adoption of such systems is contingent on development of technologies with low knowledge-engineering, ...
Explainable AI tools for legal reasoning about cases: A study on the European Court of Human Rights
Abstract
In this paper we report on a significant research project undertaken to design, implement and evaluate explainable decision-support tools for deciding legal cases. We provide a model of a legal domain, Article 6 of the European ...
Predicting trade secret case outcomes using argument schemes and learned quantitative value effect tradeoffs
ICAIL '17: Proceedings of the 16th edition of the International Conference on Articial Intelligence and Law

This paper presents the Value Judgment Formalism and its experimental implementation in the VJAP system, which is capable of arguing about, and predicting outcomes of, a set of trade secret misappropriation cases. VJAP creates an argument graph for each ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

ICAIL '19: Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law

June 2019

312 pages

ISBN:9781450367547

DOI:10.1145/3322640

Copyright © 2019 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGAI: ACM Special Interest Group on Artificial Intelligence

In-Cooperation

Univ. of Montreal: University of Montreal
AAAI
IAAIL: Intl Asso for Artifical Intel & Law

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 June 2019

Check for updates

Author Tags

Qualifiers

Demonstration
Research
Refereed limited

Conference

ICAIL '19

Sponsor:

SIGAI

ICAIL '19: Seventeenth International Conference on Artificial Intelligence and Law

June 17 - 21, 2019

QC, Montreal, Canada

Acceptance Rates

Overall Acceptance Rate 69 of 169 submissions, 41%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
671
Total Downloads

Downloads (Last 12 months)58
Downloads (Last 6 weeks)4

Reflects downloads up to 22 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Jakab R(2024)National Report on Automation in Decision-Making in Public Administration in SlovakiaAUC IURIDICA10.14712/23366478.2024.2870:2(147-157)Online publication date: 23-May-2024
https://doi.org/10.14712/23366478.2024.28
Savelka JAshley K(2023)The unreasonable effectiveness of large language models in zero-shot semantic annotation of legal textsFrontiers in Artificial Intelligence10.3389/frai.2023.12797946Online publication date: 17-Nov-2023
https://doi.org/10.3389/frai.2023.1279794
Mozer RMiratrix LRelyea JKim J(2023)Combining Human and Automated Scoring Methods in Experimental Assessments of Writing: A Case Study TutorialJournal of Educational and Behavioral Statistics10.3102/10769986231207886Online publication date: 8-Nov-2023
https://doi.org/10.3102/10769986231207886
Savelka JAndrade FGrabmair M(2023)Unlocking Practical Applications in Legal DomainProceedings of the Nineteenth International Conference on Artificial Intelligence and Law10.1145/3594536.3595161(447-451)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3594536.3595161
Nielsen ASkylaki SNorkute MStremitzer AAndrade FGrabmair M(2023)Effects of XAI on Legal ProcessProceedings of the Nineteenth International Conference on Artificial Intelligence and Law10.1145/3594536.3595128(442-446)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3594536.3595128
Blass JForbus KAndrade FGrabmair M(2023)Analogical Reasoning, Generalization, and Rule Learning for Common Law ReasoningProceedings of the Nineteenth International Conference on Artificial Intelligence and Law10.1145/3594536.3595121(32-41)Online publication date: 19-Jun-2023
https://dl.acm.org/doi/10.1145/3594536.3595121
Collenette JAtkinson KBench-Capon T(2023)Explainable AI tools for legal reasoning about cases: A study on the European Court of Human RightsArtificial Intelligence10.1016/j.artint.2023.103861317(103861)Online publication date: Apr-2023
https://doi.org/10.1016/j.artint.2023.103861
Ganguly DConrad JGhosh KGhosh SGoyal PBhattacharya PNigam SPaul S(2023)Legal IR and NLP: The History, Challenges, and State-of-the-ArtAdvances in Information Retrieval10.1007/978-3-031-28241-6_34(331-340)Online publication date: 16-Mar-2023
https://doi.org/10.1007/978-3-031-28241-6_34
Vaissnave V. Deepalakshmi P. (2022)Analysis on Hybrid Deep Neural Networks for Legal Domain MultitasksInternational Journal of e-Collaboration10.4018/IJeC.30125718:1(1-22)Online publication date: 1-Jun-2022
https://doi.org/10.4018/IJeC.301257
Chen HPieptea LDing J(2022)Construction and Evaluation of a High-Quality Corpus for Legal Intelligence Using Semiautomated ApproachesIEEE Transactions on Reliability10.1109/TR.2022.315612671:2(657-673)Online publication date: Jun-2022
https://doi.org/10.1109/TR.2022.3156126
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents