Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

A Comparison of Logistic Regression, Classification and Regression Tree, and Neural Networks Models in Predicting Violent Re-Offending

  • Original Paper
  • Published:
Journal of Quantitative Criminology Aims and scope Submit manuscript

Abstract

Previous studies that have compared logistic regression (LR), classification and regression tree (CART), and neural networks (NNs) models for their predictive validity have shown inconsistent results in demonstrating superiority of any one model. The three models were tested in a prospective sample of 1225 UK male prisoners followed up for a mean of 3.31 years after release. Items in a widely-used risk assessment instrument (the Historical, Clinical, Risk Management-20, or HCR-20) were used as predictors and violent reconvictions as outcome. Multi-validation procedure was used to reduce sampling error in reporting the predictive accuracy. The low base rate was controlled by using different measures in the three models to minimize prediction error and achieve a more balanced classification. Overall accuracy of the three models varied between 0.59 and 0.67, with an overall AUC range of 0.65–0.72. Although the performance of NNs was slightly better than that of LR and CART models, it did not demonstrate a significant improvement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Notes

  1. $$ {\text{sensitivity}} = {\frac{\text{number of true positives}}{{{\text{number of true positives}} + {\text{number of false negatives}}}}} $$
    $$ {\text{specificity}} = {\frac{\text{number of true negatives}}{{{\text{number of true negatives}} + {\text{number of false positives}}}}} $$

  2. MCR is a statistic for measuring the accuracy of instruments to predict recidivism, by means of complex computation. For more details about MCR see: Greene et al. (1994).

  3. CART is designed to fit binary classification trees, while CHAID and some other CT models perform multi-level splits rather than binary splits when computing classification trees. It should be noted that there is no inherent advantage of multi-level splits, because any multi-level split can be represented as a series of binary splits. Please see http://www.statsoft.com/textbook/classification-trees/.

References

  • SPSS Inc. (2008) SPSS for windows, Rel. 16.0.1. 2008. SPSS Inc., Chicago. http://www.spss.com/

  • StatSoft Inc. (2008) STATISTICA (data analysis software system), version 8.0. http://www.statsoft.com

  • Banks S, Robbins PC, Silver E, Vesselinov R, Steadman HJ, Monahan J (2004) A multiple-models approach to violence risk assessment among people with mental disorder. Crim Justice Behav 31:324–340

    Article  Google Scholar 

  • Bigi R, Gregori D, Cortigiani L, Desideri A, Chiarotto FA, Toffolo GM (2005) Artificial neural networks and robust Bayesian classifiers for risk stratification following uncomplicated myocardial infarction. Int J Cardiol 101:481–487

    Article  Google Scholar 

  • Bishop C (1995) Neural networks for pattern recognition. Oxford University Press, New York

    Google Scholar 

  • Breiman L (2001) Decision tree forests. Mach Learn 45:5–32

    Article  Google Scholar 

  • Breiman L, Friedman JH, Olshen RA, Stone CJ (1984) Classification and regression trees. Wadsworth and Brooks/Cole, Monterey, CA

    Google Scholar 

  • Brodzinski JD, Crable EA, Scherer RF (1994) Using artificial intelligence to model juvenile recidivism patterns. Comput Hum Serv 10:1–18

    Google Scholar 

  • Caulkins J, Cohen J, Gorr W, Wei J (1996) Predicting criminal recidivism: a comparison of neural network models with statistical methods. J Crim Just 24:227–240

    Article  Google Scholar 

  • Cicchetti DV (1992) Neural network and diagnosis in the clinical laboratory: state of the art. Clin Chem 38:9–10

    Google Scholar 

  • Cohen J (1990) Things I have learned (so far). Am Psychol 45:1304–1312

    Article  Google Scholar 

  • Coid JW, Yang M, Ullrich S, Zhang TQ, Sizmur S, Farrington DP (2010) Improving accuracy of risk prediction for violence: does changing the outcome matter? Int J Offender Ther (Submitted)

  • Coid JW, Yang M, Ullrich S, Zhang TQ, Sizmur S, Roberts C, Farrington DP (2011) Most items in structured risk assessment instruments do not predict violence. J Forensic Psychiatr Psychol 22. doi:10.1080/14789949.2010.495990

  • Colombet I, Ruelland A, Chatellier G, Gueyffier F, Degoulet P, Jaulent MC (2000) Models to predict cardiovascular risk: comparison of CART, multilayer perceptron and logistic regression. In: Proceedings/AMIA annual symposium. AMIA symposium 2000, pp 156–160

  • Comp.ai.neural-nets FAQ, Part 3: Generalization. http://www.faqs.org/faqs/ai-faq/neural-nets/part3/preamble.html

  • Cooke DJ, Michie C, Ryan J (2001) Evaluating risk for violence: a preliminary study of the HCR-20, PCL-R and VRAG in a Scottish prison sample. Scottish Prison Service occasional papers

  • Dahle KP (2006) Strengths and limitations of actuarial prediction of criminal reoffence in a German prison sample: a comparative study of LSI-R, HCR-20 and PCL-R. Int J Law Psychiat 29:431–442

    Article  Google Scholar 

  • Dawes RM, Faust D, Meehl PE (1989) Clinical versus actuarial judgement. Science 243:1668–1674

    Article  Google Scholar 

  • Derogatis LR, Melisaratos N (1983) The brief symptom inventory: an introductory report. Psychol Med 13:595–605

    Article  Google Scholar 

  • Dillard E, Luchette FA, Sears BW, Norton J, Schermer CR, Reed RL (2007) Clinician vs mathematical statistical models: which is better at predicting an abnormal chest radiograph finding in injured patients? Am J Emerg Med 25:823–830

    Article  Google Scholar 

  • Dolan M, Khawaja A (2004) The HCR-20 and post-discharge outcome in male patients discharged from medium security in the UK. Aggress Behav 30:469–483

    Article  Google Scholar 

  • Douglas KS, Ogloff JRP, Nicholls TL, Grant I (1999) Assessing risk for violence among psychiatric patients: the HCR-20 violence risk assessment scheme and the psychopathy checklist: screening version. J Consult Clin Psych 67:917–930

    Article  Google Scholar 

  • Doyle M, Dolan M (2006) Predicting community violence from patients discharged from mental health services. Brit J Psychiat 189:520–526

    Article  Google Scholar 

  • Farrington DP, Jolliffe D, Johnstone L (2008) Assessing violence risk: a framework for practice. Institute of Criminology, Cambridge University, Cambridge

    Google Scholar 

  • Florio T, Einfeld S, Levy F (1994) Neural networks and psychiatry: candidate applications in clinical decision making. Aust NZ J Psychiat 28:651–666

    Article  Google Scholar 

  • Forthofer MS, Bryant CA (2000) Using audience-segmentation techniques to tailor health behavior change strategies. Am J Health Behav 24:36–43

    Google Scholar 

  • Friedman JH (1999a) Greedy function approximation: a gradient boosting machine. IMS 1999 Reitz Lecture

  • Friedman JH (1999b) Stochastic gradient boosting. Stanford University, Stanford

    Google Scholar 

  • Gardner W, Lidz CW, Mulvey EP, Shaw EC (1996) A comparison of actuarial methods for identifying repetitively violent patients with mental illnesses. Law Human Behav 20:35–48

    Article  Google Scholar 

  • Gendreau P, Goggin C, Smith P (2002) Is the PCL-R really the “unparalleled” measure of offender risk? A lesson in knowledge cumulation. Crim Justice Behav 29:397–426

    Article  Google Scholar 

  • Gigerenzer G, Todd PM, Group AR (1999) Simple heuristics that makes us smart. Oxford University Press, New York

    Google Scholar 

  • Glover A, Nicholson D, Hemmati T, Bernfeld G, Quinsey V (2002) A comparison of predictors of general and violent recidivism among high risk federal offenders. Crim Justice Behav 29:235–249

    Article  Google Scholar 

  • Gottfredson SD, Moriarty LJ (2006) Statistical risk assessment: old problems and new applications. Crime Delinquency 52:178–200

    Article  Google Scholar 

  • Grann M, Langstrom N (2007) Actuarial Assessment of Violence Risk: To Weigh or Not to Weigh? Crim Justice Behav 34:22–36

    Article  Google Scholar 

  • Gray NS, Hill C, McGleish A, Timmons D, MacCulloch MJ, Snowden RJ (2003) Prediction of violence and self-harm in mentally disordered offenders: a prospective study of the efficacy of HCR-20, PCL-R and psychiatric symptomology. J Consult Clin Psych 71:443–451

    Article  Google Scholar 

  • Gray NS, Taylor J, Snowden RJ (2008) Predicting violent reconvictions using the HCR-20. Brit J Psychiat 192:384–387

    Article  Google Scholar 

  • Green DM, Swets JA (1966) Signal detection theory and psychophysics. Wiley, New York

    Google Scholar 

  • Greene MA, Hoffman PB, Beck JL (1994) The mean cost rating (MCR) is Somers’ D: a methodological note. J Crim Justice 22:63–69

    Article  Google Scholar 

  • Grevatt M, Thomas-Peter B, Hughes G (2004) Violence, mental disorder and risk assessment: can structured clinical assessments predict the short-term risk of inpatient violence? J Forensic Psychiat Psychol 15:278–292

    Article  Google Scholar 

  • Grove WM, Meehl PE (1996) Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: The clinical-statistical controversy. Psychol Public Pol L 2:293–323

    Article  Google Scholar 

  • Guerriere MR, Detsky AS (1991) Neural networks: what are they? Ann Intern Med 115:906–907

    Google Scholar 

  • Hanson RK (2005) Twenty years of progress in violence risk assessment. J Interpers Violence 20:212–217

    Article  Google Scholar 

  • Harper PR (2005) A review and comparison of classification algorithms for medical decision making. Health Policy 71:315–331

    Article  Google Scholar 

  • Hart SD, Webster CD, Menzies RJ (1993) A note on portraying the accuracy of violence predictions. Law Human Behav 17:695–700

    Article  Google Scholar 

  • Hartvig P, Alfarnes S, Ostberg B, Skjønberg M, Moger TA (2006) Brief checklists for assessing violence risk among patients discharged from acute psychiatric facilities: a preliminary study. Nord J Psychiat 60:243–248

    Article  Google Scholar 

  • Hemphill JF, Hare RD, Wong S (1998) Psychopathy and recidivism: a review. Legal Criminol Psychol 3:139–170

    Article  Google Scholar 

  • Henderson AR (1993) Assessing test accuracy and its clinical consequences: a primer for receiver operating characteristic curve analysis. Ann Clin Biochem 30:521–539

    Google Scholar 

  • Hosmer DW, Lemeshow S (1989) Applied logistic regression. Wiley, New York

    Google Scholar 

  • Howard P, Kershaw C (2000) Using criminal career data in evaluation. British criminology conference: selected proceedings, 3. Available online at www.lboro.ac.uk/departments/ss/bsc/bccsp/vol03/howard.html

  • Kass GV (1980) An exploratory technique for investigating large quantities of categorical data. Appl Stat 29:119–127

    Article  Google Scholar 

  • Kroner DG, Mills JF (2001) The accuracy of five risk appraisal instruments in predicting institutional misconduct and new convictions. Crim Justice Behav 28:471–489

    Article  Google Scholar 

  • Kroner DG, Mills JF, Reddon JR (2005) A coffee can, factor analysis, and prediction of antisocial behavior: the structure of criminal risk. Int J Law Psychiat 28:360–374

    Article  Google Scholar 

  • Lemon SC, Roy J, Clark MA, Friedmann PD, Rakowski W (2003) Classification and regression tree analysis in public health: methodological review and comparison with logistic regression. Ann Behav Med 26:172–181

    Article  Google Scholar 

  • Lidz CW, Mulvey EP, Gardner W (1993) The accuracy of predictions of violence to others. J Am Med Assn 269:1007–1011

    Article  Google Scholar 

  • Lin CH, Chou LS, Lin CH, Hsu CY, Chen YS, Lane HY (2007) Early prediction of clinical response in schizophrenia patients receiving the atypical antipsychotic zotepine. J Clin Psychiat 68:1522–1527

    Article  Google Scholar 

  • Loh WY, Shih YS (1997) Split selection methods for classification trees. Stat Sinica 7:815–840

    Google Scholar 

  • Manly BFJ (2005) Multivariate statistical methods: a primer, 3rd edn. Chapman and Hall/CRC, Boca Raton

    Google Scholar 

  • Maria-Pia Victoria-Feser (2000) Robust logistic regression for binomial responses. University of Geneva, Geneva

    Google Scholar 

  • McCulloch WS, Pitts W (1943) A logical calculus of ideas immanent in nervous activity. Bull Math Biophys 5:115–133

    Article  Google Scholar 

  • Meehl PE, Rosen A (1955) Antecedent probability and the efficiency of psychometric signs, patterns, or cutting scores. Psychol Bull 52:194–216

    Article  Google Scholar 

  • Monahan J, Steadman HJ, Appelbaum PS, Robbins PC, Mulvey EP, Silver E (2000) Developing a clinically useful actuarial tool for assessing violence risk. Brit J Psychiat 176:312–320

    Article  Google Scholar 

  • Monahan J, Steadman HJ, Robbins PC, Appelbaum P, Banks S, Grisso T (2005) An actuarial model of violence risk assessment for persons with mental disorders. Psychiatr Serv 56:810–815

    Article  Google Scholar 

  • Monahan J, Steadman HJ, Appelbaum PS, Grisso T, Mulvey EP, Roth LH (2006) The classification of violence risk. Behav Sci Law 24:721–730

    Article  Google Scholar 

  • Mossman D (1994) Assessing predictions of violence: being accurate about accuracy. J Consult Clin Psych 62:783–792

    Article  Google Scholar 

  • National Institute of Justice (1992) Data resources of the National Institute of Justice, 5th edn. National Institute of Justice, Washington, DC

    Google Scholar 

  • Nicholls TL, Ogloff JRP, Douglas KS (2004) Assessing risk for violence among male and female civil psychiatric patients: the HCR-20, PCL: SV, and VSC. Behav Sci Law 22:127–158

    Article  Google Scholar 

  • Ning GM, Su J, Li YQ, Wang XY, Li CH, Yan WM (2006) Artificial neural network based model for cardiovascular risk stratification in hypertension. Med Biol Eng Comput 44:202–208

    Article  Google Scholar 

  • Palocsay SW, Wang P, Brookshire RG (2000) Predicting criminal recidivism using neural networks. Socio Econ Plan Sci 34:271–284

    Article  Google Scholar 

  • Patterson D (1996) Artificial neural networks. Prentice Hall, Singapore

    Google Scholar 

  • Price RK, Spitznagel EL, Downey TJ, Meyer DJ, Risk NK, El-Ghazzawy OG (2000) Applying artificial neural network models to clinical decision making. Psychol Assess 12:40–51

    Article  Google Scholar 

  • Quinlan JR (1993) C4.5: programs for machine learning. Morgan Kaufmann Publishers, San Mateo

    Google Scholar 

  • Quinlan JR (1996) Improved use of continuous attributes in C4.5. J Artif Intell Res 4:77–90

    Google Scholar 

  • Reiss AJ (1951) The accuracy, efficiency, and validity of a prediction instrument. Am J Sociol 61:552–561

    Article  Google Scholar 

  • Rice ME, Harris GT (1995a) Violent recidivism: assessing predictive validity. J Consult Clin Psych 63:737–748

    Article  Google Scholar 

  • Rice ME, Harris GT (1995b) Comparing effect sizes in follow-up studies: ROC area, Cohen’s d and r. Law Human Behav 29:615–620

    Article  Google Scholar 

  • Ripley BD (1996) Pattern recognition and neural networks. Cambridge University Press, Cambridge

    Google Scholar 

  • Rosenfeld B, Harmon R (2002) Factors associated with violence in stalking and obsessional harassment cases. Crim Justice Behav 29:671–691

    Article  Google Scholar 

  • Rosenfeld B, Lewis C (2005) Assessing violence risk in stalking cases: a regression tree approach. Law Human Behav 29:343–357

    Article  Google Scholar 

  • Rumelhart DE, McClelland J (1986) Parallel distributed processing, vol 1. MIT Press, Cambridge, MA

    Google Scholar 

  • Shepherd AJ (1997) Second-order methods for neural networks. Springer, New York

    Book  Google Scholar 

  • Silver E, Chow-Martin L (2002) A multiple-models approach to assessing recidivism risk: implications for judicial decision making. Crim Justice Behav 29:538–568

    Article  Google Scholar 

  • Silver E, Smith WR, Banks S (2000) Constructing actuarial devices for predicting recidivism: a comparison of methods. Crim Justice Behav 27:733–764

    Article  Google Scholar 

  • Sjöstedt G, Grann M (2002) Risk assessment: what is being predicted by actuarial “prediction instruments”? Int J Forensic Ment Health 1:179–183

    Google Scholar 

  • Smith WR (1996) The effects of base rate and cutoff point choice on commonly used measures of association and accuracy in recidivism research. J Quant Criminol 12:83–111

    Article  Google Scholar 

  • Smith WR, Smith DR (1998) The consequences of error: recidivism prediction and civil-libertarian ratios. J Crim Just 26:481–502

    Article  Google Scholar 

  • Stalens LJ, Yarnold PR, Seng M, Olson DE, Repp M (2004) Identifying three types of violent offenders and predictinv violent recidivism while on probation: A classification tree analysis. Law Human Behav 28:253–271

    Article  Google Scholar 

  • Starzomska M (2003) Use of artificial neural networks in clinical psychology and psychiatry. Psychiat Polska 37:349–357

    Google Scholar 

  • StatSoft (2008) Data mining, predictive analytics, statistics, StatSoft electronic textbook. http://www.statsoft.com/textbook/

  • Steadman HJ, Monahan J (1994) Toward a rejuvenation of risk assessment research. In: Monahan J, Steadman HJ (eds) Violence and mental disorder. University of Chicago Press, Chicago, pp 10–16

    Google Scholar 

  • Steadman HJ, Mulvey E, Monahan J, Robbins P, Appelbaum P, Grisso T (1998) Violence by people discharged from acute psychiatric inpatient facilities and by others in the same neighborhoods. Arch Gen Psychiat 55:393–401

    Article  Google Scholar 

  • Steadman HJ, Silver E, Monahan J, Appelbaum PS, Robbins PC, Mulvey EP (2000) A classification tree approach to the development of actuarial violence risk assessment tools. Law Human Behav 24:83–100

    Article  Google Scholar 

  • Tam KY, Kiang MY (1992) Managerial applications of neural networks: the case of bank failure predictions. Manage Sci 20:879–888

    Google Scholar 

  • Thomas S, Leese M (2003) A green-fingered approach can improve the clinical utility of violence risk assessment tools. Crim Behav Ment Health 13:153–158

    Article  Google Scholar 

  • Thomas S, Leesea M, Walsh E, McCrone P, Moran P, Burns T (2005) A comparison of statistical models in predicting violence in psychotic illness. Compr Psychiat 46:296–303

    Article  Google Scholar 

  • Trujillano J, Sarria-Santamera A, Esquerda A, Badia M, Palma M, March J (2008) Approach to the methodology of classification and regression trees. Gaceta Sanitaria/SESPAS 22:65–72

    Article  Google Scholar 

  • Tu JV (1996) Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. J Clin Epidemiol 9:1225–1231

    Article  Google Scholar 

  • UK700 Group (1999) Predictors of quality of life in people with severe mental illness. Study methodology with baseline analysis in the UK700 trial. Brit J Psychiat 175:426–432

    Article  Google Scholar 

  • Vogel V, Ruiter C, Hildebrand M, Bos B, Ven P (2004) Type of discharge and risk of recidivism measured by the HCR-20: a retrospective study in a Dutch sample of treated forensic psychiatric patients. Int J Forensic Ment Health 3:149–165

    Google Scholar 

  • Webster CD, Douglas KS, Eaves D, Hart S (1997) HCR-20: assessing risk for violence (version 2). Simon Fraser University, Vancouver, Canada

    Google Scholar 

  • Webster CD, Muller-Isberner R, Frannson G (2002) Violence risk assessment: using structured clinical guidelines professionally. Int J Ment Health 2:185–193

    Google Scholar 

  • Yang M, Liu YY, Coid JW (2010) Applying neural networks and classification tree models to the classification of serious offenders and the prediction of recidivism. Research Summary, Ministry of Justice, UK. Available online at www.justice.gov.uk/publications/research.htm

  • Yarnold PR (1996) Discriminating geriatric and nongeriatric patients using functional status information: an example of classification tree analysis via UniODA. Educ Psychol Meas 56:656–667

    Article  Google Scholar 

Download references

Acknowledgments

The project was funded by Ministry of Justice (England and Wales) and a grant from China Scholarship Council. Professor Min Yang and Professor Jeremy Coid were funded from the National Institute of Health Research Programme Grant (RP-PG-0407-10500). Malcolm Ramsay works for the Ministry of Justice. His contribution here is made in a personal capacity.

Conflict of interest

None.

Author information

Authors and Affiliations

  1. Department of Health Statistics, School of Public Health, Sichuan University, 610041, Chengdu, China

    Yuan Y. Liu, Min Yang & Xiao S. Li

  2. Division of Psychiatry, School for Community Health Sciences, University of Nottingham, 9 Triumph Road, Nottingham, NG7 2TU, UK

    Min Yang

  3. Partnerships and Health Strategy Unit, Ministry of Justice, 102 Petty France (9th floor), London, SW1H 9AJ, UK

    Malcolm Ramsay

  4. Forensic Psychiatry Research Unit, Queen Mary University of London, William Harvey House, 61 St. Bartholomew’s Close, London, EC1A 7BE, UK

    Jeremy W. Coid

Authors
  1. Yuan Y. Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Min Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Malcolm Ramsay
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiao S. Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jeremy W. Coid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Min Yang or Xiao S. Li.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, Y.Y., Yang, M., Ramsay, M. et al. A Comparison of Logistic Regression, Classification and Regression Tree, and Neural Networks Models in Predicting Violent Re-Offending. J Quant Criminol 27, 547–573 (2011). https://doi.org/10.1007/s10940-011-9137-7

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10940-011-9137-7

Keywords

Search

Navigation