Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Model-based clustering and classification with non-normal mixture distributions

  • Published:
Statistical Methods & Applications Aims and scope Submit manuscript

Abstract

Non-normal mixture distributions have received increasing attention in recent years. Finite mixtures of multivariate skew-symmetric distributions, in particular, the skew normal and skew \(t\)-mixture models, are emerging as promising extensions to the traditional normal and \(t\)-mixture models. Most of these parametric families of skew distributions are closely related, and can be classified into four forms under a recently proposed scheme, namely, the restricted, unrestricted, extended, and generalised forms. In this paper, we consider some of these existing proposals of multivariate non-normal mixture models and illustrate their practical use in several real applications. We first discuss the characterizations along with a brief account of some distributions belonging to the above classification scheme, then references for software implementation of EM-type algorithms for the estimation of the model parameters are given. We then compare the relative performance of restricted and unrestricted skew mixture models in clustering, discriminant analysis, and density estimation on six real datasets from flow cytometry, finance, and image analysis. We also compare the performance of mixtures of skew normal and \(t\)-component distributions with other non-normal component distributions, including mixtures with multivariate normal-inverse-Gaussian distributions, shifted asymmetric Laplace distributions and generalized hyperbolic distributions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  • Aghaeepour N, Finak G, Consortium TF, Consortium TD, Hoos H, Mosmann TR, Brinkman R, Gottardo R, Scheuermann RH (2013) Critical assessment of automated flow cytometry data analysis techniques. Nat Methods 10:228–238

    Article  Google Scholar 

  • Altman EI (1968) Financial ratios, discriminant analysis and the prediction of corporate bankruptcy. J Finance 23(4):589–609

    Article  Google Scholar 

  • Arellano-Valle RB, Azzalini A (2006) On the unification of families of skew-normal distributions. Scand J Stat 33:561–574

    Article  MathSciNet  MATH  Google Scholar 

  • Arellano-Valle RB, Genton MG (2005) On fundamental skew distribtuions. J Multivar Anal 96:93–116

    Article  MathSciNet  MATH  Google Scholar 

  • Arellano-Valle RB, Genton MG (2010a) Multivariate extended skew-\(t\) distributions and related families. Metron—special issue on ‘Skew-symmetric and flexible distributions’ 68:201–234

    MathSciNet  Google Scholar 

  • Arellano-Valle RB, Genton MG (2010b) Multivariate unified skew-elliptical distributions. Chil J Stat 1: 17–33

    MathSciNet  MATH  Google Scholar 

  • Arellano-Valle RB, del Pino G, Martin ES (2002) Definition and probabilistic properties of skew-distributions. Stat Probab Lett 58(2):111–121

    Article  MATH  Google Scholar 

  • Arellano-Valle RB, Branco MD, Genton MG (2006) A unified view on skewed distributions arising from selections. Can J Stat 34:581–601

    Article  MathSciNet  MATH  Google Scholar 

  • Arnold BC, Beaver RJ, Meeker WQ (1993) The nontruncated marginal of a truncated bivariate normal distribution. Psychometrika 58:471–488

    Article  MathSciNet  MATH  Google Scholar 

  • Azzalini A (1985) A class of distributions which includes the normal ones. Scand J Stat 12:171–178

    MathSciNet  MATH  Google Scholar 

  • Azzalini A, Capitanio A (1999) Statistical applications of the multivariate skew-normal distribution. J R Stat Soc Ser B 61(3):579–602

    Article  MathSciNet  MATH  Google Scholar 

  • Azzalini A, Capitanio A (2003) Distribution generated by perturbation of symmetry with emphasis on a multivariate skew t distribution. J R Stat Soc Ser B 65(2):367–389

    Article  MathSciNet  MATH  Google Scholar 

  • Azzalini A, Dalla Valle A (1996) The multivariate skew-normal distribution. Biometrika 83(4):715–726

    Article  MathSciNet  MATH  Google Scholar 

  • Banfield JD, Raftery AE (1993) Model-based Gaussian and non-Gaussian clustering. Biometrics 49: 803–821

    Google Scholar 

  • Barndorff-Nielsen OE (1977) Exponentially decreasing distributions from the logarithm of of particle size. Proc R Soc Lond A353:401–419

    Article  Google Scholar 

  • Basso RM, Lachos VH, Cabral CRB, Ghosh P (2010) Robust mixture modeling based on scale mixtures of skew-normal distributions. Comput Stat Data Anal 54:2926–2941

    Article  MathSciNet  Google Scholar 

  • Böhning D (1999) Computer-assisted analysis of mixtures and applications: meta-analysis, disease mapping and others. Chapman and Hall/CRC Press, London

    Google Scholar 

  • Branco MD, Dey DK (2001) A general class of multivariate skew-elliptical distributions. J Multivar Anal 79:99–113

    Article  MathSciNet  MATH  Google Scholar 

  • Browne RP, McNicholas PD (2013) A mixture of generalized hyperbolic distributions. arXiv:13051036 [statME]

  • Cabral CS, Lachos VH, Prates MO (2012) Multivariate mixture modeling using skew-normal independent distributions. Comput Stat Data Anal 56:126–142

    Article  MathSciNet  MATH  Google Scholar 

  • Calò AG, Montanari A, Viroli C (2013) A hierarchical modeling approach for clustering probability density functions. Comput Stat Data Anal. doi:10.1016/j.csda.2013.04.013

  • Charytanowicz M, Niewczas J, Kulczycki P, Kowalski P, Lukasik S, Zak S (2010) A complete gradient clustering algorithm for features analysis of x-ray images. In: Pietka E, Kawa J (eds) Information technologies in biomedicine. Springer, Berlin, pp 15–24

    Chapter  Google Scholar 

  • Choi P, Min I (2011) A comparison of conditional and unconditional approaches in value-at-risk estimation. J Jpn Econ Assoc 62:99–115

    MathSciNet  Google Scholar 

  • Christoffersen PF (1998) Evaluating interval forecasts. Int Econ Rev 39:841–862

    Article  MathSciNet  Google Scholar 

  • Contreras-Reyes JE, Arellano-Valle RB (2012) Growth curve based on scale mixtures of skew-normal distributions to model the age-length relationship of cardinalfish (epigonus crassicaudus). arXiv:12125180 [statAP]

  • Cook RD, Weisberg S (1994) An introduction to regression graphics. Wiley, New York

    Book  MATH  Google Scholar 

  • Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Ser B 39:1–38

    MathSciNet  MATH  Google Scholar 

  • Everitt BS, Hand DJ (1981) Finite mixture distributions. Chapman and Hall, London

    Book  MATH  Google Scholar 

  • Fang KT, Kotz S, Ng K (1990) Symmetric multivariate and related distributions. Chapman & Hall, London

    Book  MATH  Google Scholar 

  • Fraley C, Raftery AE (1999) How many clusters? Which clustering methods? Answers via model-based cluster analysis. Comput J 41:578–588

    Article  Google Scholar 

  • Franczak BC, Browne RP, McNicholas PD (2012) Mixtures of shifted asymmetric laplace distributions. arXiv:12071727 [statME]

  • Frühwirth-Schnatter S (2006) Finite mixture and Markov switching models. Springer, New York

    MATH  Google Scholar 

  • Frühwirth-Schnatter S, Pyne S (2010) Bayesian inference for finite mixtures of univariate and multivariate skew-normal and skew-\(t\) distributions. Biostatistics 11:317–336

    Article  Google Scholar 

  • Ganesalingam S, McLachlan GJ (1978) The efficiency of a linear discriminant function based on unclassified initial samples. Biometrika 65:658–662

    Article  MathSciNet  MATH  Google Scholar 

  • González-Farás G, Domínguez-Molinz JA, Gupta AK (2004) Additive properties of skew normal random vectors. J Stat Plan Inference 126:521–534

    Article  Google Scholar 

  • Gupta AK (2003) Multivariate skew-\(t\) distribution. Statistics 37:359–363

    Article  MathSciNet  MATH  Google Scholar 

  • Gupta AK, González-Faríaz G, Domínguez-Molina JA (2004) A multivariate skew normal distribution. J Multivar Anal 89:181–190

    Google Scholar 

  • Hubert L, Arabie P (1985) Comparing partitions. J Classif 2:193–218

    Article  Google Scholar 

  • Jones PN, McLachlan GJ (1989) Modelling mass-size particle data by finite mixtures. Commun Stat Theory Methods 18:2629–2646

    Article  MathSciNet  MATH  Google Scholar 

  • Jordan MI, Jacobs RA (1992) Hierarchies of adaptive experts. In: Moody J, Hanson S, Lippmann R (eds) Advances in neural information processing systems 4. Morgan Kaufmann, California, pp 985–993

    Google Scholar 

  • Karlis D, Santourian A (2009) Model-based clustering with non-elliptically contoured distributions. Stat Comput 19:73–83

    Article  MathSciNet  Google Scholar 

  • Karlis D, Xekalaki E (2003) Choosing initial values for the EM algorithm for finite mixtures. Comput Stat Data Anal 41:577–590

    Article  MathSciNet  MATH  Google Scholar 

  • Kotz S, Kozubowski TJ, Podgórski K (2001) The Laplace distribution and generalizations: a revisit with applications to communications, economics, engineering, and finance. Birkhauser, Boston

    Book  Google Scholar 

  • Kupiec P (1995) Techniques for verifying the accuracy of risk management models. J Deriv 3:73–84

    Article  Google Scholar 

  • Lachos VH, Ghosh P, Arellano-Valle RB (2010) Likelihood based inference for skew normal independent linear mixed models. Statistica Sinica 20:303–322

    MathSciNet  MATH  Google Scholar 

  • Lee S, McLachlan GJ (2011) On the fitting of mixtures of multivariate skew \(t\)-distributions via the EM algorithm. arXiv:11094706 [statME]

  • Lee S, McLachlan GJ (2013a) Finite mixtures of multivariate skew \(t\)-distributions: some recent and new results. Stat Comput. doi:10.1007/s11222-012-9362-4

  • Lee SX, McLachlan GJ (2013b) EMMIX-uskew: an R package for fitting mixtures of multivariate skew \(t\)-distributions via the EM algorithm. J Stat Softw. Preprint arXiv:1211.5290

  • Lee SX, McLachlan GJ (2013c) On mixtures of skew-normal and skew \(t\)-distributions. Adv Data Anal Classif. doi:10.1007/s11634-013-0132-8

  • Lin TI (2009) Maximum likelihood estimation for multivariate skew-normal mixture models. J Multivar Anal 100:257–265

    Article  MATH  Google Scholar 

  • Lin TI (2010) Robust mixture modeling using multivariate skew \(t\) distribution. Stat Comput 20:343–356

    Article  MathSciNet  Google Scholar 

  • Lin TI, Ho HJ, Lee CR (2013) Flexible mixture modelling using the multivariate skew-\(t\)-normal distribution. Stat Comput. doi:10.1007/s11222-013-9386-4

  • Lindsay BG (1995) Mixture models: theory, geometry, and applications. In: NSF-CBMS regional conference series in probability and statistics, vol 5, Institute of Mathematical Statistics and the American Statistical Association, Alexandria, VA

  • Liseo B, Loperfido N (2003) A Bayesian interpretation of the multivariate skew-normal distribution. Stat Probab Lett 61:395–401

    Article  MathSciNet  MATH  Google Scholar 

  • Lo K, Brinkman RR, Gottardo R (2008) Automated gating of flow cytometry data via robust model-based clustering. Cytom Part A 73:312–332

    Google Scholar 

  • Lo K, Hahne F, Brinkman RR, Gottardo R (2009) Flowclust: a bioconductor package for automated gating of flow cytometry data. BMC Bioinform 10:145

    Article  Google Scholar 

  • Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proc Int Conf Comput Vis 2:416–423

    Google Scholar 

  • McLachlan GJ, Basford KE (1988) Mixture models: inference and applications. Marcel Dekker, New York

    MATH  Google Scholar 

  • McLachlan GJ, Krishnan T (2008) The EM algorithm and extensions, 2nd edn. Wiley-Interscience, Hokoben, NJ

    Book  MATH  Google Scholar 

  • McLachlan GJ, Peel D (1998) Robust cluster analysis via mixtures of multivariate \(t\)-distributions. In: Amin A, Dori D, Pudil P, Freeman H (eds) Lecture notes in computer science. Springer, Berlin, pp 658–666

  • McLachlan GJ, Peel D (2000) Finite mixture models. Wiley series in probability and statistics, New York

  • McNeil AJ, Frey R, Embrechts P (2005) Quantitative risk management: concepts, techniques and tools. Princeton University Press, USA

    Google Scholar 

  • Meignen S, Meignen H (2006) On the modeling of small sample distributions with generalized gaussian density in a maximum likelihood framework. IEEE Trans Image Process 15:1647–1652

    Article  MathSciNet  Google Scholar 

  • Meilă M (2005) Comparing clusterings—an axiomatic view. In: In ICML ’05: proceedings of the 22nd international conference on machine learning, ACM Press, pp 577–584

  • Mengersen KL, Robert CP, Titterington DM (2011) Mixtures: estimation and applications. Wiley, NewYork

    Book  Google Scholar 

  • Nadarajah S (2008) Skewed distributions generated by the student’s \(t\) kernel. Monte Carlo Methods Appl 13:289–404

    Article  Google Scholar 

  • Nadarajah S, Kotz S (2003) Skewed distributions generated by the normal kernel. Stat Probab Lett 65: 269–277

    Article  MathSciNet  MATH  Google Scholar 

  • Nguyen TM, Wu QMJ (2013) A nonsymmetric mixture model for unsupervised image segmentation. IEEE Trans Cybern 43:751–765

    Article  Google Scholar 

  • Nikolic R (2010) flowKoh: self-organizing map for flow cytometry data analysis. http://commons.bcit.ca/radina_nikolic/docs/flowKoh_R_Code.zip

  • Prates M, Lachos V, Cabral C (2011) mixsmsn: fitting finite mixture of scale mixture of skew-normal distributions. R package version 0.3-2. http://CRAN.R-project.org/package=mixsmsn

  • Pyne S, Hu X, Wang K, Rossin E, Lin TI, Maier LM, Baecher-Allan C, McLachlan GJ, Tamayo P, Hafler DA, De Jager PL, Mesirow JP (2009a) Automated high-dimensional flow cytometric data analysis. Proc Natl Acad Sci USA 106:8519–8524

    Article  Google Scholar 

  • Pyne S, Hu X, Wang K, Rossin E, Lin TI, Maier LM, Baecher-Allan C, McLachlan GJ, Tamayo P, Hafler DA, De Jager PL, Mesirow JP (2009b) FLAME: flow analysis with automated multivariate estimation. http://www.broadinstitute.org/cancer/software/genepattern/modules/FLAME/published_data

  • Qian Y, Wei C, Lee F, Campbell J, Halliley J, Lee J, Cai J, Kong Y, Sadat E, Thomson E (2010) Elucidation of seventeen human peripheral blood b-cell subsets and quantification of the tetanus response using a density-based method for the automated identification of cell populations in multidimensional flow cytometry data. Cytom Part B 78:S69–S82

    Article  Google Scholar 

  • R Development Team (2011) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org/. ISBN 3-900051-07-0

  • Rand WM (1971) Objective criteria for the evaluation of clustering methods. J Am Stat Assoc 66:846–850

    Article  Google Scholar 

  • Riggi S, Ingrassia S (2013) Modeling high energy cosmic rays mass composition data via mixtures of multivariate skew-\(t\) distributions. arXiv:13011178 [astro-phHE]

  • Rodrigues J (2006) A bayesian inference for the extended skew-normal measurement error model. Brazilian J Probab Stat 20:179–190

    MATH  Google Scholar 

  • Sahu SK, Dey DK, Branco MD (2003) A new class of multivariate skew distributions with applications to Bayesian regression models. Can J Stat 31:129–150

    Article  MathSciNet  MATH  Google Scholar 

  • Soltyk S, Gupta R (2011) Application of the multivariate skew normal mixture model with the EM algorithm to value-at-risk. In: MODSIM 2011—19th International Congress on Modelling and Simulation, Perth, Australia, 12–16 Dec 2011

  • Titterington DM, Smith AFM, Markov UE (1985) Statistical analysis of finite mixture distributions. Wiley, New York

    MATH  Google Scholar 

  • Vrbik I, McNicholas PD (2012) Analytic calculations for the EM algorithm for multivariate skew \(t\)-mixture models. Stat Probab Lett 82:1169–1174

    Article  MathSciNet  MATH  Google Scholar 

  • Wang K, McLachlan GJ, Ng SK, Peel D (2009) EMMIX-skew: EM algorithm for mixture of multivariate skew normal/\(t\) distributions. R package version 1.0-12. http://www.maths.uq.edu.au/~gjm/mix_soft/EMMIX-skew

  • Zhang Y, Brady M, Smith S (2001) Segmentation of brain MR images through a hidden Markov random field model and the expectation maximization algorithm. IEEE Trans Med Imaging 20:45–57

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, University of Queensland, 4072, St Lucia, Brisbane, Australia

    Sharon X. Lee & Geoffrey J. McLachlan

Authors
  1. Sharon X. Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Geoffrey J. McLachlan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Geoffrey J. McLachlan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lee, S.X., McLachlan, G.J. Model-based clustering and classification with non-normal mixture distributions. Stat Methods Appl 22, 427–454 (2013). https://doi.org/10.1007/s10260-013-0237-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10260-013-0237-4

Keywords

Search

Navigation