Abstract
We develop a metric ψ, based upon the RAND index, for the comparison and evaluation of dimensionality reduction techniques. This metric is designed to test the preservation of neighborhood structure in derived lower dimensional configurations. We use a customer information data set to show how ψ can be used to compare dimensionality reduction methods, tune method parameters, and choose solutions when methods have a local optimum problem. We show that ψ is highly negatively correlated with an alienation coefficient K that is designed to test the recovery of relative distances. In general a method with a good value of ψ also has a good value of K. However the monotonic regression used by Nonmetric MDS produces solutions with good values of ψ, but poor values of K.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Akkucuk, U.: Nonlinear mapping: Approaches based on optimizing an index of continuity and applying classical metric MDS on revised distances. PhD dissertation: Rutgers University (2004)
Akkucuk, U., Carroll, J.D.: PARAMAP vs. ISOMAP: A Comparison of Two Nonlinear Mapping Algorithms. Journal of Classification (forthcoming, 2007)
Akkucuk, U., Carroll, J.D.: Parametric Mapping (PARAMAP): An Approach to Nonlinear Mapping, 2006. In: Proceedings of the American Statistical Association, Section on Statistical Computing [CD-ROM], Alexandria, VA: American Statistical Association, pp. 1980–1986 (2006)
Andrews, R.L., Manrai, A.K.: MDS Maps for Product Attributes and Market Response: An Application to Scanner Panel Data. Marketing Science 18(4), 584–604 (1999)
Bijmolt, T.H.A., Wedel, M.: A comparison of Multidimensional Scaling Methods for Perceptual Mapping. Journal of Marketing Research 36(2), 277–285 (1999)
Borg, I., Leutner, D.: Measuring the Similarity Between MDS Configurations. Multivariate Behavioral Research 20, 325–334 (1985)
Buja, A., Swayne, D.F.: Visualization Methodology for Multidimensional Scaling. Journal of Classification 19, 7–43 (2004)
Carroll, J.D., Arabie, P.: Multidimensional scaling. In: Birnbaum, M.H. (ed.) Handbook of Perception and Cognition. Measurement, Judgment and Decision Making, vol. 3, pp. 179–250. Academic Press, San Diego, CA (1998)
Carroll, J.D., Green, P.E., Schaffer, C.M.: Interpoint Distance Comparisons in Correspondence Analysis. Journal of Marketing Research 23(3), 271–280 (1986)
Carroll, J.D., Green, P.E.: Psychometric Methods in Marketing Research: Part II. Multidimensional Scaling, Journal of Marketing Research 34(2), 193–204 (1997)
Chen, L., Buja, A.: Local Multidimensional Scaling for Nonlinear Dimension Reduction, Graph Layout, and Proximity Analysis, Working Paper, University of Pennsylvania (2006)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society B, 39, 1–38 (1977)
DeSarbo, W.S., Kim, J., Choi, S.C., Spaulding, M.: A Gravity-Based Multidimensional Scaling Model for Deriving Spatial Structures Underlying Consumer Preference/Choice Judgements. Journal of Consumer Reseach 29(1), 91–100 (2002)
DeSarbo, W.S., Hoffman, D.L.: Constructing MDS Joint Spaces from Binary Choice Data: A Multidimensional Unfolding Threshold Model for Marketing Research. Journal of Marketing Research 26(1), 40–54 (1987)
DeSarbo, W.S., Manrai, A.K.: A new Multidimensional Scaling Methodology for the Analysis of Asymmetric Proximity Data in Marketing Research 11(1), 1–20 (1992)
Fodor, I.K.: A Survey of Dimension Reduction Techniques. LLNL technical report (2002)
Green, P.E.: Marketing Applications of MDS: Assessment and Outlook. Journal of Marketing 39, 24–31 (1975)
Ham, J., Lee, D.D., Mika, S., Schölkopf, B.: A kernel view of the dimensionality reduction of manifolds. In: Greiner, R., Schuurmans, D. (eds.) Proceedings of the Twenty-First International Conference on Machine Learning, pp. 369–376 (2006)
Hubert, L., Arabie, P.: Comparing Partitions. Journal of Classification 2, 193–218 (1985)
Kohonen, T.: Self-Organizing Map. Springer, New York (2001)
Kruskal, J.B.: Multidimensional scaling for optimizing a goodness of fit metric to a nonmetric hypothesis. Psychometrika 29, 1–27 (1964a)
Kruskal, J.B.: Nonmetric Multidimensional scaling: A numerical method. Psychometrika 29, 115–129 (1964b)
Lafon, S., Lee, A.B.: Diffusion Maps and Coarse-Graining: A Unified Framework for Dimensionality Reduction. Graph Partitioning, and Data Set Paramaterization 28(9), 1393–1403 (2006)
Law, M.H.C., Jain, A.K.: Incremental Nonlinear Dimensionality Reduction by Manifold Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence 28(3), 377–391 (2006)
Levina, E.M., Bickel, P.J.: Maximum likelihood estimation of intrinsic dimension. In: Advances in Neural Information Processing Systems 17, MIT Press, Boston (2005)
Moore, W.L., Winer, R.S.: A Panel-Data Based Method for Merging Joint Space and Market Response Function Estimation. Marketing Science 6(1), 25–42 (1987)
Murtagh, F.: Correspondence Analysis and Data Coding with R and Java. Chapman and Hall/CRC Press, London (2005)
Neslin, S.A., Gupta, S., Kamakura, W., Lu, J., Mason, C.H.: Defection Detection: Measuring and Understanding the Predictive Accuracy of Customer Churn Models. Journal of Marketing Research 43(2), 204–211 (2006)
Pettis, K., Bailey, T., Jain, A.K., Dubes, R.: An intrinsic dimensionality estimator from near-neighbor information. IEEE Transactions on Pattern Analysis and Machine Intelligence 1(1), 25–36 (1979)
Rand, W.M.: Objective Criteria for the Evaluation of Clustering methods. Journal of the American Statistical Association 66, 846–850 (1971)
Roweis, S.T., Saul, L.K.: Nonlinear Dimensionality Reduction by Locally Linear Embedding. Science 290(5500), 2323–2326 (2000)
Sha, F., Saul, L.K.: Analysis and Extension of Spectral Methods for Nonlinear Dimensionality Reduction. In: Proceedings of the 22nd International Conference on Machine Learning, Bonn, Germany (2005)
Shepard, R.N.: The analysis of proximities: Multidimensional scaling with an unknown distance function, Parts I and II. Psychometrika. 27, pp. 125–140, pp. 219–246 (1962)
Shepard, R.N., Carroll, J.D.: Parametric representation of nonlinear data structures. In: Krishnaiah, P.R. (ed.) Multivariate Analysis, pp. 561–592. Academic Press, New York (1966)
Steen, J.E.M., Trijup, H.C.M.V., Ten Berge, J.M.F.: Perceptual Mapping Based on Idiosyncratic Sets of Attributes. Journal of Marketing Research 31(1), 15–27 (1994)
Tenenbaum, J.B., de Silva, V., Langford, J.C.: A Global Geometric Framework for Nonlinear Dimensionality Reduction. Science 290(5500), 2319–2323 (2000)
Torgerson, W.S.: Multidimensional Scaling, I: theory and method. Psychometrika 17, 401–419 (1952)
Torgerson, W.S.: Theory and Methods of Scaling, vol. 32. Wiley, New York (1958)
Weinberger, K.Q., Saul, L.K.: Unsupervised Learning of Image Manifolds by Semidefinite Programming. International Journal of Computer Vision 70(1), 77–90 (2006)
Young, G., Householder, A.A.: Discussion of a Set of Points in Terms of their Mutual Distances. Psychometrika 3, 19–22 (1938)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
France, S., Carroll, D. (2007). Development of an Agreement Metric Based Upon the RAND Index for the Evaluation of Dimensionality Reduction Techniques, with Applications to Mapping Customer Data. In: Perner, P. (eds) Machine Learning and Data Mining in Pattern Recognition. MLDM 2007. Lecture Notes in Computer Science(), vol 4571. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73499-4_38
Download citation
DOI: https://doi.org/10.1007/978-3-540-73499-4_38
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73498-7
Online ISBN: 978-3-540-73499-4
eBook Packages: Computer ScienceComputer Science (R0)