Nothing Special   »   [go: up one dir, main page]

skip to main content
10.1145/3292500.3330845acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article
Public Access

Learning Interpretable Metric between Graphs: Convex Formulation and Computation with Graph Mining

Published: 25 July 2019 Publication History

Abstract

Graph is a standard approach to modeling structured data. Although many machine learning methods depend on the metric of the input objects, defining an appropriate distance function on graph is still a controversial issue. We propose a novel supervised metric learning method for a subgraph-based distance, called interpretable graph metric learning (IGML). IGML optimizes the distance function in such a way that a small number of important subgraphs can be adaptively selected. This optimization is computationally intractable with naive application of existing optimization algorithms. We construct a graph mining based efficient algorithm to deal with this computational difficulty. Important advantages of our method are 1) guarantee of the optimality from the convex formulation, and 2) high interpretability of results. To our knowledge, none of the existing studies provide an interpretable subgraph-based metric in a supervised manner. In our experiments, we empirically verify superior or comparable prediction performance of IGML to other existing graph classification methods which do not have clear interpretability. Further, we demonstrate usefulness of IGML through some illustrative examples of extracted subgraphs and an example of data analysis on the learned metric space.

References

[1]
B. Adhikari, Y. Zhang, N. Ramakrishnan, and B. A. Prakash. 2018. Sub2Vec: Feature Learning for Subgraphs. In PAKDD. Springer, 170--182.
[2]
J. Atwood and D. Towsley. 2016. Diffusion-convolutional neural networks. In Advances in NIPS. 1993--2001.
[3]
A. Bellet, A. Habrard, and M. Sebban. 2012. Good edit similarity learning by loss minimization. Machine Learning, Vol. 89, 1--2 (2012), 5--35.
[4]
K. M. Borgwardt and H.-P. Kriegel. 2005. Shortest-path kernels on graphs. In Proc. of the 5th ICDM. IEEE, 8--pp.
[5]
H. Cheng, X. Yan, J. Han, and P. S. Yu. 2008. Direct Discriminative Pattern Mining for Effective Classification. In Proc. of 24th ICDE. 169--178.
[6]
F. Costa and K. D. Grave. 2010. Fast neighborhood subgraph pairwise distance kernel. In Proc. of the 27th ICML. Omnipress, 255--262.
[7]
J. V. Davis, B. Kulis, P. Jain, S. Sra, and I. S. Dhillon. 2007. Information-theoretic metric learning. In Proc. of the 24th ICML. ACM, 209--216.
[8]
D. K. Duvenaud, D. Maclaurin, J. Iparraguirre, R. Bombarell, T. Hirzel, A. Aspuru-Guzik, and R. P. Adams. 2015. Convolutional Networks on Graphs for Learning Molecular Fingerprints. In Advances in NIPS. Curran Associates, Inc., 2224--2232.
[9]
R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. 2008. LIBLINEAR: A Library for Large Linear Classification. JMLR, Vol. 9 (2008), 1871--1874.
[10]
A. Feragen, N. Kasenburg, J. Petersen, M. de Bruijne, and K. Borgwardt. 2013. Scalable kernels for graphs with continuous attributes. In Advances in NIPS. 216--224.
[11]
J. Friedman, T. Hastie, H. Höfling, and R. Tibshirani. 2007. Pathwise coordinate optimization. The Annals of Applied Statistics, Vol. 1, 2 (12 2007), 302--332.
[12]
T. G"artner, P. Flach, and S. Wrobel. 2003. On graph kernels: Hardness results and efficient alternatives. In Learning Theory and Kernel Machines. Springer, 129--143.
[13]
L. E. Ghaoui, V. Viallon, and T. Rabbani. 2010. Safe feature elimination for the lasso and sparse supervised learning problems. arXiv:1009.4219 (2010).
[14]
K. Kersting, N. M. Kriege, C. Morris, P. Mutzel, and M. Neumann. 2016. Benchmark Data Sets for Graph Kernels. http://graphkernels.cs.tu-dortmund.de.
[15]
R. Kondor and K. M. Borgwardt. 2008. The skew spectrum of graphs. In Proc. of the 25th ICML. ACM, 496--503.
[16]
R. Kondor and H. Pan. 2016. The multiscale laplacian graph kernel. In Advances in NIPS. 2990--2998.
[17]
R. Kondor, N. Shervashidze, and K. M. Borgwardt. 2009. The graphlet spectrum. In Proc. of the 26th ICML. ACM, 529--536.
[18]
N. Kriege and P. Mutzel. 2012. Subgraph matching kernels for attributed graphs. arXiv preprint arXiv:1206.6483 (2012).
[19]
J. B. Lee, R. Rossi, and X. Kong. 2018. Graph classification using structural attention. In Proc. of the 24th KDD. ACM, 1666--1674.
[20]
D. Li and Y. Tian. 2018. Survey and experimental study on metric learning methods. Neural Networks, Vol. 105 (2018), 447 -- 462.
[21]
C. Morris, N. M. Kriege, K. Kersting, and P. Mutzel. 2016. Faster kernels for graphs with continuous attributes via hashing. In Data Mining (ICDM), 2016 IEEE 16th International Conference on. IEEE, 1095--1100.
[22]
M. L. Morvan and J.-P. Vert. 2018. WHInter: A Working set algorithm for High-dimensional sparse second order Interaction models. In Proc. of the 35th ICML, Vol. 80. PMLR, 3635--3644.
[23]
K. Nakagawa, S. Suzumura, M. Karasuyama, K. Tsuda, and I. Takeuchi. 2016. Safe pattern pruning: An efficient approach for predictive pattern mining. In Proc. of the 22nd KDD. ACM, 1785--1794.
[24]
A. Narayanan, M. Chandramohan, R. Venkatesan, L. Chen, Y. Liu, and S. Jaiswal. 2017. graph2vec: Learning Distributed Representations of Graphs. In Proc. of 13th MLGWorkshop .
[25]
M. Neuhaus and H. Bunke. 2007. Automatic learning of cost functions for graph edit distance. Information Sciences, Vol. 177, 1 (2007), 239--247.
[26]
P. C. Nguyen, K. Ohara, A. Mogi, H. Motoda, and T. Washio. 2006. Constructing decision trees for graph-structured data by chunkingless graph-based induction. In PAKDD. Springer, 390--399.
[27]
M. Niepert, M. Ahmed, and K. Kutzkov. 2016. Learning convolutional neural networks for graphs. In Proc. of the 33rd ICML. 2014--2023.
[28]
P. K. Novak, N. Lavravc, and G. I. Webb. 2009. Supervised Descriptive Rule Discovery: A Unifying Survey of Contrast Set, Emerging Pattern and Subgroup Mining. JRML, Vol. 10 (2009), 377--403.
[29]
F. Orsini, P. Frasconi, and L. De Raedt. 2015. Graph invariant kernels. In Proc. of the 24th IJCAI. 3756--3762.
[30]
H. Saigo, S. Nowozin, T. Kadowaki, T. Kudo, and K. Tsuda. 2009. gBoost: a mathematical programming approach to graph classification and regression. Machine Learning, Vol. 75, 1 (2009), 69--89.
[31]
N. Shervashidze and K. M. Borgwardt. 2009. Fast subtree kernels on graphs. In Advances in NIPS. 1660--1668.
[32]
N. Shervashidze, P. Schweitzer, E. J. v. Leeuwen, K. Mehlhorn, and K. M. Borgwardt. 2011. Weisfeiler-lehman graph kernels. JMLR, Vol. 12, Sep (2011), 2539--2561.
[33]
N. Shervashidze, S. Vishwanathan, T. Petri, K. Mehlhorn, and K. Borgwardt. 2009. Efficient graphlet kernels for large graph comparison. In AIStats. 488--495.
[34]
M. Simonovsky and N. Komodakis. 2017. Dynamic edge-conditioned filters in convolutional neural networks on graphs. In Proc. CVPR .
[35]
Y. Su, F. Han, R. E. Harang, and X. Yan. 2016. A fast kernel for attributed graphs. In Proc. of the 2016 SDM. SIAM, 486--494.
[36]
M. Sugiyama and K. Borgwardt. 2015. Halting in random walk kernels. In Advances in NIPS. 1639--1647.
[37]
M. Thoma, H. Cheng, A. Gretton, J. Han, H.-P. Kriegel, A. Smola, L. Song, P. S. Yu, X. Yan, and K. M. Borgwardt. 2010. Discriminative frequent subgraph mining with optimality guarantees. Stat. Anal. Data Min., Vol. 3, 5 (2010), 302--318.
[38]
A. J.-P. Tixier, G. Nikolentzos, P. Meladianos, and M. Vazirgiannis. 2018. Graph Classification with 2D Convolutional Neural Networks. (2018).
[39]
S. Verma and Z.-L. Zhang. 2017. Hunt For The Unique, Stable, Sparse And Fast Feature Learning On Graphs. In Advances in NIPS. 88--98.
[40]
S. V. N. Vishwanathan, N. N. Schraudolph, R. Kondor, and K. M. Borgwardt. 2010. Graph kernels. JMLR, Vol. 11, Apr (2010), 1201--1242.
[41]
K. Q. Weinberger and L. K. Saul. 2009. Distance metric learning for large margin nearest neighbor classification. JMLR, Vol. 10, Feb (2009), 207--244.
[42]
X. Yan and J. Han. 2002. gspan: Graph-based substructure pattern mining. In Proc. of the 2nd ICDM. IEEE, 721--724.
[43]
P. Yanardag and S. Vishwanathan. 2015. Deep graph kernels. In Proc. of the 21th KDD. ACM, 1365--1374.
[44]
T. Yoshida, I. Takeuchi, and M. Karasuyama. 2018. Safe Triplet Screening for Distance Metric Learning. In Proc. of the 24th KDD. 2653--2662.
[45]
M. Zhang, Z. Cui, M. Neumann, and Y. Chen. 2018. An end-to-end deep learning architecture for graph classification. In Proc. of 32nd AAAI .
[46]
Z. Zhang, M. Wang, Y. Xiang, Y. Huang, and A. Nehorai. 2018. RetGK: Graph Kernels based on Return Probabilities of Random Walks. In Advances in NIPS. 3968--3978.

Cited By

View all
  • (2023)Multilevel Graph Matching Networks for Deep Graph Similarity LearningIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2021.310223434:2(799-813)Online publication date: Feb-2023
  • (2023)Evidence based Employee Analytics using Taxonomy Enabled Knowledge Graph2023 2nd Edition of IEEE Delhi Section Flagship Conference (DELCON)10.1109/DELCON57910.2023.10127301(1-5)Online publication date: 24-Feb-2023
  • (2022)Meta-Learned Metrics over Multi-Evolution Temporal GraphsProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3534678.3539313(367-377)Online publication date: 14-Aug-2022
  • Show More Cited By

Index Terms

  1. Learning Interpretable Metric between Graphs: Convex Formulation and Computation with Graph Mining

      Recommendations

      Comments

      Please enable JavaScript to view thecomments powered by Disqus.

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      KDD '19: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
      July 2019
      3305 pages
      ISBN:9781450362016
      DOI:10.1145/3292500
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 25 July 2019

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. graph mining
      2. metric learning
      3. structured data

      Qualifiers

      • Research-article

      Funding Sources

      • MEXT KAKENHI
      • PRESTO
      • JST CREST
      • Collaborative Research Program of Institute for Chemical Research Kyoto University
      • MI2I project of the Support Program for Starting Up Innovation Hub from JST
      • RIKEN Center for AIP

      Conference

      KDD '19
      Sponsor:

      Acceptance Rates

      KDD '19 Paper Acceptance Rate 110 of 1,200 submissions, 9%;
      Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

      Upcoming Conference

      KDD '25

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)90
      • Downloads (Last 6 weeks)11
      Reflects downloads up to 16 Dec 2024

      Other Metrics

      Citations

      Cited By

      View all
      • (2023)Multilevel Graph Matching Networks for Deep Graph Similarity LearningIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2021.310223434:2(799-813)Online publication date: Feb-2023
      • (2023)Evidence based Employee Analytics using Taxonomy Enabled Knowledge Graph2023 2nd Edition of IEEE Delhi Section Flagship Conference (DELCON)10.1109/DELCON57910.2023.10127301(1-5)Online publication date: 24-Feb-2023
      • (2022)Meta-Learned Metrics over Multi-Evolution Temporal GraphsProceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3534678.3539313(367-377)Online publication date: 14-Aug-2022
      • (2021)Slow learning and fast inferenceProceedings of the 35th International Conference on Neural Information Processing Systems10.5555/3540261.3541342(14110-14121)Online publication date: 6-Dec-2021
      • (2021)Multi-objective Explanations of GNN Predictions2021 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM51629.2021.00052(409-418)Online publication date: Dec-2021
      • (2021)Distance metric learning for graph structured dataMachine Learning10.1007/s10994-021-06009-3110:7(1765-1811)Online publication date: 16-Jun-2021
      • (2021)Sequential recommendation with metric models based on frequent sequencesData Mining and Knowledge Discovery10.1007/s10618-021-00744-wOnline publication date: 12-Mar-2021
      • (2021)Deep graph similarity learning: a surveyData Mining and Knowledge Discovery10.1007/s10618-020-00733-535:3(688-725)Online publication date: 24-Mar-2021
      • (2020)Exchangeable Deep Neural Networks for Set-to-Set Matching and LearningComputer Vision – ECCV 202010.1007/978-3-030-58520-4_37(626-646)Online publication date: 19-Nov-2020

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Login options

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media