Nothing Special   »   [go: up one dir, main page]
Skip to main content
Springer Nature Link
Log in

Toward New Evaluation Metrics for Relational Learning

  • Chapter
  • First Online:
Advances in Integrations of Intelligent Methods

Part of the book series: Smart Innovation, Systems and Technologies ((SIST,volume 170))

  • 254 Accesses

  • 1 Citation

Abstract

Statistical relational learning (SRL) is a field of machine learning that enables effective and robust reasoning about relational data structures. Several conventional data mining methods have been adapted for direct application to relational data representation. Probabilistic relational models extend Bayesian networks to a relational data mining context. To use this model, it is first necessary to build it: The structure and parameters of a probabilistic relational model must be set manually or learned from a relational observational dataset. Learning the structure remains the most complicated issue as it is a NP-hard problem. Existing approaches for probabilistic relational models structure learning are inspired from classical methods of learning the structure of Bayesian networks. The evaluation of learning approaches requires testing datasets and evaluation measurements. Processes to randomly generate the model and the data are already established in the relational context. However, metrics to evaluate a probabilistic relational model structure learning algorithm are not yet proposed. In fact, only the classical Recall and Precision have been used as evaluation metrics. In this work, we discuss why these metrics are not appropriate in this context, and we propose an adaptation of them to fit probabilistic relational models representation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Following Maier et al. [42], h is called hop threshold.

References

  1. Heckerman, D., Meek, C., Koller, D.: Probabilistic entity-relationship models, PRMs, and plate models. In: Getoor, L., Taskar, B. (eds.) Introduction to Statistical Relational Learning. MIT Press, Cambridge, MA (2007)

    Google Scholar 

  2. Lavrac, N., Dzeroski, S.: Inductive Logic Programming: Techniques and Applications. Routledge, New York (1993)

    MATH  Google Scholar 

  3. De Raedt, L.: Attribute-value learning versus inductive logic programming: the missing links. In: Proceedings of the Eighth International Conference on Inductive Logic Programming, pp. 1–8 (1998)

    Google Scholar 

  4. Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Francisco (1988)

    MATH  Google Scholar 

  5. Heckerman, D., Chickering, D.M., Meek, C., Rounthwaite, R., Kadiel, C.: Dependency networks for inference, collaborative filtering, and data visualization. J. Mach. Learn. Res. 1, 49–75 (2001)

    MATH  Google Scholar 

  6. Koller, D., Pfeffer, A.: Probabilistic frame-based systems. In: Proceedings of AAAI, pp. 580–587. AAAI Press (1998)

    Google Scholar 

  7. Heckerman, D., Meek, C., Koller, D.: Probabilistic models for relational data. Technical report, Microsoft Research, Redmond, WA (2004)

    Google Scholar 

  8. Kazemi, S.M., Buchman, D., Kersting, K., Natarajan, S., Poole, D.: Relational logistic regression. In: Principles of Knowledge Representation and Reasoning: Proceedings of the Fourteenth International Conference, KR 2014, Vienna, Austria, July 20–24, 2014 (2014)

    Google Scholar 

  9. Kazemi, S.M., Buchman, D., Kersting, K., Natarajan, S., Poole, D.: Relational logistic regression: the directed analog of Markov logic networks. In: Statistical Relational Artificial Intelligence, Papers from the 2014 AAAI Workshop, Québec City, Québec, Canada, July 27, 2014 (2014)

    Google Scholar 

  10. Taskar, B., Abbeel, P., Koller, D.: Discriminative probabilistic models for relational data. In: Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence, pp. 485–492 (2002)

    Google Scholar 

  11. Bach, S.H., Broecheler, M., Huang, B., Getoor, L.: Hinge-loss Markov random fields and probabilistic soft logic. J. Mach. Learn. Res. (JMLR) 18(109), 1–67 (2017)

    MathSciNet  MATH  Google Scholar 

  12. Kimmig, A., Bach, S.H., Broecheler, M., Huang, B., Getoor, L.: A short introduction to probabilistic soft logic. In: NIPS Workshop on Probabilistic Programming: Foundations and Applications (2012)

    Google Scholar 

  13. Neville, J., Jensen, D.: Relational dependency networks. J. Mach. Learn. Res. 8, 653–692 (2007)

    MATH  Google Scholar 

  14. Ben Ishak, M., Leray, P., Ben Amor, N.: Probabilistic relational model benchmark generation: principle and application. Intell. Data Anal. Int. J. pp. 615–635 (2016)

    Article  Google Scholar 

  15. Ben Ishak, M., Leray, P., Ben Amor, N.: A hybrid approach for probabilistic relational models structure learning. In: Proceedings of the 15th International Symposium on Intelligent Data Analysis (IDA 2016), pp. 38–49 (2016)

    Google Scholar 

  16. Friedman, N., Getoor, L., Koller, D., Pfeffer, A.: Learning probabilistic relational models. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 1300–1309 (1999)

    Google Scholar 

  17. Maier, M., Marazopoulou, K., Arbour, D., Jensen, D.: A sound and complete algorithm for learning causal models from relational data. In: Proceedings of the Twenty-Ninth Conference on Uncertainty in Artificial Intelligence, pp. 371–380 (2013)

    Google Scholar 

  18. Pearl, J.: Causality: Models, Reasoning and Inference. MIT Press, Cambridge, MA (2000)

    MATH  Google Scholar 

  19. Verma, T., Pearl, J.: Equivalence and synthesis of causal models. In: Proceedings of the 6th UAI, pp. 220–227 (1990)

    Google Scholar 

  20. Andersson, S., Madigan, D., Perlman, M.D.: A characterization of Markov equivalence classes for acyclic digraphs. Ann. Stat. 25(2), 505–541 (1997)

    Article  MathSciNet  Google Scholar 

  21. Chickering, D.M.: Optimal structure identification with greedy search. J. Mach. Learn. Res. 3, 507–554 (2002)

    MathSciNet  MATH  Google Scholar 

  22. Daly, R., Shen, Q., Aitken, S.: Learning Bayesian networks: approaches and issues. Knowl. Eng. Rev. 26, 99–157 (2011)

    Article  Google Scholar 

  23. de Campos, L.M.: A scoring function for learning Bayesian networks based on mutual information and conditional independence tests. J. Mach. Learn. Res. 7, 2149–2187 (2006)

    MathSciNet  MATH  Google Scholar 

  24. Kullback, S., Leibler, R.: On information and sufficiency. Ann. Math. Stat. 22, 79–86 (1951)

    Article  MathSciNet  Google Scholar 

  25. Fast, A.S.: Learning the structure of Bayesian networks with constraint satisfaction. Ph.D. thesis, University of Massachusetts Amherst (2010)

    Google Scholar 

  26. Tsamardinos, I., Brown, L.E., Aliferis, C.F.: The max-min hill-climbing Bayesian network structure learning algorithm. Mach. Learn. 65, 31–78 (2006)

    Article  Google Scholar 

  27. Pfeffer, A.J.: Probabilistic Reasoning for Complex Systems. Ph.D. thesis, Stanford University (2000)

    Google Scholar 

  28. Pfeffer, A., Koller, D.: Semantics and inference for recursive probability models. In: AAAI/IAAI, pp. 538–544. AAAI Press/The MIT Press (2000)

    Google Scholar 

  29. Gonzales, C., Wuillemin, P.H.: PRM inference using Jaffray & Faÿ’s local conditioning. Theory Decis. 71(1), 33–62 (2011)

    Article  Google Scholar 

  30. Kisynski, J., Poole, D.: Lifted aggregation in directed first-order probabilistic models. In: Proceedings of the 21st International Joint Conference on Artificial Intelligence, pp. 1922–1929. IJCAI’09, Morgan Kaufmann Publishers Inc., San Francisco, CA (2009)

    Google Scholar 

  31. Milch, B., Zettlemoyer, L.S., Kersting, K., Haimes, M., Pack Kaelbling, L.: Lifted probabilistic inference with counting formulas. In: Proceedings of the 23rd Conference on Artificial Intelligence (AAAI) (2008)

    Google Scholar 

  32. Wuillemini, P.H., Torti, L.: Structured probabilistic inference. Int. J. Approx. Reason. 53(7), 946–968 (2012)

    Article  MathSciNet  Google Scholar 

  33. Koller, D., Friedman, N.: Probabilistic Graphical Models. MIT Press, Cambridge, MA (2009)

    MATH  Google Scholar 

  34. Maier, M., Taylor, B., Oktay, H., Jensen, D.: Learning causal models of relational domains. In: Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence, pp. 531–538 (2010)

    Google Scholar 

  35. Torti, L., Wuillemin, P.H., Gonzales, C.: Reinforcing the Object-Oriented aspect of probabilistic relational models. In: Proceedings of the 5th Probabilistic Graphical Models, pp. 273–280 (2010)

    Google Scholar 

  36. El Abri, M., Leray, P., Essoussi, N.: Learning probabilistic relational models with (partially structured) graph databases. In: 14th IEEE/ACS International Conference on Computer Systems and Applications, AICCSA 2017, Hammamet, Tunisia, October 30–Nov. 3, 2017, pp. 256–263 (2017)

    Google Scholar 

  37. El Abri, M., Leray, P., Essoussi, N.: DAPER joint learning from partially structured graph databases. In: Digital Economy. Emerging Technologies and Business Innovation—Third International Conference, ICDEc 2018, Brest, France, May 3–5, 2018, Proceedings, pp. 129–138 (2018)

    Google Scholar 

  38. Lee, S., Honavar, V.: On learning causal models from relational data. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence (AAAI 2016), pp. 3263–3270 (2016)

    Google Scholar 

  39. Marazopoulou, K., Maier, M., Jensen, D.: Learning the structure of causal models with relational and temporal dependence. In: Proceedings of the Thirty-First Conference on Uncertainty in Artificial Intelligence, pp. 572–581 (2015)

    Google Scholar 

  40. Munch, M., Wuillemin, P., Manfredotti, C., Dibie, J., Dervaux, S.: Learning probabilistic relational models using an ontology of transformation processes. In: Panetto, H., et al. (eds.) On the Move to Meaningful Internet Systems. OTM 2017. Lecture Notes in Computer Science, vol. 10574, pp. 198–215. Springer, Cham (2017)

    Chapter  Google Scholar 

  41. Heckerman, D.: A tutorial on learning with Bayesian network. In: Proceedings of the NATO Advanced Study Institute on Learning in graphical models, pp. 301–354. Kluwer Academic Publishers, Norwell, MA (1998)

    Chapter  Google Scholar 

  42. Maier, M., Marazopoulou, K., Jensen, D.: Reasoning about independence in probabilistic models of relational data. CoRR abs/1302.4381 (2013)

    Google Scholar 

  43. Abadi, D., Boncz, P., Harizopoulos, S., Idreos, S., Madden, S.: The design and implementation of modern column-oriented database systems. Found. Trends Databases 5, 197–280 (2013)

    Article  Google Scholar 

  44. Robinson, I., Webber, J., Eifreml, E.: Graph Databases. O’Reilly Media, Inc. (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. LARODEC Laboratory, Institut Supérieur de Gestion de Tunis, Tunis, Tunisia

    Mouna Ben Ishak

Authors
  1. Mouna Ben Ishak
    View author publications

    Search author on:PubMed Google Scholar

Corresponding author

Correspondence to Mouna Ben Ishak .

Editor information

Editors and Affiliations

  1. Department of Computer Engineering and Informatics, School of Engineering, University of Patras, Patras, Greece

    Ioannis Hatzilygeroudis

  2. Department of Computer Engineering and Informatics, University of Patras, Patras, Greece

    Isidoros Perikos

  3. Department of Computer Engineering and Informatics, University of Patras, Patras, Greece

    Foteini Grivokostopoulou

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Ben Ishak, M. (2020). Toward New Evaluation Metrics for Relational Learning. In: Hatzilygeroudis, I., Perikos, I., Grivokostopoulou, F. (eds) Advances in Integrations of Intelligent Methods. Smart Innovation, Systems and Technologies, vol 170. Springer, Singapore. https://doi.org/10.1007/978-981-15-1918-5_4

Download citation

Publish with us

Policies and ethics