Nothing Special   »   [go: up one dir, main page]
Skip to main content

Advertisement

Springer Nature Link
Log in

Don’t Ignore the Drive of Curiosity: Rethinking Subtleties Between Universality of Commonsense Knowledge and Excellence of Large Language Models

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Commonsense reasoning is one of the abilities necessary for artificial intelligence to be as intelligent as humans. However, how to make AI understand commonsense has been a problem that has plagued artificial intelligence for more than 60 years. Existing efforts focus more on the means of knowledge acquisition and strive to enrich the capacity of commonsense knowledge (CSK) bases and dimensions of CSK through advanced methods. Unfortunately, this exuberance has obscured a general consideration of CSK, such as how to follow human habits to obtain the most representative knowledge we need to understand the world. In this paper, this representative knowledge is referred to as core CSK. The influence of core CSK is extensive, and it constitutes almost the fundamental element of human life and the most fundamental cognition of the world. Harnessing human curiosity to find solutions to the above problems is an effective and straightforward route. Specifically, we focus on a special corpus to mine core CSK, namely, why-questions. For example, we can harvest “the sky is blue” from “why is the sky blue?”. To this end, we propose a novel method to extract CSK from why-questions, which mainly consist of two modules. The first is a question classification module used to determine whether a question contains CSK. In this module, we propose a classifier based on a one-sided bootstrapping method and design several informative features for the classifier. The second is a crowdsourcing module used to improve the quality of the extracted commonsense. We conduct extensive experiments, and the experimental results show that our method effectively mines CSK from question corpora. Furthermore, statistical analysis demonstrates the feasibility of this curiosity-driven approach, implying that we provide a basic idea for collecting core CSK. Remarkably, today’s outstanding large language models do not have such simple knowledge summarization capabilities, demonstrating the barrier between the excellence of language models and the universality of CSK.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availibility Statement

Data will be available upon request.

Code Availibility Statement

Code will be available upon request.

Notes

  1. Another class of curiosity is called perceptual curiosity.

  2. http://www.keenage.com/html/c_index.html.

  3. https://www.bing.com/.

  4. https://www.google.com/.

  5. https://www.kaggle.com/c/quora-question-pairs.

References

  1. Wang C, Liu J, Liu J, Wang W. Inference of ISA commonsense knowledge with lexical taxonomy. Appl Intell. 2022;1–14.

  2. Amodei D, Ananthanarayanan S, Anubhai R, Bai J, Battenberg E, Case C, Casper J, Catanzaro B, Cheng Q, Chen G et al. Deep speech 2: End-to-end speech recognition in English and mandarin. In: International Conference on Machine Learning, 2016;173–182.

  3. Hassan H, Aue A, Chen C, Chowdhary V, Clark J, Federmann C, Huang X, Junczys-Dowmunt M, Lewis W, Li M, et al. Achieving human parity on automatic Chinese to English news translation. arXiv preprint arXiv:1803.05567 2018.

  4. Levesque H, Davis E, Morgenstern L. The winograd schema challenge. In: Thirteenth International Conference on the Principles of Knowledge Representation and Reasoning 2012. Citeseer

  5. Lenat DB, Guha RV, Pittman K, Pratt D, Shepherd M. Cyc: toward programs with common sense. Commun ACM. 1990;33(8):30–49.

    Article  Google Scholar 

  6. Singh P, Lin T, Mueller ET, Lim G, Perkins T, Zhu WL. Open mind common sense: knowledge acquisition from the general public. In: OTM Confederated International Conferences" On the Move to Meaningful Internet Systems", 2002:1223–37. Springer

  7. Pasca M, Van Durme B. What you seek is what you get: extraction of class attributes from query logs. IJCAI. 2007;7:2832–7.

    Google Scholar 

  8. Miller GA. Wordnet: a lexical database for English. Commun ACM. 1995;38(11):39–41.

    Article  Google Scholar 

  9. Fabian M, Gjergji K, Gerhard W et al. Yago: A core of semantic knowledge unifying wordnet and Wikipedia. In: 16th International World Wide Web Conference, WWW, 2007:697–706

  10. Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J. Freebase: a collaboratively created graph database for structuring human knowledge. In: Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, 2008:1247–50.

  11. Hoffart J, Suchanek FM, Berberich K, Weikum G. Yago2: A spatially and temporally enhanced knowledge base from Wikipedia. Artif Intell. 2013;194:28–61.

    Article  MathSciNet  Google Scholar 

  12. Auer S, Bizer C, Kobilarov G, Lehmann J, Cyganiak R, Ives Z. Dbpedia: a nucleus for a web of open data. In: The Semantic Web: 6th International Semantic Web Conference, 2nd Asian Semantic Web Conference, ISWC 2007+ ASWC 2007, Busan, Korea, November 11-15, 2007. Proceedings, 2007:722–35. Springer

  13. Tandon N, Melo G, Weikum G. Acquiring comparative commonsense knowledge from the web. In: Proceedings of the AAAI Conference on Artificial Intelligence, 2014:28.

  14. Litman JA. Interest and deprivation factors of epistemic curiosity. Personality Individ Differ. 2008;44(7):1585–95.

    Article  Google Scholar 

  15. Collins RP, Litman JA, Spielberger CD. The measurement of perceptual curiosity. Personality Individ Differ. 2004;36(5):1127–41.

    Article  Google Scholar 

  16. Vracheva VP, Moussetis R, Abu-Rahma A. The mediational role of engagement in the relationship between curiosity and student development: a preliminary study. J Happiness Stud. 2020;21(4):1529–47.

    Article  Google Scholar 

  17. Berlyne DE. A theory of human curiosity. Br J Psychol. 1954;45(3):180.

    Google Scholar 

  18. Darlington K. Common sense knowledge, crucial for the success of AI systems. OpenMind BBVA 2020.

  19. Bougie N, Ichise R. Hierarchical learning from human preferences and curiosity. Appl Intell. 2022;52(7):7459–79.

    Article  Google Scholar 

  20. Wang C, Zhu T, Li Z, Liu J. Slr: A million-scale comprehensive crossword dataset for simultaneous learning and reasoning. Neurocomputing 2023;126591.

  21. Lecler A, Duron L, Soyer P. Revolutionizing radiology with gpt-based models: current applications, future possibilities and limitations of chatgpt. Diagn Interv Imaging. 2023;104(6):269–74.

    Article  Google Scholar 

  22. Zang L-J, Cao C, Cao Y-N, Wu Y-M, Cun-Gen C. A survey of commonsense knowledge acquisition. J Comput Sci Technol. 2013;28(4):689–719.

    Article  MathSciNet  Google Scholar 

  23. Xu FF, Lin BY, Zhu K. Automatic extraction of commonsense locatednear knowledge. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2018:96–101 .

  24. Tandon N, De Melo G, Suchanek F, Weikum G. Webchild: Harvesting and organizing commonsense knowledge from the web. In: Proceedings of the 7th ACM International Conference on Web Search and Data Mining, 2014:523–32.

  25. Wang C, Liu J, Liu J, Wang W. Inference of ISA commonsense knowledge with lexical taxonomy. Appl Intell. 2023;53(5):5290–303.

    Google Scholar 

  26. Sherin B. Common sense clarified: the role of intuitive knowledge in physics problem solving. J Res Sci Teach. 2006;43(6):535–55.

    Article  Google Scholar 

  27. Liu H, Singh P. Conceptnet-a practical commonsense reasoning tool-kit. BT Technol J. 2004;22(4):211–26.

    Article  Google Scholar 

  28. Davis E. Representations of commonsense knowledge. Morgan Kaufmann; 2014.

    Google Scholar 

  29. Paul D. Social commonsense reasoning with structured knowledge in text. PhD thesis 2024.

  30. Wang C, Liu J, Liu J, Jiang S, Li Z, Xiao Y. Sweet apple, company? or food? adjective-centric commonsense knowledge acquisition with taxonomy-guided induction. Knowl-Based Syst. 2023;280:110988.

    Article  Google Scholar 

  31. Liu J, Chen T, Wang C, Liang J, Chen L, Xiao Y, Chen Y, Jin K. Vocsk: Verb-oriented commonsense knowledge mining with taxonomy-guided induction. Artif Intell. 2022;310:103744.

    Article  MathSciNet  Google Scholar 

  32. He M, Fang T, Wang W, Song Y. Acquiring and modeling abstract commonsense knowledge via conceptualization. Artif Intell. 2024:104149.

  33. Tandon N, Varde AS, de Melo G. Commonsense knowledge in machine intelligence. ACM SIGMOD Rec. 2018;46(4):49–52.

    Article  Google Scholar 

  34. Tandon N, De Melo G, Weikum G. Webchild 2.0: Fine-grained commonsense knowledge distillation. In: Proceedings of ACL 2017, System Demonstrations, 2017:115–20.

  35. Hwang JD, Bhagavatula C, Le Bras R, Da J, Sakaguchi K, Bosselut A, Choi Y. (comet-) atomic 2020: On symbolic and neural commonsense knowledge graphs. In: Proceedings of the AAAI Conference on Artificial Intelligence, 2021:35;6384–92.

  36. Zhao Z, Lee WS, Hsu D. Large language models as commonsense knowledge for large-scale task planning. Adv Neural Inf Process Syst. 2024;36.

  37. Liu C, Wang C, Peng Y, Li Z. Zvqaf: Zero-shot visual question answering with feedback from large language models. Neurocomputing. 2024;580:127505.

    Article  Google Scholar 

  38. Tang X, Zheng Z, Li J, Meng F, Zhu S-C, Liang Y, Zhang M. Large language models are in-context semantic reasoners rather than symbolic reasoners. arXiv preprint arXiv:2305.14825 2023.

  39. Wei J, Wang X, Schuurmans D, Bosma M, Xia F, Chi E, Le QV, Zhou D, et al. Chain-of-thought prompting elicits reasoning in large language models. Adv Neural Inf Process Syst. 2022;35:24824–37.

    Google Scholar 

  40. West P, Bhagavatula C, Hessel J, Hwang JD, Jiang L, Bras RL, Lu X, Welleck S, Choi Y. Symbolic knowledge distillation: from general language models to commonsense models. arXiv preprint arXiv:2110.07178 2021.

  41. Gu Y, Mishra BD, Clark P. Do language models have coherent mental models of everyday things? arXiv preprint arXiv:2212.10029 2022.

  42. Bian N, Han X, Sun L, Lin H, Lu Y, He B, Jiang S, Dong B. Chatgpt is a knowledgeable but inexperienced solver: an investigation of commonsense problem in large language models. arXiv preprint arXiv:2303.16421 2023.

  43. Chen J, Shi W, Fu Z, Cheng S, Li L, Xiao Y. Say what you mean! large language models speak too positively about negative commonsense knowledge. arXiv preprint arXiv:2305.05976 2023.

  44. Cheng Q, Sun T, Liu X, Zhang W, Yin Z, Li S, Li L, Chen K, Qiu X. Can AI assistants know what they don’t know? arXiv preprint arXiv:2401.13275 2024.

  45. Wu W, Li H, Wang H, Zhu KQ. Probase: A probabilistic taxonomy for text understanding. In: Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data, 2012:481–92.

  46. Wang Z, Wang H, Wen J-R, Xiao Y. An inference approach to basic level of categorization. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, 2015:653–62.

  47. Han D. On indexical. Journal of Renmin University of China 2015.

  48. Zhou L-Z, He Y-K, Wang J-Y. Survey on research of sentiment analysis. J Comput Appl. 2008;28(11):2725–8.

    Google Scholar 

  49. Wang X-D, Wang J, Zhang Z. Computation on orientation for subjective sentence based on sentiment words ontology. J Comput Appl. 2012;32(6):1678–81.

    Google Scholar 

  50. Jones KS. A statistical interpretation of term specificity and its application in retrieval. J Document. 1972.

  51. Reimers N, Gurevych I. Making monolingual sentence embeddings multilingual using knowledge distillation. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020:4512–25.

  52. Devlin J, Chang M-W, Lee K, Toutanova K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 2018.

  53. Fleiss JL. Measuring nominal scale agreement among many raters. Psychol Bull. 1971;76(5):378.

    Article  Google Scholar 

  54. McHugh ML. The chi-square test of independence. Biochemia medica. 2013;23(2):143–9.

    Article  Google Scholar 

  55. Speer R, Chin J, Havasi C. Conceptnet 5.5: An open multilingual graph of general knowledge. In: Thirty-First AAAI Conference on Artificial Intelligence 2017.

  56. Tandon N. Commonsense knowledge acquisition and applications. PhD thesis, Saarland University, Saarbrücken, Germany 2016.

  57. Wang C, Liu J, Zhuang T, Li J, Liu J, Xiao Y, Wang W, Xie R. A sequence-to-sequence model for large-scale chinese abbreviation database construction. In: Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining, 2022:1063–71.

  58. Song R, Liu Z, Chen X, An H, Zhang Z, Wang X, Xu H. Label prompt for multi-label text classification. Appl Intell. 2022:1–15.

  59. Faal F, Schmitt K, Yu JY. Reward modeling for mitigating toxicity in transformer-based language models. Appl Intell. 2022:1–15.

  60. Zhang Z, Pang J, Xie X, Zhou Y. Research on crowdsourcing quality control strategies and evaluation algorithm. Chin J Comput. 2013;36(8):1636–49.

    Article  Google Scholar 

  61. Liu X, Zheng Y, Du Z, Ding M, Qian Y, Yang Z, Tang J. Gpt understands, too. arXiv preprint arXiv:2103.10385 2021.

  62. Sunday-Grève S, Williamson T. Philosophy and common sense 2: cultivating curiosity. Philosophers’ Magazine. 2022;96:24–30.

    Article  Google Scholar 

Download references

Acknowledgements

We thank all the anonymous reviewers for their constructive suggestions for the manuscript. This work was supported by the Natural Science Foundation of Shanghai (No. 23ZR1422800).

Author information

Authors and Affiliations

  1. School of Future Technology, Shanghai University, Shanghai, China

    Chao Wang

  2. Institute of Artificial Intelligence, Shanghai University, Shanghai, China

    Chao Wang

  3. Shanghai Key Laboratory of Data Science, School of Computer Science, Fudan University, Shanghai, China

    Tao Chen

  4. School of Information Science and Engineering, East China University of Science and Technology, Shanghai, China

    Jingping Liu

Authors
  1. Chao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tao Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jingping Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the main ideas present in this manuscript. The first draft of the manuscript were written by Chao Wang, Tao Chen and Jingping Liu. All authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Chao Wang.

Ethics declarations

Conflict of Interest

The authors have no relevant financial or non-financial interests to disclose.

Consent for Publication

The authors all agree to publish the article in Machine Learning.

Consent to Participate

Not Applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, C., Chen, T. & Liu, J. Don’t Ignore the Drive of Curiosity: Rethinking Subtleties Between Universality of Commonsense Knowledge and Excellence of Large Language Models. SN COMPUT. SCI. 5, 798 (2024). https://doi.org/10.1007/s42979-024-03165-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-024-03165-w

Keywords

Search

Navigation