Nothing Special   »   [go: up one dir, main page]

Skip to main content

Automatic Academic Paper Rating Based on Modularized Hierarchical Attention Network

  • Conference paper
  • First Online:
Natural Language Processing and Chinese Computing (NLPCC 2022)

Abstract

Automatic academic paper rating (AAPR) remains a difficult but useful task to automatically predict whether to accept or reject a paper. Having found more task-specific structure features of academic papers, we present a modularized hierarchical attention network (MHAN) to predict paper quality. MHAN uses a three-level hierarchical attention network to shorten the sequence for each level. In the network, the modularized parameter distinguishes the semantics of functional chapters. And a label-smoothing mechanism is used as a loss function to avoid inappropriate labeling. Compared with MHCNN and plain HAN on an AAPR dataset, MHAN achieves a state-of-the-art accuracy of 65.33%. Ablation experiments show that the proposed methods are effective.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Models can be found at https://huggingface.co/prajjwal1/bert-medium..

References

  1. Kang, D., et al.: A dataset of peer reviews (PeerRead): collection, insights and NLP applications. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1647–1661 (2018)

    Google Scholar 

  2. Yang, P., Sun, X., Li, W., Ma, S.: Automatic academic paper rating based on modularized hierarchical convolutional neural network. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 496–502 (2018)

    Google Scholar 

  3. Qiao, Feng, Xu, Lizhen, Han, Xiaowei: Modularized and attention-based recurrent convolutional neural network for automatic academic paper aspect scoring. In: Meng, Xiaofeng, Li, Ruixuan, Wang, Kanliang, Niu, Baoning, Wang, Xin, Zhao, Gansen (eds.) WISA 2018. LNCS, vol. 11242, pp. 68–76. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-02934-0_7

    Chapter  Google Scholar 

  4. Leng, Y., Yu, L., Xiong, J.: DeepReviewer: collaborative grammar and innovation neural network for automatic paper review. In: 2019 International Conference on Multimodal Interaction, pp. 395–403 (2019)

    Google Scholar 

  5. Skorikov, M., Momen, S.: Machine learning approach to predicting the acceptance of academic papers. In: 2020 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT). IEEE (2020)

    Google Scholar 

  6. Vincent-Lamarre, P., Larivière, V.: Textual analysis of artificial intelligence manuscripts reveals features associated with peer review outcome. Quant. Sci. Stud. 2(2), 662–677 (2021)

    Article  Google Scholar 

  7. Shen, A., Salehi, B., Baldwin, T., Qi, J.: A joint model for multimodal document quality assessment. In: 2019 ACM/IEEE Joint Conference on Digital Libraries (JCDL), pp. 107–110. IEEE (2019)

    Google Scholar 

  8. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  9. Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation (2014)

    Google Scholar 

  10. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  11. Kenton, J.D.M.W.C., Toutanova, L.K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, pp. 4171–4186 (2019)

    Google Scholar 

  12. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A., Hovy, E.: Hierarchical attention networks for document classification. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1480–1489 (2016)

    Google Scholar 

  13. Beltagy, I., Peters, M.E., Cohan, A.: Longformer: the long-document transformer. arXiv preprint arXiv:2004.05150 (2020)

  14. Zaheer, M., et al.: Big bird: transformers for longer sequences. Adv. Neural Inf. Process. Syst. 33, 17283–17297 (2020)

    Google Scholar 

  15. Langford, J., Guzdial, M.: The arbitrariness of reviews, and advice for school administrators. Commun. ACM 58(4), 12–13 (2015)

    Article  Google Scholar 

  16. Lin, J., Song, J., Zhou, Z., Shi, X.: Automated scholarly paper review: possibility and challenges. arXiv preprint arXiv:2111.07533 (2021)

  17. Shibayama, S., Yin, D., Matsumoto, K.: Measuring novelty in science with word embedding. PLoS ONE 16(7), e0254034 (2021)

    Article  Google Scholar 

  18. Daudaravicius, V.: Automated evaluation of scientific writing: AESW shared task proposal. In: Proceedings of the Tenth Workshop on Innovative Use of NLP for Building Educational Applications, pp. 56–63 (2015)

    Google Scholar 

  19. Springstein, M., Müller-Budack, E., Ewerth, R.: QuTI! quantifying text-image consistency in multimodal documents. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2575–2579 (2021)

    Google Scholar 

  20. Hou, Y., Jochim, C., Gleize, M., Bonin, F., Ganguly, D.: TDMSci: a specialized corpus for scientific literature entity tagging of tasks datasets and metrics. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 707–714 (2021)

    Google Scholar 

  21. Gupta, Y., et al.: The effect of pretraining on extractive summarization for scientific documents. In: Proceedings of the Second Workshop on Scholarly Document Processing, pp. 73–82 (2021)

    Google Scholar 

  22. Wang, Q., Zeng, Q., Huang, L., Knight, K., Ji, H., Rajani, N.F.: ReviewRobot: explainable paper review generation based on knowledge synthesis. In: Proceedings of the 13th International Conference on Natural Language Generation, pp. 384–397 (2020)

    Google Scholar 

  23. Yuan, W., Liu, P., Neubig, G.: Can we automate scientific reviewing? arXiv preprint arXiv:2102.00176 (2021)

  24. de Buy Wenniger, G.M., van Dongen, T., Aedmaa, E., Kruitbosch, H.T., Valentijn, E.A., Schomaker, L.: Structure-tags improve text classification for scholarly document quality prediction. In: Proceedings of the First Workshop on Scholarly Document Processing, pp. 158–167 (2020)

    Google Scholar 

  25. Huang, J.B.: Deep paper gestalt. arXiv preprint arXiv:1812.08775 (2018)

  26. Kim, Y.: Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746–1751. Association for Computational Linguistics, Doha, Qatar (2014)

    Google Scholar 

Download references

Acknowledgments

This work is partly supported by the Beijing Natural Science Foundation (No. 4212026) and the Fundamental Strengthening Program Technology Field Fund (No. 2021-JCJQ-JJ-0059).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Huaping Zhang .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kang, K., Zhang, H., Li, Y., Luo, X., Wushour, S. (2022). Automatic Academic Paper Rating Based on Modularized Hierarchical Attention Network. In: Lu, W., Huang, S., Hong, Y., Zhou, X. (eds) Natural Language Processing and Chinese Computing. NLPCC 2022. Lecture Notes in Computer Science(), vol 13551. Springer, Cham. https://doi.org/10.1007/978-3-031-17120-8_52

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-17120-8_52

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-17119-2

  • Online ISBN: 978-3-031-17120-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics