Abstract
This paper describes the system proposed by the BIT-WOW team for NLPCC2022 shared task in Task5 Track1. The track is about multi-label towards abstracts of academic papers in scientific domain, which includes hierarchical dependencies among 1,530 labels. In order to distinguish semantic information among hierarchical label structures, we propose the Label-aware Graph Convolutional Network (LaGCN), which uses Graph Convolutional Network to capture the label association through context-based label embedding. Besides, curriculum learning is applied for domain adaptation and to mitigate the impact of a large number of categories. The experiments show that: 1) LaGCN effectively models the category information and makes a considerable improvement in dealing with a large number of categories; 2) Curriculum learning is beneficial for a single model in the complex task. Our best results were obtained by an ensemble model. According to the official results, our approach proved the best in this track.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Zhou, J., et al.: Hierarchy-aware global model for hierarchical text classification. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1106–1117. Association for Computational Linguistics (2020). https://doi.org/10.18653/v1/2020.acl-main.104
Liu, M., et al.: Overview of the NLPCC 2022 shared task 5 track 1: multi-label classification model for English scientific literature (2022)
Zhang, Y., David, P., Foroosh, H., Gong, B.: A curriculum domain adaptation approach to the semantic segmentation of urban scenes. IEEE Trans. Pattern Anal. Mach. Intell. 42(8), 1823–1841 (2020). https://doi.org/10.1109/TPAMI.2019.2903401
Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/n19-1423
Lee, J., et al.: BioBERT: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36(4), 1234–1240 (2020). https://doi.org/10.1093/bioinformatics/btz682
Bengio, S., Weston, J., Grangier, D.: Label embedding trees for large multi-class tasks. In: Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, pp. 163–171. Curran Associates, Inc. (2010)
Ma, Y., Cambria, E., Gao, S.: Label embedding for zero-shot fine-grained named entity typing. In: Proceedings of the Conference on International Conference on Computational Linguistics: Technical Papers, pp. 171–180. ACL (2016). https://aclanthology.org/C16-1017/
Liu, X., Huang, H., Shi, G., Wang, B.: Dynamic prefix-tuning for generative template-based event extraction. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pp. 5216–5228. Association for Computational Linguistics (2022). https://aclanthology.org/2022.acl-long.358
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. In: 5th International Conference on Learning Representations. OpenReview.net (2017). https://openreview.net/forum?id=SJU4ayYgl
Kumar, M.P., Packer, B., Koller, D.: Self-paced learning for latent variable models. In: Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, Vancouver, British Columbia, Canada, 6–9 December 2010, pp. 1189–1197. Curran Associates, Inc. (2010)
Liu, X., et al.: BIT-event at NLPCC-2021 task 3: subevent identification via adversarial training. In: Wang, L., Feng, Y., Hong, Yu., He, R. (eds.) NLPCC 2021. LNCS (LNAI), vol. 13029, pp. 400–411. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88483-3_32
Hamilton, W.L., Ying, R., Leskovec, J.: Representation learning on graphs: methods and applications. IEEE Data Eng. Bull. 40(3), 52–74 (2017). http://sites.computer.org/debull/A17sept/p52.pdf
Li, Y., Tarlow, D., Brockschmidt, M., Zemel, R.S.: Gated graph sequence neural networks. In: 4th International Conference on Learning Representations (2016). http://arxiv.org/abs/1511.05493
Hamilton, W.L., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. In: Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, pp. 1024–1034 (2017)
Liu, X., Huang, H., Zhang, Y.: Open domain event extraction using neural latent variable models. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, pp. 2860–2871. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/p19-1276
Gilmer, J., Schoenholz, S.S., Riley, P.F., Vinyals, O., Dahl, G.E.: Neural message passing for quantum chemistry. In: Proceedings of the 34th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 70, pp. 1263–1272. PMLR (2017). http://proceedings.mlr.press/v70/gilmer17a.html
Liu, X., Huang, H., Zhang, Y.: End-to-end event factuality prediction using directional labeled graph recurrent network. Inf. Process. Manag. 59(2), 102836 (2022). https://doi.org/10.1016/j.ipm.2021.102836
Liu, X., Luo, Z., Huang, H.: Jointly multiple events extraction via attention-based graph information aggregation. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 1247–1256. Association for Computational Linguistics (2018). https://doi.org/10.18653/v1/d18-1156
Banerjee, S., Akkaya, C., Perez-Sorrosal, F., Tsioutsiouliklis, K.: Hierarchical transfer learning for multi-label text classification. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, pp. 6295–6300. Association for Computational Linguistics (2019). https://doi.org/10.18653/v1/p19-1633
Peng, H., et al.: Large-scale hierarchical text classification with recursively regularized deep graph-CNN. In: Proceedings of the 2018 World Wide Web Conference on World Wide Web, pp. 1063–1072. ACM (2018). https://doi.org/10.1145/3178876.3186005
Dumais, S.T., Chen, H.: Hierarchical classification of web content. In: Proceedings of the 23rd Annual International Conference on Research and Development in Information Retrieval, pp. 256–263. ACM (2000). https://doi.org/10.1145/345508.345593
Kowsari, K., et al.: HDLTex: hierarchical deep learning for text classification. In: 16th International Conference on Machine Learning and Applications, pp. 364–371. IEEE (2017). https://doi.org/10.1109/ICMLA.2017.0-134
Cerri, R., Barros, R.C., de Carvalho, A.C.P.L.F.: Hierarchical multi-label classification using local neural networks. J. Comput. Syst. Sci. 80(1), 39–56 (2014). https://doi.org/10.1016/j.jcss.2013.03.007
Gopal, S., Yang, Y.: Recursive regularization for large-scale classification with hierarchical and graphical dependencies. In: The 19th International Conference on Knowledge Discovery and Data Mining, pp. 257–265. ACM (2013). https://doi.org/10.1145/2487575.2487644
Naik, A., Rangwala, H.: HierFlat: flattened hierarchies for improving top-down hierarchical classification. Int. J. Data Sci. Anal. 4(3), 191–208 (2017). https://doi.org/10.1007/s41060-017-0070-1
Wehrmann, J., Cerri, R., Barros, R.C.: Hierarchical multi-label classification networks. In: Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 80, pp. 5225–5234. PMLR (2018). http://proceedings.mlr.press/v80/wehrmann18a.html
Amigó, E., Delgado, A.D.: Evaluating extreme hierarchical multi-label classification. In: Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5809–5819. Association for Computational Linguistics (2022). https://aclanthology.org/2022.acl-long.399
Xu, L., et al.: Hierarchical multi-label text classification with horizontal and vertical category correlations. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 2459–2468. Association for Computational Linguistics (2021). https://doi.org/10.18653/v1/2021.emnlp-main.190
Zhang, H., Xiao, L., Chen, W., Wang, Y., Jin, Y.: Multi-task label embedding for text classification. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4545–4553. Association for Computational Linguistics (2018). https://aclanthology.org/D18-1484/
Du, C., Chen, Z., Feng, F., Zhu, L., Gan, T., Nie, L.: Explicit interaction model towards text classification. In: The 33rd Conference on Artificial Intelligence, The 31st Innovative Applications of Artificial Intelligence Conference, The 9th Symposium on Educational Advances in Artificial Intelligence, pp. 6359–6366. AAAI Press (2019). https://doi.org/10.1609/aaai.v33i01.33016359
Acknowledgement
This work was supported by the Joint Funds of the National Natural Science Foundation of China (Grant No. U19B2020). We would like to thank the anonymous reviewers for their thoughtful and constructive comments.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, B. et al. (2022). BIT-WOW at NLPCC-2022 Task5 Track1: Hierarchical Multi-label Classification via Label-Aware Graph Convolutional Network. In: Lu, W., Huang, S., Hong, Y., Zhou, X. (eds) Natural Language Processing and Chinese Computing. NLPCC 2022. Lecture Notes in Computer Science(), vol 13552. Springer, Cham. https://doi.org/10.1007/978-3-031-17189-5_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-17189-5_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17188-8
Online ISBN: 978-3-031-17189-5
eBook Packages: Computer ScienceComputer Science (R0)