Abstract
This is an overview paper of the NLPCC 2021 shared task on AutoIE2, which aims to evaluate the sub-event identification systems with limited annotated data. Given definitions of specific sub-events, 100K unannotated samples and 300 annotated seed samples, participants are required to build a sub-event identification system. 30 teams registered and 14 of them submitted results. The top system achieves \(8.43\%\) and \(8.25\%\) accuracy score improvement upon the baseline system with or without extra annotated data respectively. The evaluation result indicates that it is possible to use less human annotation and large unlabeled corpora for the sub-event identification system. ALL information about this task can be found at https://github.com/IIGROUP/AutoIE2.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Allan, J., Papka, R., Lavrenko, V.: On-line new event detection and tracking. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 37–45 (1998)
Becker, H., Naaman, M., Gravano, L.: Learning similarity metrics for event identification in social media. In: Davison, B.D., Suel, T., Craswell, N., Liu, B. (eds.) Proceedings of the Third International Conference on Web Search and Web Data Mining, WSDM 2010, New York, NY, USA, 4–6 February 2010, pp. 291–300. ACM (2010). https://doi.org/10.1145/1718487.1718524
Bekoulis, G., Deleu, J., Demeester, T., Develder, C.: Sub-event detection from twitter streams as a sequence labeling problem. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (Long and Short Papers), pp. 745–750. Association for Computational Linguistics, Minneapolis, Minnesota (2019). https://doi.org/10.18653/v1/N19-1081, https://www.aclweb.org/anthology/N19-1081
Chen, H., Liu, X., Yin, D., Tang, J.: A survey on dialogue systems: recent advances and new frontiers. SIGKDD Explor. Newsl. 19(2), 25–35 (2017)
Cowie, J., Lehnert, W.: Information extraction. Commun. ACM 39(1), 80–91 (1996)
Du, J., Grave, E., Gunel, B., Chaudhary, V., Celebi, O., Auli, M., Stoyanov, V., Conneau, A.: Self-training improves pre-training for natural language understanding. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. pp. 5408–5418. Association for Computational Linguistics, Online (2021), https://www.aclweb.org/anthology/2021.naacl-main.426
Ein-Dor, L., et al.: Active Learning for BERT: An Empirical Study. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 7949–7962. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.emnlp-main.638, https://www.aclweb.org/anthology/2020.emnlp-main.638
Fung, G.P.C., Yu, J.X., Yu, P.S., Lu, H.: Parameter free bursty events detection in text streams. In: Proceedings of the 31st International Conference on Very Large Data Bases, pp. 181–192. Citeseer (2005)
Gao, Y., Wang, F., Luan, H., Chua, T.S.: Brand data gathering from live social media streams. In: Proceedings of International Conference on Multimedia Retrieval, pp. 169–176 (2014)
Gao, Y., Zhao, S., Yang, Y., Chua, T.-S.: Multimedia social event detection in microblog. In: He, X., Luo, S., Tao, D., Xu, C., Yang, J., Hasan, M.A. (eds.) MMM 2015. LNCS, vol. 8935, pp. 269–281. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-14445-0_24
Greengrass, E.: Information retrieval: a survey (2000)
Jie, Z., Xie, P., Lu, W., Ding, R., Li, L.: Better modeling of incomplete annotations for named entity recognition. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (Long and Short Papers), pp. 729–734. Association for Computational Linguistics, Minneapolis (2019). https://doi.org/10.18653/v1/N19-1079, https://www.aclweb.org/anthology/N19-1079
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: ALBERT: a lite BERT for self-supervised learning of language representations. In: 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, 26–30 April 2020. OpenReview.net (2020). https://openreview.net/forum?id=H1eA7AEtvS
Liu, Y., et al.: Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Mann, G.S., McCallum, A.: Generalized expectation criteria for semi-supervised learning with weakly labeled data. J. Mach. Learn. Res. 11(2), 955–984 (2010)
Meladianos, P., Nikolentzos, G., Rousseau, F., Stavrakas, Y., Vazirgiannis, M.: Degeneracy-based real-time sub-event detection in twitter stream. In: Proceedings of the International AAAI Conference on Web and Social Media, vol. 9 (2015)
Meladianos, P., Xypolopoulos, C., Nikolentzos, G., Vazirgiannis, M.: An optimization approach for sub-event detection and summarization in twitter. In: Pasi, G., Piwowarski, B., Azzopardi, L., Hanbury, A. (eds.) ECIR 2018. LNCS, vol. 10772, pp. 481–493. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-76941-7_36
Raffel, C., et al.: Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 (2019)
Reuter, T., Cimiano, P.: Event-based classification of social media streams. In: Proceedings of the 2nd ACM International Conference on Multimedia Retrieval, pp. 1–8 (2012)
Sarawagi, S.: Information Extraction. Now Publishers Inc., Norwell (2008)
Unankard, S., Li, X., Sharaf, M., Zhong, J., Li, X.: Predicting elections from social networks based on sub-event detection and sentiment analysis. In: Benatallah, B., Bestavros, A., Manolopoulos, Y., Vakali, A., Zhang, Y. (eds.) WISE 2014. LNCS, vol. 8787, pp. 1–16. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11746-1_1
Wang, Y., et al.: Adaptive self-training for few-shot neural sequence labeling. arXiv preprint arXiv:2010.03680 (2020)
Weng, J., Lee, B.S.: Event detection in twitter. In: Proceedings of the International AAAI Conference on Web and Social Media, vol. 5 (2011)
Yang, X., Wu, B., Jie, Z., Liu, Y.: Overview of the NLPCC 2020 shared task: AutoIE. In: Zhu, X., Zhang, M., Hong, Yu., He, R. (eds.) NLPCC 2020. LNCS (LNAI), vol. 12431, pp. 558–566. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-60457-8_46
Yang, Y., Pierce, T., Carbonell, J.: A study of retrospective and on-line event detection. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 28–36 (1998)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Guo, W. et al. (2021). Overview of the NLPCC 2021 Shared Task: AutoIE2. In: Wang, L., Feng, Y., Hong, Y., He, R. (eds) Natural Language Processing and Chinese Computing. NLPCC 2021. Lecture Notes in Computer Science(), vol 13029. Springer, Cham. https://doi.org/10.1007/978-3-030-88483-3_42
Download citation
DOI: https://doi.org/10.1007/978-3-030-88483-3_42
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88482-6
Online ISBN: 978-3-030-88483-3
eBook Packages: Computer ScienceComputer Science (R0)