Abstract
In the paper, we report the results for the NLPCC2021 shared-task of Few-shot Learning for Chinese NLP. This shared task is proposed in the context of pre-trained language models, where models only have access to limited human-labeled data. The goal of the task is to compare different learning schemes. The task includes nine sub-tasks and three task forms: single sentence classification, sentence pair classification, and machine reading comprehension. In order to accommodate the properties of few-shot learning, we sampled the examples using various sampling methods, some with 32 examples in total for one dataset, while others with 4 to 16 examples per class. Ninety teams registered for the shared task, employing a wide range of learning schemes, including data augmentation, utilizing multiple templates rather than a single template, using unlabeled data for pre-training or semi-supervised training. The best model achieved 65.3 in the mean accuracy, compared with the human score of 83.9. This result is 8 points higher than our baseline model (using the PET scheme). We believe our few-shot learning tasks and results demonstrate the potential of the recently introduced few-shot learning methods and provide guidance and important empirical evidence for future research.
L. Xu, X. Lu, C. Yuan, X. Zhang, H. Yuan and H. Xu—Contributed equally.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
It is a voice assistant developed by OPPO for mobile phones and IOT devices.
- 2.
References
Brown, T.B., et al.: Language models are few-shot learners (2020)
Cui, Y., Che, W., Liu, T., Qin, B., Wang, S., Hu, G.: Revisiting pre-trained models for Chinese natural language processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pp. 657–668. Association for Computational Linguistics, Online, November 2020. https://www.aclweb.org/anthology/2020.findings-emnlp.58
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Zhang, Y.W.E., Wang, Z.: Entailment method based on template selection for FewCLUE evaluation (2021)
Gao, T., Fisch, A., Chen, D.: Making pre-trained language models better few-shot learners. In: Association for Computational Linguistics (ACL) (2021)
Hu, H., Richardson, K., Liang, X., Lu, L., Kübler, S., Moss, L.: OCNLI: original Chinese natural language inference. In: Findings of Empirical Methods for Natural Language Processing (Findings of EMNLP) (2020)
IFLYTEK CO. L: Iflytek: a multiple categories Chinese text classifier. competition official website (2019). http://challenge.xfyun.cn/2019/gamelist
Zeng, J., Jiang, S.W.Y., Li, M.: Enhanced few-shot learning with multiple-pattern-exploiting training. In: CCF International Conference on Natural Language Processing and Chinese Computing (2021)
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., Soricut, R.: ALBERT: a lite BERT for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942 (2019)
Liu, X., et al.: GPT understands, too. arXiv preprint arXiv:2103.10385 (2021)
Liu, Y., et al.: RoBERTa: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019)
Conversational-AI Center of OPPO XiaoBu: BUSTM: OPPO Xiaobu dialogue short text matching dataset (2021). https://github.com/xiaobu-coai/BUSTM
Schick, T., Schütze, H.: Exploiting cloze-questions for few-shot text classification and natural language inference. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 255–269. Association for Computational Linguistics, Online, April 2021. https://aclanthology.org/2021.eacl-main.20
Schick, T., Schütze, H.: It’s not just size that matters: small language models are also few-shot learners. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2339–2352. Association for Computational Linguistics, Online, June 2021. https://doi.org/10.18653/v1/2021.naacl-main.185, https://aclanthology.org/2021.naacl-main.185
Tam, D., Menon, R.R., Bansal, M., Srivastava, S., Raffel, C.: Improving and simplifying pattern exploiting training (2021)
Wang, S., Fang, H., Khabsa, M., Mao, H., Ma, H.: Entailment as few-shot learner (2021)
Wei, J., et al.: NEZHA: neural contextualized representation for Chinese language understanding. arXiv preprint arXiv:1909.00204 (2019)
Xu, L., et al.: CLUE: a Chinese language understanding evaluation benchmark. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 4762–4772. International Committee on Computational Linguistics, Barcelona, Spain (Online), December 2020. https://doi.org/10.18653/v1/2020.coling-main.419, https://aclanthology.org/2020.coling-main.419
Zheng, C., Huang, M., Sun, A.: ChID: a Large-scale Chinese IDiom dataset for Cloze test. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 778–787 (2019)
Xu, Z., et al.: When few-shot learning meets large-scale knowledge-enhanced pre-training: Alibaba at FewCLUE. In: CCF International Conference on Natural Language Processing and Chinese Computing (2021)
Acknowledgements
Many thanks to NLPCC for offering us this opportunity to organize this task and people who took part in this task.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, L. et al. (2021). Few-Shot Learning for Chinese NLP Tasks. In: Wang, L., Feng, Y., Hong, Y., He, R. (eds) Natural Language Processing and Chinese Computing. NLPCC 2021. Lecture Notes in Computer Science(), vol 13029. Springer, Cham. https://doi.org/10.1007/978-3-030-88483-3_33
Download citation
DOI: https://doi.org/10.1007/978-3-030-88483-3_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88482-6
Online ISBN: 978-3-030-88483-3
eBook Packages: Computer ScienceComputer Science (R0)