Dual-Detector: An Unsupervised Learning Framework for Chinese Spelling Check
Authors: 
Feiran Shao, Jinlong LiAuthors Info & Claims
Pages 162 - 173
Abstract
The task of Chinese Spelling Check (CSC) is to detect and correct spelling errors in Chinese sentences. Since the scale of labeled CSC training set is quite small, we propose an unsupervised Chinese spelling correction framework based on detectors. Two kinds of detectors: Dec-Err and Dec-Eva, are proposed to leverage the contextual information to detect misspelled characters and evaluate the corrections respectively. Both detectors are fine-tuned with our proposed hybrid mask strategy. Dec-Eva is a transformer encoder based detector, of which we modify the attention connections to reuse the contextual information and parallel evaluate possible corrections. Compared with supervised and unsupervised state-of-the-art methods, experimental studies show that our method achieves competitive results. Further empirical studies reveal the efficiency and flexibility of our method.
References
[1]
Chao, Y.C., Chang, C.H.: Automatic spelling correction for ASR corpus in traditional Chinese language using Seq2Seq models. In: 2020 International Computer Symposium (ICS), pp. 553–558. IEEE (2020)
[2]
Cheng, X., et al.: SpellGCN: incorporating phonological and visual similarities into language models for Chinese spelling check. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 871–881. Association for Computational Linguistics (2020). https://aclanthology.org/2020.acl-main.81
[3]
Clark, K., Luong, M.T., Le, Q.V., Manning, C.D.: ELECTRA: pre-training text encoders as discriminators rather than generators. In: ICLR (2020). https://openreview.net/pdf?id=r1xMH1BtvB
[4]
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, pp. 4171–4186. Association for Computational Linguistics (2019). https://aclanthology.org/N19-1423
[5]
Hong, Y., Yu, X., He, N., Liu, N., Liu, J.: FASPell: a fast, adaptable, simple, powerful Chinese spell checker based on DAE-decoder paradigm. In: Proceedings of the 5th Workshop on Noisy User-Generated Text (W-NUT 2019), pp. 160–169 (2019)
[6]
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 7–9 May 2015, Conference Track Proceedings (2015). http://arxiv.org/abs/1412.6980
[7]
Li, P.: uChecker: masked pretrained language models as unsupervised Chinese spelling checkers. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 2812–2822. International Committee on Computational Linguistics, Gyeongju, Republic of Korea (2022). https://aclanthology.org/2022.coling-1.248
[8]
Liu, C.L., Lai, M.H., Chuang, Y.H., Lee, C.Y.: Visually and phonologically similar characters in incorrect simplified Chinese words. In: Coling 2010: Posters, pp. 739–747 (2010)
[9]
Liu, S., Yang, T., Yue, T., Zhang, F., Wang, D.: Plome: pre-training with misspelled knowledge for Chinese spelling correction. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 2991–3000 (2021)
[10]
Martins B and Silva MJ Vicedo JL, Martínez-Barco P, Muńoz R, and Saiz Noeda M Spelling correction for search engine queries Advances in Natural Language Processing 2004 Heidelberg Springer 372-383
[11]
Nguyen TTH, Jatowt A, Coustaty M, and Doucet A Survey of post-OCR processing approaches ACM Comput. Surv. (CSUR) 2021 54 6 1-37
[12]
Ramesh, D., Sanampudi, S.K.: An automated essay scoring systems: a systematic literature review. Artif. Intell. Rev. 1–33 (2021)
[13]
Vaswani, A., et al.: Attention is all you need. In: Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS 2017, pp. 6000–6010. Curran Associates Inc., Red Hook (2017)
[14]
Wang, D., Song, Y., Li, J., Han, J., Zhang, H.: A hybrid approach to automatic corpus generation for Chinese spelling check. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2517–2527 (2018)
[15]
Xie, W., et al.: Chinese spelling check system based on n-gram model. In: Proceedings of the Eighth SIGHAN Workshop on Chinese Language Processing, pp. 128–136 (2015)
[16]
Yeh, J.F., Li, S.F., Wu, M.R., Chen, W.Y., Su, M.C.: Chinese word spelling correction based on n-gram ranked inverted index list. In: Proceedings of the Seventh SIGHAN Workshop on Chinese Language Processing, pp. 43–48 (2013)
[17]
Yu, J., Li, Z.: Chinese spelling error detection and correction based on language model, pronunciation, and shape. In: Proceedings of The Third CIPS-SIGHAN Joint Conference on Chinese Language Processing, pp. 220–223 (2014)
[18]
Zhang, R., et al.: Correcting Chinese spelling errors with phonetic pre-training. In: Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 2250–2261 (2021)
Recommendations
Improve Chinese Spelling Check by Reevaluation
Advances in Knowledge Discovery and Data MiningAbstractChinese Spelling Check (CSC) aims to detect and correct the spelling errors in Chinese. Most Chinese spelling errors are misused semantically, phonetically or graphically similar characters. Previous state-of-the-art works on the CSC task pursue ...
A Hybrid Model for Chinese Spelling Check
Spelling check for Chinese has more challenging difficulties than that for other languages. A hybrid model for Chinese spelling check is presented in this article. The hybrid model consists of three components: one graph-based model for generic errors ...
An Unsupervised Domain-Adaptive Framework for Chinese Spelling Checking
Chinese Spelling Check (CSC) is a meaningful task in the area of natural language processing, which aims at detecting spelling errors in Chinese texts and then correcting these errors. Current typical CSC models have shown impressive performance in ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
May 2023
359 pages
ISBN:978-3-031-33382-8
DOI:10.1007/978-3-031-33383-5
- Editors:
- Hisashi Kashima,
- Tsuyoshi Ide,
- Wen-Chih Peng
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2023.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 26 May 2023
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 21 Nov 2024
Other Metrics
Citations
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in