Nothing Special   »   [go: up one dir, main page]

Journal of Natural Language Processing
Online ISSN : 2185-8314
Print ISSN : 1340-7619
ISSN-L : 1340-7619
General Paper (Peer-Reviewed)
Character-Based Thai Word Segmentation with Multiple Attentions
Thodsaporn Chay-intrHidetaka KamigaitoManabu Okumura
Author information
JOURNAL FREE ACCESS

2023 Volume 30 Issue 2 Pages 372-400

Details
Abstract

Character-based word segmentation models have been extensively applied to Asian languages, including Thai, owing to their promising performance. These models estimate the word boundaries from a character sequence; however, a Thai character unit in a sequence has no inherent meaning, in contrast with word, subword, and character cluster units that represent more meaningful linguistic information. In this paper, we propose a Thai word segmentation model that uses various types of information, including words, subwords, and character clusters, from a character sequence. Our model applies multiple attentions to refine segmentation inferences by estimating the significant relationships among characters and various unit types. We evaluated our model on three Thai datasets, and the experimental results show that our model outperforms other Thai word segmentation models, demonstrating the validity of using character clusters over subword units. A case study on sample Thai text supported these results. Thus, according to our analysis, particularly the case study, our model can segment Thai text accurately, while other existing models yield incorrect results that violate the Thai writing system.

Content from these authors
© 2023 The Association for Natural Language Processing
Previous article Next article
feedback
Top