Affiliations: [a] School of Mathematics and Big Data, Guizhou Education University, Guiyang, China
| [b] Big Data Science and Intelligent Engineering Research Institute, Guizhou Education University, Guiyang, China
Abstract: Chinese fill-in-the-blank questions contain both objective and subjective characteristics, and thus it has always been difficult to score them automatically. In this paper, fill-in-the-blank items are divided into those with word-level or sentence-level granularity; then, the items are automatically scored by different strategies. The automatic scoring framework combines semantic dictionary matching and semantic similarity calculations. First, fill-in-the-blank items with word-level granularity are divided into two types of test sites: the subject term test site, and the common word test site. We propose an algorithm for identifying an item’s test site. Then, a subject term dictionary with self-feedback learning ability is constructed to support the scoring of subject term test sites. The Tongyici Cilin semantic dictionary is used for scoring common word test sites. For fill-in-the-blank items with sentence-level granularity, an improved P-means model is used to generate a sentence vector of the standard answer and the examinee’s answer, and then the semantic similarity between the two answers is obtained by calculating the cosine distance of the sentence vector. Experimental results on actual test data show that the proposed algorithm has a maximum accuracy of 94.3% and achieves good results.
Keywords: Fill-in-the-blank question, automatic scoring, dictionary matching, semantic similarity, word vector, sentence vector, subject terminology, P-means model