Nothing Special   »   [go: up one dir, main page]

Skip to content

cubenlp/NLPCC-2023-Shared-Task7

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

31 Commits
 
 
 
 
 
 

Repository files navigation

Shared-Task-7 Chinese Essay Discourse Coherence Evaluation

最新消息

时间 消息
5月23日 track4的训练集和验证集有更新,训练集增加250条数据,验证集增加10条数据
5月20日 测试集已发布,结果提交链接已发送至各组队长邮箱,提交结果榜单详见各赛道Github主页
5月8日 已通过邮件的方式将官方微信群二维码发送给5月5日之前报名的队伍

排行榜

  结果统计截止至2023年5月31日,榜单更新时间:2023年5月31日。

Track 1 (每队历史最好成绩排行榜)

Team Name Organization Precision Recall Macro-F1 Accuracy
EssayFlow 北京大学 38.50 43.54 32.54 43.99
Evay Info AI Team 山东省计算机科学中心 35.64 35.70 35.61 36.05
ouchnai 国家开放大学 36.38 41.32 33.22 34.92
CLsuper 广东外语外贸大学 34.13 34.28 32.80 32.88

Track 1 (2023年5月31日提交结果,未排名)

Team Name Email Precision Recall Macro-F1 Accuracy
EssayFlow 210***@stu.pku.edu.cn 35.83 35.78 33.05 40.14
Evay Info AI Team 921***@qq.com 35.64 35.70 35.61 36.05

Track 2(每队历史最好成绩排行榜)

Team Name Organization Paragraph Accuracy Full Accuracy Final Accuracy Paragraph Similarity Full Similarity
wuwuwu 上海交通大学 61.27 34.92 42.82 87.34 80.37
ouchnai 国家开放大学 62.61 33.33 42.12 85.20 79.16

Track 2 (2023年5月31日提交结果,未排名)

Team Name Email Paragraph Accuracy Full Accuracy Final Accuracy Paragraph Similarity Full Similarity
ouchnai zhe***@ouchn.edu.cn 62.61 33.33 42.12 85.20 79.16

Track 3 (每队历史最好成绩排行榜)

Team Name Organization Precision Recall Macro-F1 Accuracy
ouchnai 国家开放大学 54.66 52.45 52.16 71.03
wuwuwu 上海交通大学 29.26 28.98 28.77 46.97
Lrt123 北京师范大学 28.19 30.26 27.54 48.81
BLCU_teamworkers 北京语言大学 27.17 27.65 25.95 48.73

Track 3 (2023年5月31日提交结果,未排名)

Team Name Email Precision Recall Macro-F1 Accuracy
BLCU_teamworkers sol***@163.com 24.91 25.56 19.58 45.09
wuwuwu don***@sjtu.edu.cn 25.75 16.25 4.88 7.53
Lrt123 sun***@mail.bnu.edu.cn 28.19 30.26 27.54 48.81

Track 4 (每队历史最好成绩排行榜)

Team Name Organization Precision Recall Macro-F1 Accuracy
ouchnai 国家开放大学 36.63 36.36 34.38 53.95
wuwuwu 上海交通大学 23.49 25.37 23.67 39.94
BLCU_teamworkers 北京语言大学 7.55 6.30 6.32 18.35

Track 4 (2023年5月31日提交结果,未排名)

Team Name Email Precision Recall Macro-F1 Accuracy
wuwuwu don***@sjtu.edu.cn 23.49 25.37 23.67 39.94
ouchnai zhe***@ouchn.edu.cn 35.86 39.22 32.68 55.90
BLCU_teamworkers sol***@163.com 7.55 6.30 6.32 18.35

Introduction

In the scoring of the Chinese National College Entrance Examination (NCEE) and the Senior High School Entrance Examination, essay assessment is the most time-consuming and controversial task. While existing research has focused on language factors such as characters, words, and sentences, it has not explored the relationship between discourse coherence and text quality. The logical structure and coherence within an essay are essential for evaluation, but the lack of large-scale, high-quality discourse coherence evaluation data resources has hindered the development of AI essay grading. To address this issue, the CubeNLP laboratory of East China Normal University and Microsoft have constructed a Chinese essay coherence evaluation dataset called LEssay, which provides high-quality data resources and is significant for the development of automatic essay evaluation.

This shared task includes four tracks:
Track 1. Coherence Evaluation (CE). Given a middle school student essay, annotators will assess its coherence on a three-level scale of excellent, moderate, and poor. A score of 2 indicates excellent coherence, 1 indicates moderate coherence, and 0 indicates incoherence.
Track 2. Text Topic Extraction (TTE). Given a middle school student essay, annotators need to identify the topic sentence for each paragraph and one main topic sentence for the whole essay.
Track 3. Paragraph Logical Relation Recognition (PLRR). Given two paragraphs sorted in order from a composition, the annotator needs to determine the logical relationship between the two paragraphs based on the given definitions and examples of logical relationships.
Track 4. Sentence Logical Relation Recognition (SLRR). Given two sentences from an essay that are ordered sequentially, the annotator needs to determine what type of logical relation exists between them based on given definitions and examples.

The detailed content of the guideline can be found in the file Chinese Essay Discourse Coherence Evaluation.pdf

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published