SemEval-2016 Task 9 (Chinese Semantic Dependency Parsing) Datasets
For more information: http://alt.qcri.org/semeval2016/task9/
Thanks to all participants!
Statistics
This task provides two distinguished corpora in appreciable quantity respectively in the domain of NEWS and TEXTBOOKS (from primary school textbooks).
| Set | NEWS | TEXTBOOKS |
|---|---|---|
| train | 8301 | 10754 |
| validation | 534 | 1535 |
| test | 1233 | 3073 |
Systems
Fifteen organizations were registered to participate in this task. Finally, five systems were received from three organizations. The main evaluation results are listed below, see the task description paper for detailed description and results.
Results on NEWS corpus:
| System | LF | UF | NLF | NUF |
|---|---|---|---|---|
| IHS-RD-Belarus | 59.06 | 77.64 | 40.84 | 60.20 |
| OCLSP (lbpg) | 57.22 | 74.93 | 45.57 | 58.03 |
| OCLSP (lbpgs) | 57.81 | 75.54 | 41.56 | 54.34 |
| OCLSP (lbpg75) | 57.78 | 75.40 | 48.89 | 58.28 |
| OSU-CHGCG | 55.69 | 73.72 | 49.23 | 60.71 |
Results on TEXTBOOKS corpus:
| System | LF | UF | NLF | NUF |
|---|---|---|---|---|
| IHS-RD-Belarus | 68.59 | 82.41 | 50.57 | 64.58 |
| OCLSP (lbpg) | 65.54 | 79.39 | 51.75 | 63.21 |
| OCLSP (lbpgs) | 66.21 | 79.85 | 47.79 | 55.51 |
| OCLSP (lbpg75) | 66.38 | 79.91 | 57.51 | 63.87 |
| OSU-CHGCG | 65.17 | 78.83 | 54.70 | 65.71 |
Reference
If you wish to use this data in your research, please cite:
@InProceedings{che-EtAl:2016:SemEval,
author = {Che, Wanxiang and Shao, Yanqiu and Liu, Ting and Ding, Yu},
title = {SemEval-2016 Task 9: Chinese Semantic Dependency Parsing},
booktitle = {Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016)},
year = {2016},
publisher = {Association for Computational Linguistics},
pages = {1074-1080},
}
SemEval-2016 Task 9 Evaluation Committee