• Stars
    star
    188
  • Rank 205,563 (Top 5 %)
  • Language
    Python
  • License
    Creative Commons ...
  • Created almost 10 years ago
  • Updated about 1 year ago

Reviews

There are no reviews yet. Be the first to send feedback to the community and the maintainers!

Repository Details

English data

Universal Dependencies - English Dependency Treebank Universal Dependencies English Web Treebank v2.12 -- 2023-05-15 https://github.com/UniversalDependencies/UD_English-EWT

Summary

A Gold Standard Universal Dependencies Corpus for English, built over the source material of the English Web Treebank LDC2012T13 (https://catalog.ldc.upenn.edu/LDC2012T13).

Introduction

The corpus comprises 254,825 words and 16,621 sentences, taken from five genres of web media: weblogs, newsgroups, emails, reviews, and Yahoo! answers. See the LDC2012T13 documentation for more details on the sources of the sentences. The trees were automatically converted into Stanford Dependencies and then hand-corrected to Universal Dependencies. All the basic dependency annotations have been single-annotated, a limited portion of them have been double-annotated, and subsequent correction has been done to improve consistency. Other aspects of the treebank, such as Universal POS, features and enhanced dependencies, has mainly been done automatically, with very limited hand-correction.

License/Copyright

Universal Dependencies English Web Treebank annotations © 2013-2021 by The Board of Trustees of The Leland Stanford Junior University. All Rights Reserved.

The annotations and database rights of the Universal Dependencies English Web Treebank are licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

You should have received a copy of the license along with this work. If not, see http://creativecommons.org/licenses/by-sa/4.0/.

The underlying texts come from various sources collected for the LDC English Web Treebank. Some parts are in the public domain. Portions may be © 2012 Google Inc., © 2011 Yahoo! Inc., © 2012 Trustees of the University of Pennsylvania and/or © other original authors.

Structure

This directory contains a corpus of sentences annotated using Universal Dependencies annotation. The corpus comprises 254,825 words and 16,621 sentences, taken from various web media including weblogs, newsgroups, emails, reviews, and Yahoo! answers; see the LDC2012T13 documentation for more details on the source of the sentences. The trees were automatically converted into Stanford Dependencies and then hand-corrected to Universal Dependencies. All the dependency annotations have been single-annotated, and a limited portion of them have been double-annotated with interannotator agreement at approximately 96%. The sentence IDs include the genre and the filename of the original LDC2012T13 filename.

This corpus is compatible with the CoNLL-U format defined for Universal Dependencies. See:

http://universaldependencies.github.io/docs/format.html

The dependency taxonomy can be found on the Universal Dependencies web site:

http://www.universaldependencies.org

For the conversion to v2, we performed an automatic conversion with extensive spot-checking, and manual adjudication of ambiguous cases.

The enhanced dependencies were automatically obtained by running an adapted version of the converter by Schuster and Manning (2016). These dependencies have not been manually checked.

Deviations from UD

Version 2.12 of the English UD treebank conforms to the UD guidelines in almost all respects, but there remain the following deviations:

  • The UD dependency flat is largely only used for person names.
  • Dates are not annotated consistently.

Changelog

2023-05-15 v2.12

Highlights:

  • Implement new policy on sole iobj (#55)
  • Cleanup of free relatives (part of #278)
  • Make use of xcomp more consistent
  • For "etc.", change Number=Sing to Number=Plur

2022-11-15 v2.11

Highlights:

  • Implement :outer per multiple subjects policy (#310)
  • Implement advcl:relcl (#346)
  • Implement revised guidelines for English pronouns (lemmas, features) (issue)
  • Revise WH-adverbs to attach in a subordinate clause as advmod not mark (#88)
  • Improved lemmas/features for numeric values/entities
  • Tag "etc." as NOUN (#353)
  • Add neaten.py, which implements English-specific validation rules

2022-05-15 v2.10

Highlights:

2021-11-15 v2.9

  • Fixed all validation errors
  • Many other improvements to annotation of assorted words and constructions

2021-05-15 v2.8

  • Fixed many wrong lemmata, POS tags, and relations
  • Reanalyzed many dependencies to conform to UD validation
  • Retagged words in names to limit PROPN to true nouns
  • Fixed certain metadata issues
  • Fixed some typos, added CorrectForms
  • Added missing multiword tokens for clitics/contractions
  • Added Style=Expr feature for expressive spellings
  • Fixed many incorrectly non-projective graphs, reanalyzing as projective
  • Reannotated list items as NUM not X

2020-11-15 v2.7

  • Added multiword tokens where appropriate for contracted verb forms
  • Fixed some wrong lemmata and POS tags

2020-05-15 v2.6

  • Added paragraph boundaries
  • Fixed some wrong lemmata and POS tags
  • Fixed directionality of some goeswith dependencies

2019-11-15 v2.5

  • Fixed miscellaneous syntactic issues
  • Fixed CoNLL-U syntax error

2019-05-15 v2.4

  • Fixed some wrong lemmata and POS tags
  • Fixed miscellaneous syntactic issues
  • Fixed some punctuation attachments
  • Fixed malformed enhanced graphs

2018-11-15 v2.3

  • Fixed several lemmata

2018-04-15 v2.2

  • Repository renamed from UD_English to UD_English-EWT
  • Automatically added enhanced dependencies (These have not been manually checked!)
  • Fixed some wrong lemmata and POS tags
  • Fixed miscellaneous syntactic issues

2017-11-15 v2.1

  • Fixed some wrong lemmata, POS tags
  • Fixed miscellaneous syntactic issues
  • Added basic dependencies into the DEPS column according to the CONLL-U v2 format

2017-02-15 v2.0

  • Updated treebank to conform to v2 guidelines
  • Fixed some wrong lemmata
  • Fixed miscellaneous syntactic issues
  • Added empty nodes for gapped constructions in enhanced representation

2016-11-15 v1.4

  • Changed POS tag of fused det-noun pronouns (e.g., "somebody", "nothing") to PRON
  • Added original, untokenized sentences to CoNLL-U files
  • Fixed some POS errors, features and wrong lemmata
  • Fixed miscellaneous syntactic issues in a few sentences

2016-05-15 v1.3

  • Improved mapping of WDT to UPOS
  • Corrected lemma of "n't" to "not"
  • Fixed some errors between advcl, ccomp and parataxis
  • Fixed inconsistent analyses of sentences repeated between dev and train sets
  • Fixed miscellaneous syntactic issues in a few sentences

2015-11-15 v1.2

  • Bugfix: removed _NFP suffix from some lemmas
  • Fixed date annotations to adopt UD standard
  • Remove escaping of ( and ) from word tokens (XPOSTAGs are still -LRB- and -RRB-)
  • Improved precision of xcomp relation
  • Improved recall of name relation
  • Corrected lemmas for reduced auxiliaries
  • Corrected UPOS tags of pronominal uses of this/that/these/those (from DET to PRON)
  • Corrected UPOS tags of subordinating conjunctions (from ADP to SCONJ)
  • Corrected UPOS tags of some main verbs (from AUX to VERB)

Contributing

To help improve the corpus, please alert us to any errors you find in it. The best way to do this is to file a github issue at:

https://github.com/UniversalDependencies/UD_English-EWT/issues

We also welcome pull requests. If you want to make edits, please modify the trees in the individual files in the not-to-release/sources directory instead of making direct changes to en_ewt-ud-{dev,test,train}.conllu.

Acknowledgments

Annotation of the Universal Dependencies English Web Treebank was carried out by (in order of size of contribution):

  • Natalia Silveira
  • Timothy Dozat
  • Sebastian Schuster
  • Miriam Connor
  • Marie-Catherine de Marneffe
  • Nathan Schneider
  • Ethan Chi
  • Samuel Bowman
  • Christopher Manning
  • Hanzhi Zhu
  • Daniel Galbraith
  • John Bauer

Creation of the CoNLL-U files, including calculating UPOS, feature, and lemma information was primarily done by

  • Sebastian Schuster
  • Natalia Silveira

The construction of the Universal Dependencies English Web Treebank was partially funded by a gift from Google, Inc., which we gratefully acknowledge.

Citations

You are encouraged to cite this paper if you use the Universal Dependencies English Web Treebank:

@inproceedings{silveira14gold,
  year = {2014},
  author = {Natalia Silveira and Timothy Dozat and Marie-Catherine de
	  Marneffe and Samuel Bowman and Miriam Connor and John Bauer and
	  Christopher D. Manning},
  title = {A Gold Standard Dependency Corpus for {E}nglish},
  booktitle = {Proceedings of the Ninth International Conference on Language
    Resources and Evaluation (LREC-2014)}
}

Metadata

=== Machine-readable metadata (DO NOT REMOVE!) ================================
Data available since: UD v1.0
License: CC BY-SA 4.0
Includes text: yes
Genre: blog social reviews email web
Lemmas: automatic with corrections
UPOS: converted with corrections
XPOS: manual native
Features: automatic
Relations: manual native
Contributors: Silveira, Natalia; Dozat, Timothy; Manning, Christopher; Schuster, Sebastian; Chi, Ethan; Bauer, John; Connor, Miriam; de Marneffe, Marie-Catherine; Schneider, Nathan; Bowman, Sam; Zhu, Hanzhi; Galbraith, Daniel; Bauer, John
Contributing: here source
Contact: [email protected]
===============================================================================

More Repositories

1

docs

Universal Dependencies online documentation
HTML
247
star
2

tools

Various utilities for processing the data.
Python
192
star
3

UD_Chinese-GSD

86
star
4

UD_Portuguese-Bosque

This Universal Dependencies (UD) Portuguese treebank.
Common Lisp
49
star
5

UD_Indonesian-GSD

Indonesian conversion
42
star
6

UD_Turkish-IMST

38
star
7

universaldependencies.github.io

Universal dependencies homepage
HTML
36
star
8

UD_Chinese-GSDSimp

Conversion of UD_Chinese-GSD to simplified Chinese characters.
35
star
9

UD_Japanese-GSD

Japanese data from the Google UDT 2.0.
34
star
10

UD_Vietnamese-VTB

34
star
11

UD_English-GUM

28
star
12

UD_Persian-Seraji

UD_Persian
26
star
13

UD_Ukrainian-IU

26
star
14

UD_Cantonese-HK

Spoken Cantonese from Hong Kong.
24
star
15

UD_Classical_Chinese-Kyoto

24
star
16

UD_Spanish-AnCora

Spanish data from the AnCora corpus.
23
star
17

UD_French-GSD

22
star
18

UD_Korean-GSD

Korean UD Treebank.
22
star
19

UD_English-ESL

English as a Second Language
Python
21
star
20

UD_Hindi-HDTB

21
star
21

UD_Portuguese-GSD

Brazilian Portuguese data from the Google Universal Dependency Treebanks 2.0.
Python
20
star
22

UD_Japanese-BCCWJ

Python
20
star
23

UD_Greek-GDT

UD Greek
18
star
24

UD_Korean-Kaist

Data from KAIST (a Korean treebank).
18
star
25

UD_Italian-ISDT

17
star
26

UD_Romanian-RRT

17
star
27

UD_Norwegian-Bokmaal

16
star
28

UD_Turkish-BOUN

16
star
29

UD_German-GSD

15
star
30

UD_Thai-PUD

Parallel Universal Dependencies.
14
star
31

UD_Russian-GSD

Shell
14
star
32

UD_Chinese-CFL

Chinese as a foreign language.
14
star
33

UD_Swedish-Talbanken

Swedish data
Python
12
star
34

UD_Russian-Taiga

11
star
35

UD_Bulgarian-BTB

Perl
11
star
36

UD_Spanish-GSD

11
star
37

UD_Polish-PDB

Polish data.
10
star
38

UD_Armenian-ArmTDP

Armenian data.
9
star
39

UD_German-HDT

9
star
40

UD_Hebrew-HTB

Hebrew Universal Dependencies Treebank
9
star
41

UD_Indonesian-PUD

Parallel Universal Dependencies.
9
star
42

UD_Hebrew-IAHLTwiki

9
star
43

UD_Sanskrit-UFAL

Sanskrit data.
8
star
44

UD_Chinese-HK

Spoken mandarin Chinese from Hong Kong.
8
star
45

UD_Dutch-Alpino

Dutch data.
8
star
46

UD_Tamil-TTB

Tamil data.
8
star
47

UD_French-Sequoia

Data from the Sequoia treebank.
8
star
48

UD_English-PUD

Parallel Universal Dependencies.
8
star
49

UD_Chinese-PUD

Parallel Universal Dependencies.
7
star
50

UD_Danish-DDT

7
star
51

UD_Finnish-TDT

Finnish data
7
star
52

cairo

Cairo CICLing Corpus – a multi-lingual parallel UD-style treebank of short sentences
Perl
7
star
53

UD_Nheengatu-CompLin

6
star
54

UD_English-Pronouns

6
star
55

UD_Irish-IDT

Irish data
Shell
6
star
56

UD_Amharic-ATT

Python
6
star
57

UD_Galician-TreeGal

6
star
58

UD_Polish-LFG

6
star
59

UD_Icelandic-Modern

Python
6
star
60

UD_Urdu-UDTB

6
star
61

UD_Portuguese-PUD

Parallel Universal Dependencies.
6
star
62

UD_Telugu-MTG

Telugu data.
6
star
63

UD_Pomak-Philotis

6
star
64

UD_Norwegian-NynorskLIA

5
star
65

UD_Latin-Perseus

5
star
66

UD_Turkish-Kenet

5
star
67

UD_Hindi_English-HIENCS

Python
5
star
68

UD_Uyghur-UDT

Uyghur data.
Perl
5
star
69

UD_Turkish-PUD

Parallel Universal Dependencies.
5
star
70

UD_Kazakh-KTB

5
star
71

UD_Estonian-EDT

Estonian data
5
star
72

UD_Latin-ITTB

Latin data from the Index Thomisticus Treebank.
5
star
73

UD_Welsh-CCG

4
star
74

UD_Latin-PROIEL

Latin data from the PROIEL treebank.
4
star
75

UD_Lithuanian-ALKSNIS

Lithuanian data from the Alksnis treebank.
4
star
76

UD_Catalan-AnCora

Catalan data from the AnCora corpus.
4
star
77

UD_Tagalog-TRG

Shell
4
star
78

UD_Wolof-WTB

4
star
79

UD_Kurmanji-MG

Northern Kurdish data.
4
star
80

UD_Yoruba-YTB

Shell
4
star
81

UD_Ancient_Greek-Perseus

4
star
82

UD_Japanese-PUD

Parallel Universal Dependencies.
4
star
83

UD_Romanian-Nonstandard

4
star
84

UD_Serbian-SET

Serbian data.
4
star
85

UD_Old_Church_Slavonic-PROIEL

Old Church Slavonic data from the PROIEL project.
4
star
86

UD_Hungarian-Szeged

Hungarian data
Shell
4
star
87

UD_Dutch-LassySmall

Wikipedia sample from the Lassy Small treebank.
4
star
88

UD_French-FTB

Data from the French Treebank.
4
star
89

UD_English-ParTUT

English part of the ParTUT parallel treebank.
4
star
90

UD_Japanese-KTC

Perl 6
3
star
91

UD_Karelian-KKPP

3
star
92

UD_Japanese-GSDLUW

Long-unit-word version of UD_Japanese-GSD
3
star
93

UD_Faroese-FarPaHC

Python
3
star
94

UD_Japanese-Modern

3
star
95

UD_Tupinamba-TuDeT

3
star
96

UD_Norwegian-Nynorsk

Nynorsk version of the Norwegian Dependency Treebank.
3
star
97

UD_Icelandic-IcePaHC

Python
3
star
98

UD_Latin-LLCT

3
star
99

UD_Javanese-CSUI

3
star
100

UD_Icelandic-PUD

3
star