CoNLL-2013 Shared Task: Grammatical Error Correction
CoNLL-2013 will continue the CoNLL tradition of having a high profile shared task in natural language processing. This year's shared task will be grammatical error correction. A participating system in this shared task is given short English texts written by non-native speakers of English. The system detects the grammatical errors present in the input texts, and returns the corrected texts.
This task has gained popularity recently with the organization of the HOO (Helping Our Own) shared tasks in 2011 and 2012. In the most recent HOO shared task in 2012, on error detection and correction of determiners and prepositions, 14 teams from around the world participated and 85 systems were submitted to the shared task.
The grammatical error correction task is impactful since it is estimated that hundreds of millions of people in the world are learning English and they benefit directly from an automated grammar checker. However, for many error types, current grammatical error correction methods do not achieve a high performance and thus more research is needed.
Instead of focusing on only determiner and preposition errors as in HOO 2012, the CoNLL-2013 shared task will include a more comprehensive list of error types, including determiner, preposition, noun number, verb form, and subject-verb agreement errors. Extending into more error types introduces the possibility of correcting multiple interacting errors. Examples of such interacting errors include determiner and noun number ('that cars' → 'that car' or 'those cars') and preposition and verb form ('an interest to study’ → ‘an interest in studying').
Participating teams will be provided with common training data in which grammatical errors have been annotated. Blind test data will be used to evaluate the outputs of the participating teams using a common scoring software and evaluation metric.
Registration for the shared task has begun. Please email the following information to firstname.lastname@example.org: name, email address, and affiliation of the members of a participating team, with one team member designated as the contact person. Upon registration, each participating team will receive the training data and scorer and join a discussion group for the shared task. A signed license agreement is needed for receipt of the training data. A team can register at anytime before the registration deadline, but early registration is highly encouraged, since details of the shared task will be announced on the discussion group. 54 teams have registered to participate in the shared task this year.
- December 17, 2012: announcement of shared task
- January 17, 2013: set up of shared task website
- February 5, 2013: registration begins and release of training set and scorer
- March 5, 2013: registration deadline
- April 28, 2013: test set available
- May 1, 2013: systems' outputs collected
- May 8, 2013: system results due to participants
- May 15, 2013: shared task system papers due
- May 24, 2013: reviews due
- May 27, 2013: notification of acceptance
- June 3, 2013: camera ready version of shared task system papers due
- August 8-9, 2013: CoNLL-2013 conference (Sofia, Bulgaria)
Shared Task Organizers
- Hwee Tou Ng (Chair), National University of Singapore
- Joel Tetreault, Nuance Communications
- Siew Mei Wu, National University of Singapore
- Yuanbin Wu, National University of Singapore
- Christian Hadiwinoto, National University of Singapore
Questions about the CoNLL-2013 shared task can be sent to email@example.com.