Sentence Pair Augmentation Approach for Grammatical Error Correction

Abstract

The deep learning model requires a large amount of data when learning a task. The sentence proofreading task requires the text before and after proofreading as the training data. Usually, most of the available publications are after proofreading and are easily accessible. However, the text before proofreading is rarely seen in the general publications and is highly difficult to obtain. In this study, we assume a case where we cannot prepare sufficient amounts of data for training a sentence proofreading task, such as work procedure manuals. We propose a method that automatically generates both pre- and post-proofreading sentences. We generate pseudo post-proofread sentences by Markov chains or GPT-3. The sentences generated by Markov chains are often semantically incorrect. We identify and remove these incorrect sentences by gated recurrent unit (GRU). Then, we generate pseudo pre-proofread sentences by adding noises to the pseudo post-proofread sentences using three different methods. In the experiments, we have compared the case where seq2seq-based grammatical error correction method is trained with a small corpus only, and the case where it is trained with the pseudo-sentence pairs generated in this study in addition to the small corpus. As a result, one of our methods improved the accuracy by 12.1% in the metric BLEU.