Neural Model for Sentence Compression

Abstract

The neural sentence extraction model discussed in Chap. 3, as well as the ensemble, approaches in Chaps. 4 and 5, all solely focus on choosing a subset of sentences which gives the best ROUGE scores. However, like with any extractive techniques in general, these approaches have a limitation when generating a summary of fixed size. In the absence of reliable generative techniques, which can generate new concise sentences the next logical step is to eliminate redundant or less informative content from the extracted sentences. The two possible ways to achieve this is sentence compression and sentence simplification. While sentence compression solely deals with removing redundant information, sentence simplification usually looks into replacing a difficult phrase or word with a simpler alternative. In case of legal documents, usually replacing long legal phrases with more commonly used phrases also leads to sentence compression. This improves the precision of fixed-length summaries. We present a new approach for sentence compression for legal documents where we demonstrate how a phrase-based statistical machine translation system can be modified to generate meaningful sentence compressions. We compare this approach to an LSTM-based sentence compression technique. Next, we show how this problem can be modelled as a sequence to sequence mapping problem, thus not limiting to just deleting words, but also having a possibility of introducing new words in the target sentence.