Transforming the Natural Language Text for Improving Compression Performance
In the last 20 years, we have seen a vast explosion of textual information flow over the Web through electronic mail, Web browsing, information retrieval systems, and so on. The importance of data compression is likely to be enhanced in the future as there is a continuous increase in the amount of data that needs to be transformed or archived. In the field of data compression, researchers developed various approaches such as Huffman encoding, arithmetic encoding, Ziv— Lempel family, dynamic Markov compression, prediction with partial matching (PPM  and Burrows–Wheeler transform (BWT ) based algorithms, among others. BWT permutes the symbol of a data sequence that shares the same unbounded context by cyclic rotation followed by lexicographic sort operations. BWT uses move-to-front and an entropy coder as the backend compressor. PPM is slow and also consumes a large amount of memory to store context information but PPM achieves better compression than almost all existing compression algorithms.
KeywordsData Compression Compression Algorithm Entropy Coder Test Corpus Natural Language Text
Unable to display preview. Download preview PDF.
- 2.M. Burrows and D. Wheeler (1994) A block-sorting lossless data compression algorithm. Technical Report, SRC Research Report 124, Digital Systems Research Center, Palo Alto, CA.Google Scholar
- 3.F.S. Awan and A. Mukherjee (2001) LIPT: A lossless text transform to improve compression. In Proceedings of International Conference on Information and Theory: Coding and Computing, Las Vegas, Nevada, IEEE Computer Society.Google Scholar
- 4.R. Franceschini and A. Mukherjee (1996) Data compression using encrypted text. In Proceedings of the Third Forum on Research and Technology, Advances on Digital Libraries, 130–138. ADL.Google Scholar
- 6.M.D. Araujo, G. Navaaro, and N. Ziviani (1997) Large text searching allowing errors. In Proceedings of the 4th South American Workshop on String Processing. R. Baeza-Yates, Ed. Carleton University Press International Informatics Series, vol. 8. Carleton University Press, Ottawa, Canada, 2–20.Google Scholar
- 7.E.S. Moura, G. Navarro, and N. Ziviani (1997) Indexing Compressed text. In Proceedings of the 4th South American Workshop on String Processing. R. Baeza-Yates, Ed. Carleton University Press International Informatics Series, vol. 8. Carleton University Press, Ottawa, Canada, 95–111.Google Scholar