Abstract
Many data compression systems rely on a final stage based on an entropy coder, generating short codes for the most probable symbols. Images, multispectroscopy or hyperspectroscopy are just some examples, but the space mission concept covers many other fields. In some cases, especially when the on-board processing power available is very limited, a generic data compression system with a very simple pre-processing stage could suffice. The Consultative Committee for Space Data Systems made a recommendation on lossless data compression in the early 1990s, which has been successfully used in several missions so far owing to its low computational cost and acceptable compression ratios. Nevertheless, its simple entropy coder cannot perform optimally when large amounts of outliers appear in the data, which can be caused by noise, prompt particle events, or artifacts in the data or in the pre-processing stage. Here we discuss the effect of outliers on the compression ratio and we present efficient solutions to this problem. These solutions are not only alternatives to the CCSDS recommendation, but can also be used as the entropy coding stage of more complex systems such as image or spectroscopy compression.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
J. Portell, E. García-Berro, X. Luri, and A. G. Villafranca, “Tailored data compression using stream partitioning and prediction: application to Gaia,” Experimental Astronomy 21, 125–149 (2006).
CCSDS-101.0-B-5 Blue Book, Telemetry channel coding, 2001.
D. Solomon, Data Compression. The complete reference, Springer, 2004.
CCSDS-121.0-B-1 Blue Book, Lossless data compression, 1993.
CCSDS-120.0-G-2 Informational Report, Lossless data compression, 2006.
R. F. Rice, “Some practical universal noiseless coding techniques,” JPL Technical Report 79–22 (1979).
S. W. Golomb, “Run-lengths encodings,” IEEE Transactions on Information Theory 12, 399–401 (1966).
P.-S. Yeh, “Implementation of CCSDS lossless data compression for space and data archive applications,” Proc. CCSDS Space Operations Conf., 60–69, 2002.
P.-S. Yeh, P. Armbruster, A. Kiely, B. Masschelein, G. Moury, C. Schaefer, and C. Thiebaut, “The new CCSDS image compression recommendation,” IEEE Aerospace Conf., 4138–4145, 2005.
M. Clotet, J. Portell, A. G. Villafranca, and E. García-Berro, “Simple resiliency improvement of the CCSDS standard for lossless data compression,” Proc. SPIE 7810, 2010.
J. Portell, A. G. Villafranca, and E. García–Berro, “Designing optimum solutions for lossless data compression in space,” Proc. ESA On-Board Payload Data Compression Workshop, 35–44, 2008.
J. Portell, A. G. Villafranca, and E. García-Berro, “A resilient and quick data compression method of prediction errors for space missions,” Proc. SPIE 7455, 2009.
P.-S. Yeh, R. Rice, and W. Miller, “On the optimality of code options for a universal noiseless coder,” JPL Technical Report 91–2 (1991).
D. Huffman, “A method for the construction of minimum redundancy codes,” Proc. IRE 40, 1098–1101 (1952).
I. H. Witten, R. M. Neal, and J. G. Cleary, “Arithmetic coding for data compression,” Communicat. ACM 30, 520–540 (1987).
Teuhola, J., “A compression method for clustered bit-vectors,” Information Processing Letters 7(6), 308–311 (1978).
Howard, P. and Vitter, J., “Fast progressive lossless image compression,” in Image and Video Compression Conference, SPIE, 98–109 (1994).
M. A. C. Perryman, K. S. de Boer, G. Gilmore, E. Hoeg, M. G. Lattanzi, L. Lindegren, X. Luri, F. Mignard, O. Pace, and P. T. Zeeuw, “Gaia: Composition, formation and evolution of the Galaxy,” Astronomy & Astrophysics 369, 339–363 (2001).
Nieto-Santisteban, M. A., Fixsen, D. J., Offenberg, J. D., Hanisch, R. J. & Stockman, H. S., “Data Compression for NGST”, in Astronomical Data Analysis Software and Systems VIII, vol. 172 of Astronomical Society of the Pacific Conference Series, 137–140 (1999).
C. E. Shannon, “A mathematical theory of communication,” Bell system technical journal, vol. 27, 1948.
A. Kiely and M. Klimesh, “Generalized Golomb codes and adaptive coding of wavelettransformed image subbands,” JPL Technical Report, IPN 42–154 (2003).
C. Babusiaux, “The Gaia Instrument and Basic Image Simulator,” in The Three-Dimensional Universe with Gaia, ESA SP-576, 125–149 (2005).
F. Murtagh and R. H.Warmels, “Test image descriptions,” in Proc. 1st ESO/ST-ECF Data Analysis Workshop, 17(6), 8–19 (1989).
Portell, J., Villafranca, A. G., and García-Berro, E., “Quick outlier-resilient entropy coder for space missions,” Journal of Applied Remote Sensing 4 (2010).
A. G. Villafranca, I. Mora, P. Ruiz-Rodríguez, J. Portell, and E. García-Berro, “Optimizing GPS data transmission using entropy coding compression”, Proc. SPIE 7810, 2010.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Portell, J., Villafranca, A.G., García-Berro, E. (2012). Outlier-Resilient Entropy Coding. In: Huang, B. (eds) Satellite Data Compression. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-1183-3_5
Download citation
DOI: https://doi.org/10.1007/978-1-4614-1183-3_5
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-1182-6
Online ISBN: 978-1-4614-1183-3
eBook Packages: EngineeringEngineering (R0)