Abstract
This chapter reviews and summarizes the researches and developments on data compression techniques for satellite sensor data at the Canadian Space Agency in collaboration with its partners in other government departments, academia and Canadian industry. This chapter describes the subject matters in the order of the following sections.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
S.-E. Qian, A. Hollinger, D. Williams and D. Manak, “Fast 3D data compression of hyperspectral imagery using vector quantization with spectral-feature-based binary coding,” Opt. Eng. 35, 3242–3249 (1996) [doi:10.1117/1.601062]
S.-E. Qian, A. Hollinger, D. Williams and D. Manak, “A near lossless 3-dimensional data compression system for hyperspectral imagery using correlation vector quantization,” Proc. 47th Inter. Astron. Congress, Beijing, China (1996).
S.-E. Qian, A. Hollinger, D. Williams and D. Manak, “3D data compression system based on vector quantization for reducing the data rate of hyperspectral imagery,” in Applications of Photonic Technology II, G. Lampropoulos, Ed., pp. 641–654, Plenum Press, New York (1997)
D. Manak, S.-E. Qian, A. Hollinger and D. Williams, “Efficient Hyperspectral Data Compression using vector Quantization and Scene Segmentation,” Canadian J. Remote Sens., 24, 133–143 (1998).
S.-E. Qian, A. Hollinger, D. Williams and D. Manak, “3D data compression of hyperspectral imagery using vector quantization with NDVI-based multiple codebooks,” Proc. IEEE Geosci. Remote Sens. Symp., 3, 2680–2684 (1998).
S.-E. Qian, A. Hollinger, D. Williams and D. Manak, “Vector quantization using spectral index based multiple sub-codebooks for hyperspectral data compression,” IEEE Trans. Geosci. Remote Sens., 38(3), 1183–1190 (2000) [doi:10.1109/36.843010]
S.-E. Qian, “Hyperspectral data compression using a fast vector quantization algorithm,” IEEE Trans. Geosci. Remote Sens., 42(8), 1791–1798 (2004) [doi: 10.1109/TGRS.2004.830126]
S.-E. Qian, “Fast vector quantization algorithms based on nearest partition set search,” IEEE Trans. Image Processing, 15(8), 2422–2430 (2006) [doi:10.1109/TIP.2006.875217]
S.-E. Qian and A. Hollinger, “Current Status of Satellite Data Compression at Canadian Space Agency,” Invited chapter in Proc. SPIE 6683, 04.01-12 (2007).
S.-E. Qian, and A. Hollinger, “System and method for encoding/decoding multi-dimensional data using Successive Approximation Multi-stage Vector Quantization (SAMVQ),” U. S. Patent No. 6,701,021 B1, issued on March 2, 2004.
S.-E. Qian and A. Hollinger, “Method and System for Compressing a Continuous Data Flow in Real-Time Using Cluster Successive Approximation Multi-stage Vector Quantization (SAMVQ),” U.S. Patent No. 7,551,785 B2, issued on June 23, 2009.
S.-E. Qian, and A. Hollinger, “System and method for encoding multi-dimensional data using Hierarchical Self-Organizing Cluster Vector Quantization (HSOCVQ),” U. S. Patent No. 6,724,940 B1, issued on April 20, 2004.
S.-E. Qian and A. Hollinger, “Method and System for Compressing a Continuous Data Flow in Real-Time Using Recursive Hierarchical Self-Organizing Cluster Vector Quantization (HSOCVQ),” U.S. Patent No. 6,798,360 B1 issued on September 28, 2004.
S.-E. Qian, M. Bergeron, I. Cunningham, L. Gagnon and A. Hollinger, "Near Lossless Data Compression On-board a Hyperspectral Satellite," IEEE Trans. Aerosp. Electron. Syst., 42(3), 851-866 (2006) [doi:10.1109/TAES.2006.248183]
S.-E. Qian, A. Hollinger and Yann Hamiaux, “Study of real-time lossless data compression for hyperspectral imagery,” Proc. IEEE Int. Geosci. Remote Sens. Symp., 4, 2038–2042 (1999)
S-E Qian and A Hollinger, “Applications of wavelet data compression using modified zerotrees in remotely sensed data,” Proc. IEEE Geosci. Remote Sens. Symp., 6, 2654–2656 (2000)
CCSDS, “Lossless Data Compression,” CCSDS Recommendation for Space Data System Standards, Blue Book 120.0-B-1, May 1997.
S.-E. Qian, “Difference Base-bit Plus Overflow-bit Coding,” Journal of Infrared & Millimeter Waves, 11(1), 59–64 (1992).
J. M. Shapiro, "Embedded image coding using zerotrees of wavelet coefficients," IEEE Trans. Signal Processing, 41, 3445–3462 (1993).
A. Said and W.A. Pearlman, “A new, fast and efficient image codec based on set partitioning in hierarchical trees,” IEEE Trans. on Circuits and Systems for Video Technology, 6, 243–250 (1996).
A. Gersho and R.M. Gray, Vector Quantization and Signal Compression, Boston, MA: Kluwer, 1992.
K. Chen and T.V. Ramabadran, “Near-lossless compression of medical images through entropy-coded DPCM,” IEEE Trans. Med. Imaging 13, 538–548 (1994) [doi:10.1109/42.310885]
X. Wu and P. Bao, “Constrained high-fidelity image compression via adaptive context modeling,” IEEE Trans. Image Processing, 9, 536–542 (2000) [doi:10.1109/83.841931]
B Aiazzi, L. Alparone and S. Baronti, “Near-lossless compression of 3-D optical data,” IEEE Trans. Geosci. Remote Sens., 39(11), 2547–2557 (2001) [doi:10.1109/36.964993]
E. Magli, G. Olmo and E. Quacchio, “Optimized onboard loeelsess and near-lossless compression of hyperspectral data using CALIC,” IEEE Geosci. Remote Sens. Lett., 1, 21–25 (2004) [doi:10.1109/LGRS.2003.822312]
Low altitude AVIRIS data of the Greater Victoria Watershed District, http://aviris.jpl.nasa.gov/ql/list02.html
A. Hollinger, M. Bergeron, M. Maszkiewicz, S.-E. Qian, K. Staenz, R.A. Neville and D.G. Goodenough, “Recent Developments in the Hyperspectral Environment and Resource Observer (HERO) Mission” Proc. IEEE Geosci. Remote Sens. Symp., 3, 1620–1623 (2006).
S.-E. Qian, M. Bergeron, A. Hollinger and J. Lévesque, “Effect of Anomalies on Data Compression Onboard a Hyperspectral Satellite”, Proc. SPIE 5889, (02)1–11 (2005)
S.-E. Qian, M. Bergeron, J. Lévesque and A. Hollinger, “Impact of Pre-processing and Radiometric Conversion on Data Compression Onboard a Hyperspectral Satellite,” Proc. IEEE Geosci. Remote Sens. Symp., 2, 700–703 (2005).
S.-E. Qian, B. Hu, M. Bergeron, A. Hollinger and P. Oswald, “Quantitative evaluation of hyperspectral data compressed by near lossless onboard compression techniques,” Proc. IEEE Geosci. Remote Sens. Symp., 2, 1425–1427 (2002).
S.-E. Qian, A. Hollinger, M. Bergeron, I. Cunningham, C. Nadeau, G. Jolly and H. Zwick, “A Multi-disciplinary User Acceptability Study of Hyperspectral Data Compressed Using Onboard Near Lossless Vector Quantization Algorithm,” Inter. J. Remote Sens., 26(10), 2163–2195 (2005) [doi:10.1080/01431160500033500]
C. Nadeau, G. Jolly and H. Zwick, “Evaluation of user acceptance of compression of hyperspectral data cubes (Phase I),” final report of Canadian Government Contract No. 9F028-013456/001MTB, Feb. 6, 2003.
C. Nadeau, G. Jolly and H. Zwick, “Evaluation of user acceptance of compression of hyperspectral data cubes (Phase II),” final report of Canadian Government Contract No. 9F028-013456/001MTB, July 24, 2003.
S.-E. Qian, A. B. Hollinger, M. Dutkiewicz and H. Zwick, “Effect of Lossy Vector Quantization Hyperspectral Data Compression on Retrieval of Red Edge Indices,” IEEE Trans. Geosc. Remote Sens., 39, 1459–1470 (2001) [doi:10.1109/36.934077]
S.-E. Qian, M. Bergeron, C. Serele and A. Hollinger, “Evaluation and comparison of JPEG 2000 and VQ based on-board data compression algorithm for hyperspectral imagery,” Proc. IEEE Geosci. Remote Sens. Symp., 3, 1820–1822 (2003).
B. Hu, S.-E. Qian and A. Hollinger, “Impact of lossy data compression using vector quantization on retrieval of surface reflectance from CASI imaging spectrometry data,” Canadian J. Remote Sens., 27, 1–19 (2001).
B. Hu, S.-E. Qian, D. Haboudane, J.R. Miller, A. Hollinger and N. Tremblay, “Impact of Vector Quantization Compression on Hyperspectral Data in the Retrieval Accuracies of Crop Chlorophyll Content for Precision Agriculture,” Proc. IEEE Geosci. Remote Sens. Symp., 3, 1655–1657 (2002).
B. Hu, S.-E. Qian, D. Haboudane, J. R. Miller, A. Hollinger and N. Tremblay, “Retrieval of crop chlorophyll content and leaf area index from decompressed hyperspectral data: the effects of data compression,” Remote Sens. Environ., 92(2), 139–152 (2004) [doi:10.1016/j.rse.2004.05.009]
K. Staenz, R. Hitchcock, S.-E. Qian and R.A. Neville, “Impact of on-board hyperspectral data compression on mineral mapping products,” Int. ISPRS Conf. 2002, India (2002).
K. Staenz, R. Hitchcock, S.-E. Qian, C. Champagne and R.A. Neville, “Impact of On-Board Hyperspectral Data Compression on Atmospheric Water Vapour and Canopy Liquid Water Retrieval,” Int. ISSSR Conf. (2003).
C. Serele, S.-E. Qian, M. Bergeron, P. Treitz, A. Hollinger and F. Cavayas, “A Comparative Analysis of two Compression Algorithms for Retaining the Spatial Information in Hyperspectral Data,” Proc. 25 th Canadian Remote Sens. Symp., Montreal, Canada, 14–16 (2003).
A. Dyk, D.G. Goodenough, S. Thompson, C. Nadeau, A. Hollinger and S.-E. Qian, “Compressed Hyperspectral Imagery for Forestry,” Proc. IEEE Geosci. Remote Sens. Symp. 1, 294–296 (2003).
P. Zarrinkhat and S.-E. Qian, “Enhancement of Resilience to Bit-Errors of Compressed Data On-board a Hyperspectral Satellite using Forward Error Correction” Proc. SPIE 7084, 07.1–9 (2008)
S.-E. Qian, A. Hollinger and L. Gagnon, “Data Compression Engines and Real-Time Wideband Compressor for Multi-Dimensional Data” U.S. Patent No. 7,251,376 B2, issued on July 31, 2007.
Consultative Committee for Space Data System (CCSDS), http://public.ccsds.org/default.aspx.
S.-E. Qian, “Study of hyperspectral and multispectral images compression using vector quantization in development of CCSDS international standards” Proc. SPIE 7477A, 23.1–11 (2009)
M. Bergeron, K. Kolmaga and S.-E. Qian “Assessment of Keystone Impact on VQ Compression Fidelity,” Canadian Space Agency internal technical report on May 20th, 2003.
M. Bergeron, K. Kolmaga and S.-E. Qian “Assessment of Spectral Curvature Impact on VQ Compression Fidelity,” Canadian Space Agency internal technical report on May 20th, 2003.
Acknowledgments
The author would like to thank his colleagues A. Hollinger, M. Bergeron, I. Cunningham and M. Maszkiewicz; his post-doctor visiting fellows C. Serele, H. Othman and P. Zarrinkhat, and over 30 internship students, for their contributions to the work summarized in this chapter. The author thanks D. Goodenough at the Pacific Forestry Centre, Natural Resources Canada, K. Staenz (now at University of Lethbridge), L. Sun and R. Neville at the Canada Center for Remote Sensing, Natural Resources Canada, J. Levesque and J.-P. Ardouin at the Defence Research and Development Canada, J. Miller and B. Hu at York University, for providing data sets and for actively collaborating on the user acceptability study and the impact assessments. The author thanks the following users for their participation and contribution to the multi-disciplinary user acceptability study: A. Dyk at the Pacific Forestry Centre, B. Ricketts and N. Countway at Satlantic Inc., J. Chen at University of Toronto, H. Zwick, C. Nadeau, G. Jolly, M. Davenport and J. Busler at MacDonald Dettwiler Associates (MDA), M. Peshko at Noranda/Falconbridge, B. Rivard and J. Feng at the University of Alberta, J. Walls and R. McGregor at RJ Burnside International Ltd., M. Carignan and P. Hebert at Tecsult, J. Huntington and M. Quigley at the Commonwealth Scientific and Industrial Research Organization in Australia, R. Hitchcock at the Canada Centre for Remote Sensing. The author thanks L. Gagnon, W. Harvey, B. Barrette, and C. Black at former EMS Technologies Canada Ltd. (Now MDA Space Missions) and the technical teams for the development and fabrication of the hardware compression prototypes. The author thanks V. Szwarc and M. Caron at Communication Research Centre, Canada for discussion on enhancement of the resilience to bit-errors of the compression techniques, and P. Oswald and R. Buckingham for discussion on on-board data compression. The author also thanks the CCSDS MHDC Working Group for providing the test data sets and the members of the working group for providing the compression results. The Work in the chapter was created by a public servant acting in the course of his employment for the Government of Canada and within the scope of his duties in writing the Work, whereas the copyright in the Work vests in Her Majesty the Queen in right of Canada for all intents and purposes under the Copyright Act of Canada© Government of Canada 2011.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer Science+Business Media, LLC
About this chapter
Cite this chapter
Qian, SE. (2012). Development of On-Board Data Compression Technology at Canadian Space Agency. In: Huang, B. (eds) Satellite Data Compression. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-1183-3_1
Download citation
DOI: https://doi.org/10.1007/978-1-4614-1183-3_1
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-1182-6
Online ISBN: 978-1-4614-1183-3
eBook Packages: EngineeringEngineering (R0)