Abstract
Astronomers were among the first to embrace the digital era. In the early 1970s, long before jpegs and gifs, and digital cameras in the hands of everyday consumers, astronomers were spending huge portions of their budgets on CCD (charge couple device) technology. Many claim that the development of the CCD was the second-most important technological breakthrough in astronomy (second only to the invention of the telescope itself). For astronomers, the advantage of digital imaging is far greater than convenience. Once armed with the CCD, and its ability to quantitatively measure the intensity of incoming light, to effectively count photons one by one; they dropped the photographic plate like a hot potato.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The standard is nearly 35 years old. Occasionally a rival appears. For example, proponents of "HDF" (hierarchical data format) proposed it as a FITS replacement in the 1990s, noting that the FITS standard provides no hierarchy (not a ‘property tree’) and its flat structure made for ugly, monolithic structure. This has forced users to encode hierarchy in the individual keyword names, which, given the 8-character length limitation, makes for a unique brand of ‘alphabet soup’: Entirely true, but, by all indications, FITS is here to stay; largely as a result of its very simple nature.
- 2.
Why 2,880? Many have wondered. It somehow dates back to tapes; those giant reel-to-reel computer tapes you see spinning in the background in black and white science fiction movies. Tapes required a blocking factor and the inventors of FITS, which for decades was a tape-only format, chose this number and noted (as stated in the abstract for ref [2]), “This (peculiar) size is rich in prime factors.” Why is that helpful? Suffice it to say that the reasoning was related to the ancient formats unique to reel-to-reel tapes and is irrelevant in today’s world of terabyte disks on laptops.
- 3.
Unsigned byte, unsigned long, float, and double indicated by \(\mathrm{BITPIX} = 8, 32,-32,\) and \(-64\), respectively.
- 4.
Or, more likely, read the pixels 2 or 4 bytes at a time, directly into variables of the appropriate type. Getting this part right (e.g., for C or C++ coming up with the correct casting) is the tricky part.
- 5.
Cubes are typically used for either the ‘spectral dimension’ of an IFU (see exercise 2 below) or for frames taken in rapid succession, for example, for speckle imaging or recording the frames from an AO WFS (which is described in the adaptive optics section at the end of the control systems chapter).
- 6.
Yes. Any software purist reading this now has full permission to wince. It is true: In IDL, like Python, csh, and a few other languages popular among astronomers, variable declaration is implicit.
- 7.
1-D is also possible to, for example, store the 8 bias values calibrated for an 8-channel CCD. Even though this is only 8 constants that need to be saved on disk somewhere, it is still better to use a short, 1-D, FITS file than to re-invent a new format.
- 8.
The data products coming out of these two phases are referred to formally in the world of NASA archiving as “level one” data and “level two” data (see [3]). The going in, “raw,” data is referred to as “level zero” (or sometimes called “un-calibrated”).
- 9.
Consult the index for sections within this text book that discuss each of these concepts if any of these terms are unfamiliar.
- 10.
Strictly speaking: As identical as they can be given useful data. See Fig. 1.
- 11.
Large asteroids like (41) Daphne can now be observed from earth via adaptive optics
- 12.
The DS9 grandfather, widely used in the 1990s, was called “SAOimage.” This was the image display tool provided by Smithsonian Astrophysical Observatory (SAO). The successor was called “SAO-TNG” standing for “the next generation.” Sticking with this trekkie theme, the authors went for the name given to the third Star Trek instantiation: Deep Space 9 (DS9 for short).
- 13.
For historical reasons, an image taken at visible wavelengths is usually referred to as a “CCD” image. The first digital images in astronomy were produced with the then novel “Charge Coupled Device” (CCD). Now that digital imaging is not so new, astronomers and engineers have gone back to using the sensitivity wave band (e.g., “infrared detector”). Hence forward in this text we will use the terms “CCD” and “infrared detector” to refer to these two fundamental types. The wavelength bands for the more exotic detector types (e.g., X-ray detectors, avalanche photo diodes, etc) are shown in Fig. 14 in the next chapter.
- 14.
For example, in IDL: the tvscl function; in java: methods in jawa.awt.image; or in the C/C++ library openGL: the ilLoadImage function.
- 15.
The term “re-bin” has taken on a life of its own in image processing. The term “bin” is short for “bilinear interpolation.” That mouthful simply means: interpolate in both dimensions, as is often required for 2-D image data. But, as in our usage here, it is now sometimes used in the case where, for example a \(256\times 256\) image must be represented as a \(1024\times 1024\) image by simply replicating each pixel as a \(4\times 4\) patch of pixels with the same value. This operation, more accurately referred to as “super-sampling” is, nonetheless, more commonly referred to as “re-binning.”
- 16.
The comparison given here reflects strongly the bias of the author. Any optimization of human factors engineering is best determined through exhaustive testing with a wide spectrum of users. In the end it will be a compromise. For the truly interested, reference [4] gives a suggested approach which, at the very least, gives good examples of what NOT to do in this area.
References
FITS - a Flexible Image Transport System, Don Wells et al, Astronomy and Astrophysics Supplement, 44 363, 1981.
FITS - a Self-Describing Table Interchange Format, Don Wells et al, NSF Workshop on Database Management, University of Virginia, 1990.
Report of the EOS Data Panel on the Data and Information System, R. Arvidson et al, NASA TM-87777, 1986.
The Inmates Are Running the Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity, Alan Cooper, Sams-Pearson Education, 2004.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2014 Albert R. Conrad
About this chapter
Cite this chapter
Conrad, A.R. (2014). Data and Data Archives. In: Software Systems for Astronomy. SpringerBriefs in Astronomy. Springer, New York, NY. https://doi.org/10.1007/978-1-4614-7058-8_4
Download citation
DOI: https://doi.org/10.1007/978-1-4614-7058-8_4
Published:
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4614-7057-1
Online ISBN: 978-1-4614-7058-8
eBook Packages: Physics and AstronomyPhysics and Astronomy (R0)