Last revised: Tuesday, 12-Nov-2013 16:55:37 EST
Justifications for 64-bit integer arrays in FITS files
The following cases illustrate scientific problems that could require arrays of
integer data with more than 32-bits of precision.
Arrays of time measurements where the time is measured as the number
of small clock 'ticks' from some reference time (e.g.,
the start times of a series of binary star eclipses).
Histograms generated from data bases of astronomical sources
containing more than 2^32 objects.
Rebinned or summed images from large mosaic CCD cameras that may result in
more than 2^32 total counts per binned pixel.
Similarly, cumulative histograms of 32-bit integer arrays may exceed 2^32.
Arrays of indices to individual pixels in large images (e.g., a list of
bad pixels). The indices
could involve non-rectangular pixelizations like HTM or HEALPix so that the total
number of pixels matters (they have a 1D index, instead of having separate
X and Y coordinate indices).
The precise measurement of time or relative position (using the number
of wavelengths of light as the yard stick) in emerging fields such
as gravitational wave astronomy and/or interferometry.
Numerical simulations that need more than 32-bits of dynamic range.
Arrays of encryption keys or other numeric codes containing more than 32-bits.
Practical or sociological justifications
The following arguments give various practical reasons for supporting
64-bit integer arrays in FITS files.
The 64-bit integer data type is available in virtually every modern
programming language. Programmers will inevitably make use of
this datatype (whether or not the added precision is strictly required)
and will sometimes want to store the results to a FITS file.
Having to convert the values to 32-bit integers or 64-bit floats
before writing them
to the FITS file adds complexity to the software and can lead to errors
and confusion if some of the 64-bit integer values are accidentally truncated.
A large segment of the astronomical research community uses FITS as the
on-line analysis format for all types of data files, including temporary
and/or intermediate numerical results. When data volume constraints are not
an issue, programmers may want to save integer results using the
data type with the most range and precision, especially if there is any
uncertainty about what precision is actually required.
In order to "future proof" applications it may be prudent to use
64-bit integers even if 32-bit integers seem adequate for current needs.
Software applications can end up being used for decades, at which
point it could be difficult to retrofit the software to
support 64-bit integers.
Most other scientific data formats support the 64-bit integer data
type, so FITS suffers by comparison. Scientific research is becoming more
interdisciplinary, so it is important to be able to import data files from
other fields such as space physics, planetary astronomy, and earth sciences
into FITS. This is made more difficult if FITS does not support the same
basic data types as the other data formats.
From an abstract "data model" point of view, a primary array or image
extension is identical to a table with 1 row and 1 column that contains the
image as a vector. Conceptually, one should be able to convert any table
vector into a FITS image array. Thus, if 64-bit integers
are to be supported in FITS tables, then that data type should logically
also be supported in FITS images.
From an implementation
point of view, once FITS I/O software have been upgraded to
support 64-bit integers in FITS tables, there is very little added
cost to support 64-bit integer images (because tables and images are
interchangeable, as described in the previous bullet).
"Build it, and they will come": One should not underestimate the
ingenuity of users and programmers in finding uses for new features. If
64-bit integer arrays are supported in FITS, users will discover
novel applications for them that were not anticipated.
Contact us: fits @ fits.gsfc.nasa.gov
Hosted by: The HEASARC
(High Energy Astrophysics Science Archive Research Center)
Responsible NASA representative: Dr. Alan P. Smale
Privacy, Security & Accessibility Statements.