68

SECTION 7. STANDARD EXTENSIONS

is allocated in each row for the array data--whether it is used or not. This is fine so

long as the arrays are small or a fixed amount of array data will be stored in each field,

but if the stored array length varies for different rows, it is necessary to impose a fixed

upper limit on the size of the array that can be stored. If this upper limit is made

too large excessive wasted space can result and the binary table mechanism becomes

seriously inefficient. If the limit is set too low then storing certain types of data in the

table could become impossible.

The variable-length array construct presented here was devised to deal with this

problem. Variable-length arrays are implemented in such a way that, even if a table

contains such arrays, a simple reader program that does not understand variable-length

arrays will still be able to read the main data table (in other words a table containing

variable-length arrays conforms to the basic binary table standard). The implementation

chosen is such that the rows in the main data table remain fixed in size even if the table

contains a variable-length array field, allowing efficient random access to the main data

table.

Variable-length arrays are logically equivalent to regular static arrays, the only dif-

ferences being 1) the length of the stored array can differ for different rows, and 2) the

array data are not stored directly in the main data table. Since a field of any data

type can be a static array, a field of any data type can also be a variable-length array

(excluding the type P and Q variable-length array descriptors themselves, which are not

a data type so much as a storage class specifier). Other established FITS conventions

that apply to static arrays will generally apply as well to variable-length arrays.

A variable-length array is declared in the table header with one of the following 2

special field data type specifiers

rPt(emax)

rQt(emax)

where the `P' or `Q' indicates the presence of an array descriptor (described below),

the element count r should be 0, 1, or absent, t is a character denoting the data type of

the array data (L, X, B, I, J, K, etc., but not P or Q), and emax is a quantity guaranteed to

be equal to or greater than the maximum number of elements of type t actually stored

in any row of the table. There is no built-in upper limit on the size of a stored array

(other than the fundamental limit imposed by the range of the array descriptor, defined

below); emax merely reflects the size of the largest array actually stored in the table,

and is provided to avoid the need to preview the table when, for example, reading a

table containing variable-length elements into a database that supports only fixed size

arrays. There may be additional characters in the TFORMn keyword following the e  max.

For example,

TFORM8 = 'PB(1800)' / Variable byte array

FITS Standard