From fitsbits-request Fri Jul 1 03:57:13 1994 X-VM-Message-Order: (1 2 4 5 6 24 25 3 31 27 26 14 15 18 17 22 23 16 28 29 30 32 19 33 34 35 36 41 37 38 40 39 7 9 42 12 13 44 43 20 45 47 49 50 10 21 11 46 8 51 52 48 53 54 55 56 57 60 58 59) X-VM-Summary-Format: "%n %*%a %-17.17F %-3.3m %2d %4l/%-5c %I\"%s\"\n" X-VM-Labels: nil X-VM-VHeader: ("Resent-" "From:" "Sender:" "To:" "Apparently-To:" "Cc:" "Subject:" "Date:") nil X-VM-Bookmark: 60 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["499" "" "30" "June" "1994" "00:01:05" "-0400" "SteveS1825" "steves1825 at aol.com" nil "17" "Re: FITS on Mac via NIH Image Mods" "^From:" nil nil "6" nil nil (number " " mark " SteveS1825 Jun 30 17/499 " thread-indent "\"Re: FITS on Mac via NIH Image Mods\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA00758; Fri, 1 Jul 94 03:57:13 EDT Return-Path: Message-Id: <2utg21$cvd at search01.news.aol.com> Organization: America Online, Inc. (1-800-827-6364) Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!news.dfn.de!news.dfn.de!Germany.EU.net!EU.net!uunet!newstf01.cr1.aol.com!search01.news.aol.com!not-for-mail References: <28JUN199408331456 at nssdca.gsfc.nasa.gov> From: steves1825 at aol.com (SteveS1825) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS on Mac via NIH Image Mods Date: 30 Jun 1994 00:01:05 -0400 In article <28JUN199408331456 at nssdca.gsfc.nasa.gov>, bschlesinger at nssdca.gsfc.nasa.gov (BARRY M. SCHLESINGER) writes: Does it have limits on the values of NAXIS and BITPIX it will accept? Barry Schlesinger NOST FITS Support Office Barry, I don't know, I have yet to obtain access to FITS format images apart from the handful that came from the program, so I have yet to come near to testing it to its' limits. I would say that it was definitely worth taking a look at. From fitsbits-request Sat Jul 2 23:35:45 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1399" "Sun" " 3" "July" "1994" "03:34:19" "GMT" "Patrick P. Murphy" "pmurphy at nrao.edu" nil "33" "Re: FITS on Mac via NIH Image Mods" "^From:" nil nil "7" nil nil (number " " mark " Patrick P. Murphy Jul 3 33/1399 " thread-indent "\"Re: FITS on Mac via NIH Image Mods\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA04257; Sat, 2 Jul 94 23:35:45 EDT Return-Path: Message-Id: Organization: National Radio Astronomy Observatory Path: saips.cv.nrao.edu!news!pmurphy References: <28JUN199408331456 at nssdca.gsfc.nasa.gov> <2utg21$cvd at search01.news.aol.com> From: pmurphy at nrao.edu (Patrick P. Murphy) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS on Mac via NIH Image Mods Date: Sun, 3 Jul 1994 03:34:19 GMT (attribution corrected) In article <2utg21$cvd at search01.news.aol.com>, steves1825 at aol.com (SteveS1825) writes: > In article <28JUN199408331456 at nssdca.gsfc.nasa.gov>, > bschlesinger at nssdca.gsfc.nasa.gov (BARRY M. SCHLESINGER) writes: >> Does it have limits on the values of NAXIS and BITPIX it will accept? > I don't know, I have yet to obtain access to FITS format images > apart from the handful that came from the program, so I have yet to > come near to testing it to its' limits. If AOL gives you the ability to use ftp, try the following: fits.cv.nrao.edu (192.33.115.8) /fits/sampledata/image/ info.cv.nra.edu (192.33.115.103) /pub/aips/DDT/15JAN94/ These directories contain many samples of test fits files. If you explore the fits server, you'll find even more. - Pat -- ============================================================================= Patrick P. Murphy, Ph.D. Scientific Programming Analyst National Radio Astronomy Observatory pmurphy at nrao.edu 520 Edgemont Road Tel: (804) 296-0372 Charlottesville, VA 22903-2475 Fax: (804) 296-0278 web: http://orangutan.cv.nrao.edu/ "I don't believe in the no-win scenario" --- James T. Kirk ============================================================================= From fitsbits-request Mon Jul 4 20:11:27 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["894" "Mon" " 4" "July" "1994" "21:48:35" "GMT" "Stefan Mochnacki" "stefan at helios.physics.utoronto.ca" nil "18" "Need FITS --> uuencoded in < 64KB chunks utility." "^From:" nil nil "7" nil nil (number " " mark " Stefan Mochnacki Jul 4 18/894 " thread-indent "\"Need FITS --> uuencoded in < 64KB chunks utility.\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA08830; Mon, 4 Jul 94 20:11:27 EDT Return-Path: Message-Id: Organization: University of Toronto Physics/Astronomy/CITA Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!howland.reston.ans.net!cs.utexas.edu!utnut!helios.physics.utoronto.ca!stefan From: stefan at helios.physics.utoronto.ca (Stefan Mochnacki) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Need FITS --> uuencoded in < 64KB chunks utility. Date: Mon, 4 Jul 1994 21:48:35 GMT We are starting to get more requests to send occasional images via e-mail. This sounds silly, I know, but we have had some dreadful problems with people e-mailing giant files. Many machines don't have anonymous ftp servers, so rapid file transfer can be difficult. My question therefore is: can somebody point to a utility which takes a given file (preferably FITS) and breaks it up into uuencoded chunks which can be easily reassembled, the chunks being no more than 64KB each , preferably mailing the darn things as well ? (I don't particularly want a utility which people can use on Usenet images ...). -- Stefan W. Mochnacki INTERNET - stefan at centaur.astro.utoronto.ca Astronomy, U. of Toronto UUCP - {uunet,pyramid}!utai!helios.physics!stefan Ph. (905) 884-9562 LOCATION - David Dunlap Observatory FAX (905) 884-2672 Ph. (Mon,Wed) (416)-978-4165 (St.George Campus) From fitsbits-request Tue Jul 5 10:59:01 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["3750" "Tue" " 5" "July" "1994" "10:58:54" "-0400" "William Pence" "pence at tetra.gsfc.nasa.gov" nil "88" "FITS TSORTKEY keyword" "^From:" nil nil "7" nil nil (number " " mark " William Pence Jul 5 88/3750 " thread-indent "\"FITS TSORTKEY keyword\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA09435; Tue, 5 Jul 94 10:59:01 EDT Return-Path: Message-Id: <199407051458.KAA00475 at tetra.gsfc.nasa.gov> From: William Pence Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Cc: pence at tetra.gsfc.nasa.gov Subject: FITS TSORTKEY keyword Date: Tue, 5 Jul 1994 10:58:54 -0400 Last year there was some discussion on this newsgroup of a proposal for a FITS keyword to indicate the order in which a FITS table has been sorted. The OGIP FITS Working Group has recently readdressed this issue and is proposing that the 'TSORTKEY' keyword be used for this purpose as described below. Any comments or criticisms about this are welcome. Bill Pence Ofice of Guest Investigator Programs NASA/GSFC ----------------------------------------------------------------------- TSORTKEY: a FITS Keyword to Specify the Sort Order of a Table The TSORTKEY keyword is used to indicate the order in which the rows in a FITS TABLE or BINTABLE extension have been sorted. The value of the TSORTKEY keyword is a character string which lists the name (as given by the TTYPEn keyword) of the primary sort column, followed by the names of any (optional) secondary sort column(s). The rows in the table are sorted first by the value in the primary column; any rows which have the same value in the primary column are further sorted by the value in the secondary column and so on for all the specified sort columns. If more than one column is specified by TSORTKEY then the names must be separated by a comma. One or more spaces are also allowed between the comma and the following column name. By default, columns are sorted in ascending order, but a minus sign may precede the column name to indicate the rows are sorted in descending order. (Note that this implies that the names of any columns used to sort a table must not begin with a minus sign). Integer or floating point columns are sorted by numerical order and not by their internal ASCII representation in the case of TABLE extensions. Character columns are sorted by the ASCII collating sequence order of the entire string by default. If the table has been sorted on a substring within the character column, this may be indicated by including the range of the substring characters in parentheses after the column name (e.g., 'NAME(3:8)' indicates that the rows of the table are sorted based on the 3rd through 8th characters in the NAME character column). The sort order for logical format columns (L) in BINTABLE extensions is defined such that all false values (F) will precede the true values (T) when sorted in ascending order. The sort order of complex data type columns ('C' or 'M') is not defined by this convention. In the case of vector columns in binary tables, the TSORTKEY convention only supports sorting on the value of a particular element within the vector (except for character string vectors as described above). The vector element number that was used to sort the table is included in parentheses following the vector column name. If no element is explicitly specified it is assumed that the table is sorted on the first element in the vector. Any null/undefined elements in a sort column will appear after all the defined values when the table is sorted in ascending order. Undefined elements will precede all defined elements when sorted in descending order. Examples of TSORTKEY usage: TSORTKEY = 'X ' The table is sorted in ascending value of the X column. TSORTKEY = 'X,Y ' The table is sorted in ascending value of the X column. Rows which have the same value of X have been further sorted in ascending value of the Y column. TSORTKEY = '-TIME ' The table is sorted in descending value of the TIME column. TSORTKEY = '-OBJECT(2:4)' The table is sorted in reverse alphabetical order on the 2nd through 4th characters in the OBJECT character string column. TSORTKEY = 'ARRAY(10)' The table is sorted in ascending value of the 10th element of the ARRAY vector column. From fitsbits-request Tue Jul 5 12:45:57 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2705" "Tue" " 5" "July" "1994" "12:45:51" "-0400" "William Pence" "pence at tetra.gsfc.nasa.gov" nil "57" "FITS continuation keywords" "^From:" nil nil "7" nil nil (number " " mark " William Pence Jul 5 57/2705 " thread-indent "\"FITS continuation keywords\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA09538; Tue, 5 Jul 94 12:45:57 EDT Return-Path: Message-Id: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> From: William Pence Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Cc: pence at tetra.gsfc.nasa.gov Subject: FITS continuation keywords Date: Tue, 5 Jul 1994 12:45:51 -0400 Last summer there was considerable debate in this newgroup about a proposed convention for encoding very long character strings as the value of FITS keywords. We will soon need to begin using this convention (or some variant on it) for the FITS files generated by the XTE mission. The main purpose of this posting is to see if there is any concensus within the FITS communitiy as to which of 2 possible variations on this convention we should use. Our orginal (and still preferred) proposal is to use a backslash character (\) at the end of a string to indicate that is continued on the next keyword, and to have columns 1=10 of the following keyword(s) be filled with blanks. FITS readers that do not understand this continuation convention would just treat the continuation lines as comment keywords (since the keyword name is blank). For example: LONGSTR = 'This is a very long string keyword value that \' 'is continued over three\' 'keywords in the FITS header.' One objection that was raised to this convention was that some FITS readers apparently treat comment keywords differently from the other keywords that have a value, and the keywords might get separated, thus destroying the order of the keywords that is needed to reconstruct the long string value. One way to solve this particular problem (if it really is a problem) is to replace the blank keyword name with a reserved keyword such as 'CONTINUE' followd by an equal sign, as in: LONGSTR = 'This is a very long string keyword value that \' CONTINUE= 'is continued over three\' CONTINUE= 'keywords in the FITS header.' Is there any preference as to which of these 2 conventions we use? If not we will probably use the original proposal, with blank continuation keyword names. Incidentally, to preempt a common suggestion, it is not possible to use a numbered sequence of keywords for this convention, as in: LONGS1 = 'This is a very long string keyword value that \' LONGS2 = 'is continued over three\' LONGS3 = 'keywords in the FITS header.' because we need to use this convention in FITS tables where the keyword name already has a sequence number attached to it to indicate which column of the table it refers to. So the convention that we adopt must place no restrictions on the name of the long string keyword itself. Finally, we realize that this is a controversial proposal, and thus will restrict its use to rather mission-specific keywords that should minimize the impact on other data systems that don't understand this convention. In most cases other data systems would not know what to do with these long string values even if they could read them. -Bill Pence OGIP, NASA/GSFC From fitsbits-request Tue Jul 5 14:59:03 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1467" "Tue" " 5" "July" "1994" "14:55:06" "-0400" "Nikolai Piskunov" "piskunov at phobos.astro.uwo.ca" nil "45" "Re: FITS continuation keywords" "^Resent-Date:" nil nil "7" nil nil (number " " mark " Nikolai Piskunov Jul 5 45/1467 " thread-indent "\"Re: FITS continuation keywords\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA09725; Tue, 5 Jul 94 14:59:03 EDT Return-Path: Reply-To: Nikolai Piskunov In-Reply-To: <01HECXU36O9E9GVRIG at hylk.Helsinki.FI> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; CHARSET=US-ASCII Resent-Date: Tue, 5 Jul 1994 14:55:06 -0400 (DST) Resent-From: fitsbits-request Resent-Message-Id: <9407051859.AA09725 at fits.cv.nrao.edu> Resent-Sender: Nikolai Piskunov From: Nikolai Piskunov Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Tue, 5 Jul 1994 14:55:06 -0400 (DST) On Tue, 5 Jul 1994, William Pence wrote: > ... > Last summer there was considerable debate in this newgroup about a > proposed convention for encoding very long character strings as the > value of FITS keywords. > ... > For example: > > LONGSTR = 'This is a very long string keyword value that \' > 'is continued over three\' > 'keywords in the FITS header.' > > ... > or to replace the blank keyword name with a > reserved keyword such as 'CONTINUE' followd by an equal sign, as in: > > LONGSTR = 'This is a very long string keyword value that \' > CONTINUE= 'is continued over three\' > CONTINUE= 'keywords in the FITS header.' > > Is there any preference as to which of these 2 conventions we use? > I feel that the idea of using \' combination may cause problems for FITS readers written in C. It may also result in confusion, if the text of the comment is interpreted after extraction with a shell script. On the other hand second example in WP mail shows certain redundancy. So what if we keep the keyword CONTINUE and forget the back slash? In other words, the example above will look like: LONGSTR = 'This is a very long string keyword value that ' CONTINUE= 'is continued over three ' CONTINUE= 'keywords in the FITS header.' Dr. Nikolai Piskunov Department of Astronomy University of Western Ontario London, Ontario N6A 3K7, Canada From fitsbits-request Tue Jul 5 19:37:15 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1062" "Tue" " 5" "July" "1994" "20:37:51" "GMT" "Rob Seaman" "seaman at noao.edu" nil "27" "Re: FITS continuation keywords" "^From:" nil nil "7" nil nil (number " " mark " Rob Seaman Jul 5 27/1062 " thread-indent "\"Re: FITS continuation keywords\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA10018; Tue, 5 Jul 94 19:37:15 EDT Return-Path: Message-Id: <1994Jul5.203751.21438 at noao.edu> Organization: National Optical Astronomy Observatories, Tucson, AZ, USA Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!howland.reston.ans.net!gatech!ncar!noao!seaman References: From: seaman at noao.edu (Rob Seaman) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Tue, 5 Jul 1994 20:37:51 GMT Nikolai Piskunov writes: > LONGSTR = 'This is a very long string keyword value that ' > CONTINUE= 'is continued over three ' > CONTINUE= 'keywords in the FITS header.' This is perhaps a better suggestion - or, at least, the alternative offered of using a backslash (\) in a literal string as a continuation character seems liable to break a lot of software. However, nowhere in the FITS standard is there a mandate that the order of header keywords must be preserved. All of the options offered so far assume that the order of the header is unchangeable. Even the simple addition of a new keyword, let alone a wholesale header reordering or sorting, could disrupt a continuation card sequence. Any continuation convention must either overtly preserve the continuation order (within the context of the individual continuation cards), or the FITS standard must be amended to require more stringent limitations on header keyword operations. Are continuation cards really needed? Rob Seaman NOAO/IRAF Group seaman at noao.edu From fitsbits-request Wed Jul 6 04:53:15 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["192" "" " 6" "July" "1994" "01:28:04" "-0400" "steves1825 at aol.com" "steves1825 at aol.com" "<2vdfd4$dep at search01.news.aol.com>" "6" "Re: FITS on Mac via NIH Image Mods" "^From:" nil nil "7" "1994070605:28:04" "FITS on Mac via NIH Image Mods" (number " " mark " steves1825 at aol.co Jul 6 6/192 " thread-indent "\"Re: FITS on Mac via NIH Image Mods\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA10295; Wed, 6 Jul 94 04:53:15 EDT Return-Path: Message-Id: <2vdfd4$dep at search01.news.aol.com> Organization: America Online, Inc. (1-800-827-6364) Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!swrinde!pipex!sunic!trane.uninett.no!eunet.no!nuug!EU.net!uunet!newstf01.cr1.aol.com!search01.news.aol.com!not-for-mail References: From: steves1825 at aol.com (SteveS1825) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS on Mac via NIH Image Mods Date: 6 Jul 1994 01:28:04 -0400 In article , pmurphy at nrao.edu (Patrick P. Murphy) writes: Pat, thank you for that ftp information for FITS files! I hope I can find a way to ftp. From fitsbits-request Wed Jul 6 05:20:55 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1722" "Wed" " 6" "July" "1994" "09:07:00" "+0000" "Clive Page" "cgp at star.le.ac.uk" "" "34" "TSORTKEY and continuation proposals" "^From:" nil nil "7" "1994070609:07:00" "TSORTKEY and continuation proposals" (number " " mark " Clive Page Jul 6 34/1722 " thread-indent "\"TSORTKEY and continuation proposals\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA10894; Wed, 6 Jul 94 05:20:55 EDT Return-Path: Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 1721 From: Clive Page Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: TSORTKEY and continuation proposals Date: Wed, 6 Jul 1994 09:07:00 +0000 (GMT) (1) Sort order I agree that we need a way of specifying the sort order of FITS tables. Many existing tables are actually sorted on some column or columns; for example the astronomical catalogues distributed by NSSDC on CD-ROM mostly seem to be sorted on right ascension, but at present you have to check all through the table to discover this. Many applications will be able to access such tables much more efficiently if the sorting order is declared. The proposal by Bill Pence seems simple and effective and I'd like to support it. (2) Strings that are so long that they need to be continued. I'd prefer the second option, that is to have the continuation lines labelled with 'CONTINUE= '. This is because lines with a blank keyword may not be processed properly by current systems. It is true that the FITS Standard does not require the order of keywords to be preserved, so any system of continuations is liable to fail, but in practice lots of existing FITS files have a sequence of keywords containing COMMENTs that only makes sense if they can be read in sequence; all the systems that I know of seem to preserve their order in such cases. If so then maybe the backslash character at the end of the string being continued is not necessary. The backslash can be hard to handle in C programs, and many Fortran systems running on Unix also treat the backslash as an escape character. If we can do without it, so much the better. If we do need a continuation character then I'd prefer the ampersand "&" as this is consistent with Fortran 90. ! Clive Page, Dept of Physics & Astronomy, University of Leicester, UK. ! e-mail: cgp at star.le.ac.uk ! phone: +44 533 523551 Fax: +44 533 550182 From fitsbits-request Wed Jul 6 06:38:03 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1600" "" " 6" "July" "1994" "09:39:18" "GMT" "Martin Shepherd" "mcs at goblin.caltech.edu" "" "44" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070609:39:18" "FITS continuation keywords" (number " " mark " Martin Shepherd Jul 6 44/1600 " thread-indent "\"Re: FITS continuation keywords\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11447; Wed, 6 Jul 94 06:38:03 EDT Return-Path: Message-Id: Organization: California Institute of Technology. Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!gatech!swrinde!elroy.jpl.nasa.gov!netline-fddi.jpl.nasa.gov!nntp-server.caltech.edu!nntp-server.caltech.edu!mcs References: From: mcs at goblin.caltech.edu (Martin Shepherd) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 06 Jul 1994 09:39:18 GMT In article Nikolai Piskunov writes: :On Tue, 5 Jul 1994, William Pence wrote: :> ... :> For example: :> :> LONGSTR = 'This is a very long string keyword value that \' :> 'is continued over three\' :> 'keywords in the FITS header.' :> ... :> or to replace the blank keyword name with a :> reserved keyword such as 'CONTINUE' followd by an equal sign, as in: :> :> LONGSTR = 'This is a very long string keyword value that \' :> CONTINUE= 'is continued over three\' :> CONTINUE= 'keywords in the FITS header.' :> :> Is there any preference as to which of these 2 conventions we use? :> : :I feel that the idea of using \' combination may cause problems for :FITS readers written in C. No. Back slashes and quotes are not significant in C programs except when they appear in the text of a C file at compilation time. :... :On the other hand second example in WP mail shows certain redundancy. :So what if we keep the keyword CONTINUE and forget the back slash? :In other words, the example above will look like: : : LONGSTR = 'This is a very long string keyword value that ' : CONTINUE= 'is continued over three ' : CONTINUE= 'keywords in the FITS header.' This seems logical, but I can see the value of having some indication in the string being continued that the current keyword is incomplete, rather than having to look ahead at the next keyword. IMHO it would be better to side-step the whole issue of continuation lines in FITS headers, and place such strings in tables. From fitsbits-request Wed Jul 6 06:40:24 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["880" "" " 6" "July" "1994" "09:41:02" "GMT" "Guy Rixon" "gtr at mail.ast.cam.ac.uk" "<2vdu7e$3i0 at lyra.csx.cam.ac.uk>" "18" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070609:41:02" "FITS continuation keywords" (number " " mark " Guy Rixon Jul 6 18/880 " thread-indent "\"Re: FITS continuation keywords\"\n") "<1994Jul5.203751.21438 at noao.edu>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11461; Wed, 6 Jul 94 06:40:24 EDT Return-Path: Message-Id: <2vdu7e$3i0 at lyra.csx.cam.ac.uk> Organization: Royal Greenwich Observatory Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!gatech!swrinde!pipex!lyra.csx.cam.ac.uk!cast0.ast.cam.ac.uk!gtr References: <1994Jul5.203751.21438 at noao.edu> From: gtr at mail.ast.cam.ac.uk (Guy Rixon) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 6 Jul 1994 09:41:02 GMT I for one would prefer the CONTINUE keyword to blank keywords in continuation lines. If it's done this way, the FITS reader can detect a continuation line without having to check the value field, which simplifies the code. I'd also prefer to keep the value field free of continuation symbols. How about putting a sort key in the comment field? LONGSTR1= 'This string is going to run ' / 0001 CONTINUE= 'on to two lines... ' / 0002 LONGSTR2= 'This one is too, although ' / 0003 CONTINUE= 'I'm forcing the case... ' / 0004 Since the sort keys are unique within a header the text can be reassembled however the descriptors have been shuffled. An unenlightened FITS reader will still get all the data, and will read meaningful text in the value field, but won't do the reassembly for you. Is this any help to you? Guy. From fitsbits-request Wed Jul 6 06:59:55 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2288" "" " 6" "July" "1994" "10:05:30" "GMT" "David Robinson" "drtr at mail.ast.cam.ac.uk" "<2vdvla$4pn at lyra.csx.cam.ac.uk>" "50" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070610:05:30" "FITS continuation keywords" (number " " mark " David Robinson Jul 6 50/2288 " thread-indent "\"Re: FITS continuation keywords\"\n") "<199407051645.MAA00598 at tetra.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11490; Wed, 6 Jul 94 06:59:55 EDT Return-Path: Message-Id: <2vdvla$4pn at lyra.csx.cam.ac.uk> Organization: Institute of Astronomy, Cambridge Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!swrinde!ihnp4.ucsd.edu!agate!doc.ic.ac.uk!lyra.csx.cam.ac.uk!drtr References: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> From: drtr at mail.ast.cam.ac.uk (David Robinson) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 6 Jul 1994 10:05:30 GMT In article <199407051645.MAA00598 at tetra.gsfc.nasa.gov> William Pence writes: >... >The main purpose of this posting is to see if there is any concensus >within the FITS communitiy as to which of 2 possible variations on this >convention we should use. Our orginal (and still preferred) proposal >is to use a backslash character (\) at the end of a string to indicate >that is continued on the next keyword, and to have columns 1=10 of the >following keyword(s) be filled with blanks. FITS readers that do not >understand this continuation convention would just treat the >continuation lines as comment keywords (since the keyword name is >blank). For example: > >LONGSTR = 'This is a very long string keyword value that \' > 'is continued over three\' > 'keywords in the FITS header.' > >One objection that was raised to this convention was that some FITS >readers apparently treat comment keywords differently from the other >keywords that have a value, and the keywords might get separated, thus >destroying the order of the keywords that is needed to reconstruct the >long string value. One way to solve this particular problem (if it >really is a problem) is to replace the blank keyword name with a >reserved keyword such as 'CONTINUE' followd by an equal sign, as in: > >LONGSTR = 'This is a very long string keyword value that \' >CONTINUE= 'is continued over three\' >CONTINUE= 'keywords in the FITS header.' > >Is there any preference as to which of these 2 conventions we use? >If not we will probably use the original proposal, with blank continuation >keyword names. >... Unfortunately, a FITS file with multiple CONTINUE keywords does not conform to the NOST standard, and could be rejected by a fits reader. Your first suggestion is the only way of doing this, (apart from numbered keywords); however, I would suggest making your fits headers independent of the keyword order, e.g. LONGSTR = 'This is a very long string keyword value that \' LONGSTR 2 'is continued over three\' LONGSTR 3 'keywords in the FITS header.' The keyword name and a seqence number appear in every record. C programmers should have no problem with strings containinf '\'. David Robinson. (drtr at mail.ast.cam.ac.uk) From fitsbits-request Wed Jul 6 08:06:48 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1455" "Wed" " 6" "July" "1994" "14:12:19" "+0200" "Lucio Chiappetti" "lucio at ifctr.mi.cnr.it" "" "32" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070612:12:19" "FITS continuation keywords" (number " " mark " Lucio Chiappetti Jul 6 32/1455 " thread-indent "\"Re: FITS continuation keywords\"\n") "<2vdvla$4pn at lyra.csx.cam.ac.uk>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11577; Wed, 6 Jul 94 08:06:48 EDT Return-Path: Organization: Istituto di Fisica Cosmica e Tecnologie Relative In-Reply-To: <2vdvla$4pn at lyra.csx.cam.ac.uk> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII From: Lucio Chiappetti Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Wed, 6 Jul 1994 14:12:19 +0200 (MET DST) On 6 Jul 1994, David Robinson wrote: > > Unfortunately, a FITS file with multiple CONTINUE keywords does not conform > to the NOST standard, and could be rejected by a fits reader. > > Your first suggestion is the only way of doing this, (apart from numbered > keywords); however, I would suggest making your fits headers independent of > the keyword order, e.g. > > LONGSTR = 'This is a very long string keyword value that \' > LONGSTR 2 'is continued over three\' > LONGSTR 3 'keywords in the FITS header.' > > The keyword name and a seqence number appear in every record. > IF the need for long keyword really exists (which I feel questionable besides the case of HISTORY keywords), mainly for descriptive and mission specific purposes, why not just treat them as comments ?: > COMMENT = 'LONGSTR This is a very long string keyword value that \' > COMMENT = 'LONGSTR 2 is continued over three\' > COMMENT = 'LONGSTR 3 keywords in the FITS header.' ---------------------------------------------------------------------------- Lucio Chiappetti - IFCTR/CNR | Ma te' vugl' da' quost avis a ti' Orsign via Bassini 15 - I-20133 Milano | Buttet rabios intant te se' pisnign Internet: LUCIO at IFCTR.MI.CNR.IT | Decnet: IFCTR::LUCIO | (Rabisch, II 46, 119-120) ---------------------------------------------------------------------------- From fitsbits-request Wed Jul 6 10:01:51 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["3219" "" " 6" "July" "1994" "13:46:50" "GMT" "Paul Repacholi" "prep at yarrow.wt.uwa.edu.au" "" "75" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070613:46:50" "FITS continuation keywords" (number " " mark " Paul Repacholi Jul 6 75/3219 " thread-indent "\"Re: FITS continuation keywords\"\n") "<199407051645.MAA00598 at tetra.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA00270; Wed, 6 Jul 94 10:01:51 EDT Return-Path: Message-Id: Organization: Winthrop Technology Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!swrinde!ihnp4.ucsd.edu!munnari.oz.au!news.uwa.edu.au!news!prep References: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> From: prep at yarrow.wt.uwa.edu.au (Paul Repacholi) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 06 Jul 1994 13:46:50 GMT In article <199407051645.MAA00598 at tetra.gsfc.nasa.gov> William Pence writes: > > Last summer there was considerable debate in this newgroup about a > proposed convention for encoding very long character strings as the > value of FITS keywords. We will soon need to begin using this ^^^^ > convention (or some variant on it) for the FITS files generated by the > XTE mission. > > The main purpose of this posting is to see if there is any concensus > within the FITS communitiy as to which of 2 possible variations on this > convention we should use. Our orginal (and still preferred) proposal > is to use a backslash character (\) at the end of a string to indicate > that is continued on the next keyword, and to have columns 1=10 of the > following keyword(s) be filled with blanks. FITS readers that do not > understand this continuation convention would just treat the > continuation lines as comment keywords (since the keyword name is > blank). For example: > > LONGSTR = 'This is a very long string keyword value that \' > 'is continued over three\' > 'keywords in the FITS header.' This means interpreting the value of 'LONGSTR' as part of inputting it. I would rather see LONGSTR = ".... that'\ LONGSTR = 'is cont...'\ LONGSTR = 'to a final un-escaped line' type syntax. Thus each line is a 'LONGSTR' to old reader, and the semantics are cleaned up, and could extend to other keywords as well. > > One objection that was raised to this convention was that some FITS > readers apparently treat comment keywords differently from the other > keywords that have a value, and the keywords might get separated, thus > destroying the order of the keywords that is needed to reconstruct the > long string value. One way to solve this particular problem (if it > really is a problem) is to replace the blank keyword name with a > reserved keyword such as 'CONTINUE' followd by an equal sign, as in: > > LONGSTR = 'This is a very long string keyword value that \' > CONTINUE= 'is continued over three\' > CONTINUE= 'keywords in the FITS header.' > > Is there any preference as to which of these 2 conventions we use? > If not we will probably use the original proposal, with blank continuation > keyword names. Hum try this for parsing... LONGSTR = 'This is a long string\' CONTINUE= ' with\' COMMENT = 'This is a comment with\' CONTINUE= 'MORE THAN ONE LINE!! We think' FITS seems to have a problem with lack of clear interfaces and 'levels' of function. Putting the continue char into the value is an example of this. If the '\' is the LAST char, the reader can strip it, the closing ' and everything upto the next value start, then append the input to the buffer. Higher level code need not ever know there has been a change. Except for the size of the strings it will now be fed of course. ;-) The word "need" is the real worry though... -- ~Paul +61 (09) 257-1001 prep at yarrow.wt.uwa.edu.au ( preferred ) 1 Crescent Rd, zrepachol at cc.curtin.edu.au Kalamunda, West Aust 6076 From fitsbits-request Wed Jul 6 10:56:32 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["5523" "Wed" " 6" "July" "1994" "15:54:54" "+0100" "cur at star.rl.ac.uk" "cur at star.rl.ac.uk" "<9407061456.AA00395 at fits.cv.nrao.edu>" "105" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070614:54:54" "FITS continuation keywords" (number " " mark " cur at star.rl.ac.uk Jul 6 105/5523 " thread-indent "\"Re: FITS continuation keywords\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA00401; Wed, 6 Jul 94 10:56:32 EDT Return-Path: Message-Id: <9407061456.AA00395 at fits.cv.nrao.edu> Reply-To: cur at star.rl.ac.uk X-Vms-To: SMTP%"fitsbits at fits.cv.nrao.edu" From: cur at star.rl.ac.uk Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Wed, 6 Jul 1994 15:54:54 +0100 The first question to address is "Are long strings necessary?". Even if you are prepared to lose the comments (undesirable IMHO) you are limited to 68 characters, and 18 characters if you have a comment starting at the normal column. I can think of many instances where a value in our data structures might exceed 68 characters, albeit not a common occurrence, and where quite frequently 18 characters would be inadequate. If the OGIP people did not have this need also, I do not think they would have gone to the trouble of making the proposal. So let's take it for granted, we need a way of storing header values comprising up to a few hundred characters. I do not like having such medium-length header-type information tucked away in tables as Martin Shepherd suggests. It has to be dealt with separately, possibly being written to a separate file (what is it called?) or not at all by some readers; thus the information can all too readily become detached from the dataset it is describing. So the question boils down to which ad-hoc convention does the job with minimal disruption to existing software and does not limit further developments. Starting with the OGIP proposal. The continuation-character convention restricts the content of the string, albeit in a minimal way, in that you cannot use a nominated character in a literal sense at the end of a string. In practice this is unlikely to be a problem except perhaps for strings written by users ignorant of the convention or in values written in existing files. Since readers would have to look at the next keyword to see whether or not it is a blank string or CONTINUE, it can be handled. Should readers complain if there is a continuation character, but no continuation line, and treat the final character as part of the keyword's value? The above would suggest not, but the following damning point against the OGIP proposal RS> However, nowhere in the FITS standard is there a mandate that the order RS> of header keywords must be preserved. All of the options offered so RS> far assume that the order of the header is unchangeable. Even the RS> simple addition of a new keyword, let alone a wholesale header RS> reordering or sorting, could disrupt a continuation card sequence. argues the contrary. It is fine for systems written by observatories and satellite missions, but not for those where users can input or relocate new header cards. Given header-validation software how will it know if the continuation character was intended to be literal or not? I have probably laboured this point more than it merits from a practical sense. Whatever is decided the documentation of the convention should be clarified accordingly. WDP> LONGSTR = 'This is a very long string keyword value that \' WDP> CONTINUE= 'is continued over three\' WDP> CONTINUE= 'keywords in the FITS header.' The redundancy question raised by Nikolai occurred to me too. Why is there a continuation character and a CONTINUE keyword? Is it that the continuation character flags to the reader that it has to look at the next header card? The CONTINUE card may have been used already for another purpose. The fact that it is not part of the current standard, is less of a problem for me. We would have to modify the standard to allow multiple CONTINUE cards. Coming back to Rob Seaman's posting, RS> Any continuation convention must either overtly preserve the continuation RS> order (within the context of the individual continuation cards), or the RS> FITS standard must be amended to require more stringent limitations on RS> header keyword operations. there have been some suggestions for the former. I am not keen on numbering schemes and ones that require looking at the comments (shades of UNIX shell scripts) as Guy Rixon suggests. I should rather have the number in the string if it could be done unambiguously. The trouble with numbering is that you have to know which numbers already exist, or at least the highest index, before inserting a new card. That is not a big deal, just a little messy to code offering more opportunity for violations, and is less efficient. If you want to connect keywords why not have a linked list by keyword name rather than number? (-: I am probably prejudiced against David Robinson's suggestion because it reminds me too much of Isaac Newton Group's hierarchical keywords. Given an arbitrary blank keyword (or COMMENT in Lucio Chiappetti's proposal) you will not be able to have the first word being another keyword followed by a number. Probably not a frequent occurrence, but it is possible. The disadvantage of using COMMENT keywords is that certain readers may detach the information from the data the long string describes. I do not think it unreasonable that a dataset has some ancillary information longer than 68 characters. >From a selfish point of view in Starlink we can cope with either OGIP proposal as we store the FITS header verbatim in our standard data structures, and so order is preserved. Presumably FITSIO users will also be able to handle whatever is agreed without pain. Of the suggestions made so far, I would plump for WDP> LONGSTR = 'This is a very long string keyword value that \' WDP> 'is continued over three\' WDP> 'keywords in the FITS header.' being the least undesirable. However, I would be interested in other proposals that avoid a specific ordering of header cards to if we can improve on the OGIP proposal. Malcolm Currie Starlink Project From fitsbits-request Wed Jul 6 11:26:00 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["856" "Wed" " 6" "July" "1994" "17:22:49" "+0200" "Thierry Forveille" "forveill at gag.observ-gr.fr" "<9407061525.AA00470 at fits.cv.nrao.edu>" "15" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070615:22:49" "FITS continuation keywords" (number " " mark " Thierry Forveille Jul 6 15/856 " thread-indent "\"Re: FITS continuation keywords\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA00476; Wed, 6 Jul 94 11:26:00 EDT Return-Path: Message-Id: <9407061525.AA00470 at fits.cv.nrao.edu> Posted-Date: Wed, 6 Jul 1994 17:22:49 +0200 In-Reply-To: References: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> From: forveill at gag.observ-gr.fr (Thierry Forveille) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Wed, 6 Jul 1994 17:22:49 +0200 Since all proposed continuation syntaxes violate the present FITS rule that the header ordering is irrelevant, I'd like to see a better case for the need of >68 characters in header values. Contrary to a statement in one of the follow-ups to the original posting, some FITS readers (including ours...) do use this feature of the present standard. The disruption would be relatively minor since OGIP intends to use continuation lines for mission specific information only (and our users will presumably not look at any XTE product anyway), but this opens the door to a more general use and potential problems. The modification will put a small but non-zero burden on a number of people, so its proposers need to convince the community through a worked out example that their purpose cannot be met otherwise. Thierry Forveille Observatoire de Grenoble From fitsbits-request Wed Jul 6 14:37:04 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1963" "Wed" " 6" "July" "1994" "17:32:09" "GMT" "Frank Valdes" "valdes at tucana.tuc.noao.edu" "<1994Jul6.173209.29646 at noao.edu>" "41" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070617:32:09" "FITS continuation keywords" (number " " mark " Frank Valdes Jul 6 41/1963 " thread-indent "\"Re: FITS continuation keywords\"\n") "<9407061525.AA00470 at fits.cv.nrao.edu>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA01211; Wed, 6 Jul 94 14:37:04 EDT Return-Path: Message-Id: <1994Jul6.173209.29646 at noao.edu> Organization: IRAF Project, National Optical Astronomy Observatories Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!newsxfer.itd.umich.edu!gatech!ncar!noao!tucana.tuc.noao.edu!valdes References: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> <9407061525.AA00470 at fits.cv.nrao.edu> From: valdes at tucana.tuc.noao.edu (Frank Valdes) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Wed, 6 Jul 1994 17:32:09 GMT My personal preference is numbered keywords. All parts of the string should be preserved by any reader/writer and the order can be recovered by software interested in it that knows the root keyword. This is also an existing practice. However, the statement has been made that this is not possible for a certain dataset (which was a design decision that might have been different). I would suggest that the same keyword be used for each part and the index be part of the value field (with some variants following). Before giving the argument an example would be: LONGSTR = 1 'First part of this keyword' LONGSTR = 3 'Third part of this keyword' ANOTHER = 2 ' very long keyword' LONGSTR = 2 'Second part of this keyword' ANOTHER = 1 'Here is another' The argument for using the same keyword is that all parts of a long value are associated. Use of blank, COMMENT, HISTORY, etc. can become confused. The index allows the order to be reconstructed if some reader/writer does not preserve the original order. Finally existing readers will likely take one of four actions; just save all keywords, save the first with the same name, save the last with the same name, give a warning or error. Also the value will either be preserved completely or just the first part, whether it is the string or the index. The main problem is having two parts to the value field. A variant would be: 'Second part of this keyword\2' (or some other delimiter). In this case, unlike some other proposals, the reader doesn't necessarily have to scan the value. If the reader is prepared for this syntax it would only need to scan the value for ordering when a second instance of a keyword is found. A caveat is that I don't know without researching it whether there is any mandate in the standard against multiple occurances of the same keyword though I know that blank and HISTORY were specifically allowed. So in a sense there is a precedent. Frank Valdes IRAF Group/NOAO From fitsbits-request Wed Jul 6 19:38:49 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1601" "" " 6" "July" "1994" "23:25:15" "GMT" "Allyn Tennant" "tennant at alph.msfc.nasa.gov" "<2vfegr$96l at hammer.msfc.nasa.gov>" "41" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070623:25:15" "FITS continuation keywords" (number " " mark " Allyn Tennant Jul 6 41/1601 " thread-indent "\"Re: FITS continuation keywords\"\n") "<199407051645.MAA00598 at tetra.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA01846; Wed, 6 Jul 94 19:38:49 EDT Return-Path: Message-Id: <2vfegr$96l at hammer.msfc.nasa.gov> Organization: NASA/MSFC Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!news.msfc.nasa.gov!usenet References: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> Reply-To: tennant at alph.msfc.nasa.gov From: tennant at alph.msfc.nasa.gov (Allyn Tennant) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 6 Jul 1994 23:25:15 GMT In article <199407051645.MAA00598 at tetra.gsfc.nasa.gov> William Pence writes: > (Do you prefer:) > > LONGSTR = 'This is a very long string keyword value that \' > 'is continued over three\' > 'keywords in the FITS header.' > > (or) > > LONGSTR = 'This is a very long string keyword value that \' > CONTINUE= 'is continued over three\' > CONTINUE= 'keywords in the FITS header.' > > Is there any preference as to which of these 2 conventions we use? I vote for neither. I object to the use of the backslash to denote a continuation line. This is for two main reasons: 1) For hundreds of years publishers have used a trailing - (hyphen) to denote a continuation line. Why create a new symbol to do the same job? 2) In UNIX, C and many other parsers, backslash is an escape character which means that the OS is to pass the following character without interpretation. Thus in C the backslash at the end of the line means don't treat the following linefeed as an end of line marker but pass it through. In the two examples above the backslash precedes a single quote character. Thus someone who knows C et al. should naturally read the line to mean that the single quote is part of the string. I believe that using backslash in the proposed manner will make it very difficult to use backslash in it's more normal meaning as an escape character in the future. In other words, using backslash as a continuation character only makes sense if it is followed by a linefeed character, and this condition does not arise in FITS. Allyn From fitsbits-request Wed Jul 6 23:35:33 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2858" "" " 6" "July" "1994" "21:51:48" "-0400" "Lee E. Brotzman" "leb at clark.net" "<2vfn3k$k8o at explorer.clark.net>" "54" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070701:51:48" "FITS continuation keywords" (number " " mark " Lee E. Brotzman Jul 6 54/2858 " thread-indent "\"Re: FITS continuation keywords\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA01943; Wed, 6 Jul 94 23:35:33 EDT Return-Path: Message-Id: <2vfn3k$k8o at explorer.clark.net> Organization: Clark Internet Services, Inc., Ellicott City, MD USA Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!gatech!udel!news2.sprintlink.net!news.sprintlink.net!news.clark.net!news.clark.net!not-for-mail References: From: leb at clark.net (Lee E. Brotzman) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 6 Jul 1994 21:51:48 -0400 mcs at goblin.caltech.edu (Martin Shepherd) writes: >In article Nikolai Piskunov writes: >:I feel that the idea of using \' combination may cause problems for >:FITS readers written in C. >No. Back slashes and quotes are not significant in C programs except >when they appear in the text of a C file at compilation time. Exactly right. Parsing for backslashes poses no difficulty at all for C programs. Shells, which are written in C, do it all the time. I prefer a backslash because it does not appear in normal text (which can not be said for the amersand, &, suggested by a previous poster). >:On the other hand second example in WP mail shows certain redundancy. >:So what if we keep the keyword CONTINUE and forget the back slash? >:In other words, the example above will look like: >: >: LONGSTR = 'This is a very long string keyword value that ' >: CONTINUE= 'is continued over three ' >: CONTINUE= 'keywords in the FITS header.' >This seems logical, but I can see the value of having some indication >in the string being continued that the current keyword is incomplete, >rather than having to look ahead at the next keyword. I definitely DO NOT want a CONTINUE keyword. In order to generalize the parser I would have to change the algorithm to use look ahead for all keywords. Yuch. That is a very significant coding change. If continuations are needed at all (see below), then the keyword being continued must indicate that there is more information to follow. This can be coded much more simply as an exceptional, as opposed to a general, rule. >IMHO it would be better to side-step the whole issue of continuation >lines in FITS headers, and place such strings in tables. This I agree most strongly with, but in a different way. My impression is that these keywords would hold "one-time" descriptive information for an entire image or table column. Putting it into the table would introduce a lot of redundancy. However, I tend to doubt that there is a need here that can only be met by breaking the standard, or otherwise introducing new conventions to write code around. I have been pretty successful using conventions in standard COMMENT keywords to do exactly the same thing (kind of a mini-heiarchical keyword scheme... works great for me at least). Unless I can see a really compelling argument for this convention, I most certainly would vote against it (if I'm still on the E-mail voting body, that is, I haven't heard from Don in quite a while). -- -- Lee E. Brotzman E-mail: leb at clark.net -- Advanced Data Solutions Phone : 301-776-0324 -- "Sometimes I think the surest sign that intelligent life exists elsewhere in -- the Universe is that none of it has tried to contact us." --- Calvin, 1990 From fitsbits-request Thu Jul 7 10:46:26 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1393" "" " 7" "July" "1994" "14:27:48" "GMT" "Robert Hill" "bhill at trifle.gsfc.nasa.gov" "<2vh3d4$sm7 at paperboy.gsfc.nasa.gov>" "33" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070714:27:48" "FITS continuation keywords" (number " " mark " Robert Hill Jul 7 33/1393 " thread-indent "\"Re: FITS continuation keywords\"\n") "<2vfn3k$k8o at explorer.clark.net>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA03641; Thu, 7 Jul 94 10:46:26 EDT Return-Path: Message-Id: <2vh3d4$sm7 at paperboy.gsfc.nasa.gov> Organization: NASA Goddard Space Flight Center -- InterNetNews site Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!swrinde!ihnp4.ucsd.edu!ames!newsfeed.gsfc.nasa.gov!bhill References: <2vfn3k$k8o at explorer.clark.net> From: bhill at trifle.gsfc.nasa.gov (Robert Hill) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 7 Jul 1994 14:27:48 GMT I don't like the idea of continuations, myself. If I had this need in my project, I think I would try to find some way to embed the whole structure in history or comment cards, with some extra code to break it out of there. Alternatively, it seems to me that a long string of mission-specific information must have some substructure, so that it could be parsed into separate parameters before being written to the header. Of the schemes proposed, however, I prefer the backslash continuation onto 'blank' comment cards as having the minimum impact on FITS readers outside the project. I really dislike the CONTINUE= keyword. I think what really rankles me is the equals sign, which implies that CONTINUE is somehow a parameter of the data set, like NAXIS or CRPIX1. Really, CONTINUE is another keyword of the HISTORY/COMMENT/blank class, and the equals sign is there only because FITS allows us to invent new keywords, but not new history/comment indicators. The invention of such an indicator specifically for continuing strings would require the standard to be amended, and nobody wants to get into that. So, if you must, backslash + blank-comment card. -- Robert.S.Hill at gsfc.nasa.gov Not speaking for any of these: Hughes STX Corp., Code 681, NASA/GSFC, Greenbelt, MD 20771 "There's nothing you can't prove if your outlook is only sufficiently limited." -- Lord Peter Wimsey From fitsbits-request Thu Jul 7 17:48:41 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["3693" "" " 7" "July" "1994" "17:30" "EDT" "BARRY M. SCHLESINGER" "bschlesinger at nssdca.gsfc.nasa.gov" "<7JUL199417305565 at nssdca.gsfc.nasa.gov>" "72" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070721:30:00" "FITS continuation keywords" (number " " mark " BARRY M. SCHLESIN Jul 7 72/3693 " thread-indent "\"Re: FITS continuation keywords\"\n") "<2vh3d4$sm7 at paperboy.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA04817; Thu, 7 Jul 94 17:48:41 EDT Return-Path: Message-Id: <7JUL199417305565 at nssdca.gsfc.nasa.gov> Organization: NASA - Goddard Space Flight Center Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!news.umbc.edu!haven.umd.edu!ames!newsfeed.gsfc.nasa.gov!nssdca.gsfc.nasa.gov!bschlesinger References: <2vfn3k$k8o at explorer.clark.net> <2vh3d4$sm7 at paperboy.gsfc.nasa.gov> From: bschlesinger at nssdca.gsfc.nasa.gov (BARRY M. SCHLESINGER) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 7 Jul 1994 17:30 EDT I'm commenting on a number of points from different posts, and VNEWS doesn't have an obvious way of keeping the attribution straight from multiple non-sequential posts. On the question of repeated keywords... The NOST standard does not specifically address the general issue of keyword repetition. The order and placement of most mandatory keywords is prescribed. Presence in the header of more than one HISTORY, COMMENT, or [blank] is specifically permitted. But there is no specification on reserved or user-defined keywords. The issue of repeating this kind of keyword was left to the User's Guide for discussion, rather than being prescribed in the standard. The relevant text in the User's Guide: For keywords that have no value and are used to transfer text, such as HISTORY and COMMENT, the same keyword may be used repeatedly to indicate that the remainder of the card image contains text or comments. However, repetition of a keyword with conflicting values may cause confusion. There is no standard interpretation in this case as to which value is to be retained as the true value of the keyword. Different FITS readers may interpret the header differently. Do not repeat a keyword on different card images if the values conflict. So multiple CONTINUE keywords would not be a violation of the standard. On distinguishing keywords without values.... Except for COMMENT, HISTORY, and [blank], the contents of columns 9 and 10 distinguish whether or not the keyword has a value. Except for those three keywords, the keyword has a value if and only if columns 9 and 10 contain '= '. Otherwise, columns 9-80 are comment and there is no value. The reason for the exceptions in the cases of COMMENT, HISTORY and [blank] is historical; the original FITS papers did not forbid '= ' in columns 9 and 10, and this freedom must be preserved, following the "once FITS, always FITS" rule. The keyword name tells the reader that there is no value. For user-defined keywords, though some way of distinguishing whether there is a value is needed. One point to consider is that some readers may treat multiple occurrences of keywords without values differently from multiple occurrences of keywords with values. On the order of keywords in a header... To say that the order of optional keywords in a header is irrelevant is perhaps too strong a statement. The order is not specified. Because the order is not specified, FITS readers are not required to preserve it. Some don't. This potential for card shuffling is the reason that it is unwise to rely on card image order in a FITS header. Finally, whatever convention is accepted should not break or even bend FITS readers. An example of how to construct such a convention is the NRAO use of hidden keywords behind the HISTORY keyword -- also used in the HIERARCH convention at ESO -- illustrated in Example 1 of the User's Guide. A reader that is familiar with the convention will find the "keywords" that start in the later columns, and be able to process them. However, a reader that is unfamiliar with the convention still has no problem; it simply treats the card images in question as containing comments, and the information can be easily obtained by the human reader by reading a listing of the header. We must keep the basic purpose of FITS in mind -- to allow the community access to data without the necessity of constructing specialized software for each data set read. If we are to achieve this goal, we must avoid creating conventions that will require patches to a significant fraction of the FITS readers in use. Barry Schlesinger NOST FITS Support Office From fitsbits-request Fri Jul 8 18:01:47 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1441" "" " 8" "July" "1994" "18:43:39" "GMT" "Paul Repacholi" "prep at yarrow.wt.uwa.edu.au" "" "32" "Re: TSORTKEY and continuation proposals" "^From:" nil nil "7" "1994070818:43:39" "TSORTKEY and continuation proposals" (number " " mark " Paul Repacholi Jul 8 32/1441 " thread-indent "\"Re: TSORTKEY and continuation proposals\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA07324; Fri, 8 Jul 94 18:01:47 EDT Return-Path: Message-Id: Organization: Winthrop Technology Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!malgudi.oar.net!news.pipeline.com!news.intercon.com!howland.reston.ans.net!agate!ihnp4.ucsd.edu!munnari.oz.au!news.uwa.edu.au!news!prep References: From: prep at yarrow.wt.uwa.edu.au (Paul Repacholi) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: TSORTKEY and continuation proposals Date: 08 Jul 1994 18:43:39 GMT In article Clive Page writes: ... > may not be processed properly by current systems. It is true that the > FITS Standard does not require the order of keywords to be preserved, so > any system of continuations is liable to fail, but in practice lots of > existing FITS files have a sequence of keywords containing COMMENTs that > only makes sense if they can be read in sequence; all the systems that I > know of seem to preserve their order in such cases. In the interest of the golden rule "once FITS, always FITS" it seems that preserving order should become a REQUIRMENT in FITS. 1) No existing files will be affected. 2) Software packages will have two less free variables to worry about. 3) Any multi-line form of extention will reqire that ordering and grouping be preserved, so this will be needed as part of Bill's proposal. I am still not happy with a syntax form that embeds packing info into a value. The problem is not that it is \ or & or -, but that it apeares inside the value. So is the \ part of the string? Inband signaling is very seldom a good idea! -- ~Paul +61 (09) 257-1001 prep at yarrow.wt.uwa.edu.au ( preferred ) 1 Crescent Rd, zrepachol at cc.curtin.edu.au Kalamunda, West Aust 6076 From fitsbits-request Fri Jul 8 23:33:00 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["3511" "" " 8" "July" "1994" "23:06:51" "-0400" "Lee E. Brotzman" "leb at clark.net" "<2vl48b$k7k at explorer.clark.net>" "76" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994070903:06:51" "FITS continuation keywords" (number " " mark " Lee E. Brotzman Jul 8 76/3511 " thread-indent "\"Re: FITS continuation keywords\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA07463; Fri, 8 Jul 94 23:33:00 EDT Return-Path: Message-Id: <2vl48b$k7k at explorer.clark.net> Organization: Clark Internet Services, Inc., Ellicott City, MD USA Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!sundog.tiac.net!news3.sprintlink.net!news.sprintlink.net!news.clark.net!news.clark.net!not-for-mail References: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> From: leb at clark.net (Lee E. Brotzman) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 8 Jul 1994 23:06:51 -0400 prep at yarrow.wt.uwa.edu.au (Paul Repacholi) writes: >In article <199407051645.MAA00598 at tetra.gsfc.nasa.gov> William Pence writes: >> following keyword(s) be filled with blanks. FITS readers that do not >> understand this continuation convention would just treat the >> continuation lines as comment keywords (since the keyword name is >> blank). For example: >> >> LONGSTR = 'This is a very long string keyword value that \' >> 'is continued over three\' >> 'keywords in the FITS header.' >This means interpreting the value of 'LONGSTR' as part of inputting >it. I would rather see >LONGSTR = ".... that'\ >LONGSTR = 'is cont...'\ >LONGSTR = 'to a final un-escaped line' >type syntax. >Thus each line is a 'LONGSTR' to old reader, and the semantics are cleaned >up, and could extend to other keywords as well. BINGO! This is it. Except the syntax should be: LONGSTR = 'This is the beginning of the string...' LONGSTR = 'followed by even more of this string...' LONGSTR = 'summed up by this string' When one defines a new keyword, one must also define the keyword's value, either integer, floating point, string, etc. Therefore, to have "long strings" one simply defines the keyword syntax as a string and that the keyword can appear more than once, like COMMENT. There is no specific prohibition against repeated occurences of a keyword, so as far as I can tell, this is a legal FITS solution to the "problem" (which, of course, I still do not believe really exists). It may even be better to define the keyword without the '=', thereby creating a new form of COMMENT/HISTORY. I'll leave that to Barry to see if it is legal (frankly, his message talking about 'no value' for keywords without '=' was a little confusing to me, just what does 'no value' mean?). As a FITS programmer, how would I deal with this? Basically, my code ignores any keyword it is not specifically looking for; as is normal for FITS. So to begin with, the 'LONGSTR' keywords would have no effect whatsoever. Continuation characters and blank keywords would, however, since my code scans for blank keywords followed by text and interprets them as comments. This has the effect of all the blank keywords *following* the 'LONGSTR' keyword being put into my comment structure, thereby rendering it incomplete and invalid. If I actually *want* to scan for the 'LONGSTR' keywords, and if they are repeating keywords, all I have to do is add something like this in my parser: Comment* LONGSTR; enum keywords { ... , longstr, ... } struct keyentry { ... , { "LONGSTR ", longstr}, ... ... switch (keytype) { ... longstr: ParseComment(value, LONGSTR); A recompile, a relink, and voila! The strings are read in and stored in total. This does not qualify, in my mind, as writing new code, since I would have to do the same thing for *any* new keyword I want to scan for. Maybe it isn't the prettiest solution, but it has worked quite well for me. So, to sum up. If indeed such a need exists (which I doubt) the best legal solution is repeating keywords based on COMMENT syntax. There. We can all go home now. No wait, I'm already there! Oh well, you know what I mean. -- -- Lee E. Brotzman E-mail: leb at clark.net -- Advanced Data Solutions Phone : 301-776-0324 -- "Sometimes I think the surest sign that intelligent life exists elsewhere in -- the Universe is that none of it has tried to contact us." --- Calvin, 1990 From fitsbits-request Sat Jul 9 03:49:46 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["3361" "Sat" " 9" "July" "1994" "05:47:26" "GMT" "Doug Tody" "tody at lepus.tuc.noao.edu" "<1994Jul9.054726.18859 at noao.edu>" "61" "Re: TSORTKEY and continuation proposals" "^From:" nil nil "7" "1994070905:47:26" "TSORTKEY and continuation proposals" (number " " mark " Doug Tody Jul 9 61/3361 " thread-indent "\"Re: TSORTKEY and continuation proposals\"\n") ""] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA07576; Sat, 9 Jul 94 03:49:46 EDT Return-Path: Message-Id: <1994Jul9.054726.18859 at noao.edu> Organization: National Optical Astronomy Observatories Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!ncar!noao!lepus.tuc.noao.edu!tody References: Reply-To: tody at noao.edu From: tody at lepus.tuc.noao.edu (Doug Tody) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: TSORTKEY and continuation proposals Date: Sat, 9 Jul 1994 05:47:26 GMT In article , prep at yarrow.wt.uwa.edu.au (Paul Repacholi) writes: > > In the interest of the golden rule "once FITS, always FITS" it seems > that preserving order should become a REQUIRMENT in FITS. > > 1) No existing files will be affected. > > 2) Software packages will have two less free variables to worry about. > > 3) Any multi-line form of extention will reqire that ordering and > grouping be preserved, so this will be needed as part of Bill's > proposal. This is easy to say but it is not that simple. The possibility of reordering header keywords occurs when the FITS header is edited. Often the header contains related groups of keywords. For example, the header may contain a world coordinate system (WCS) specification stored in a number of related keywords (any number of other examples are possible). If the WCS associated with the image changes it may be that the number of keywords required to describe the WCS changes. One must delete all the old WCS keywords and insert the new WCS keywords. It is possible to insert or append the new group of keywords as a sequential block, but there is no guarantee that all the original WCS keywords will be consecutive. Hence one deletes all the old WCS keywords wherever they may appear in the header, and inserts or appends the new keywords somewhere else, the logical place being near the beginning of the header since these are standard keywords. This may mean that the order of the keywords changes. Preserving the original order could be difficult or impossible. The situation can get even worse. It may be that the header is ambiguous, containing multiple copies of the same keyword, e.g. two WCS specifications. Which is correct? If the WCS is modified do we delete both sets? If one thinks of the FITS header as time ordered the last entry would take precedence, but the defacto FITS practice is to order the header with the "standard" keywords near the top, followed by an arbitrary number of application specific keywords, comments, history etc. in the latter part of the header. Hence to preserve the logical order of the FITS header one wants to consolidate the system keywords near the top. If the input header is poorly structured it may need to be reordered when written back out by the software which edits the header. Another example would be combining two images and writing a single new output image. How do we combine the two headers without reordering at least some of the keywords, and probably removing some duplicate keywords? Obviously one wants to preserve the original order of the header keywords where possible, but the point is that this is not always possible or desirable. Note that this problem only arises when using FITS as a runtime format where headers are subject to dynamic updates. Since FITS is intended as a static archival and exchange format issues such as dynamic updates or reordering keywords are not officially an issue. Nonetheless it is an issue for real data systems, since we all ingest FITS files, generate new images, and write out new FITS files, and one wants to preserve the information in the input headers so far as possible. -- Doug Tody National Optical Astronomy Observatories IRAF project tody at noao.edu P.O. Box 26732, Tucson, Arizona, 85726 iraf at noao.edu From fitsbits-request Sun Jul 10 11:14:44 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["3029" "Sun" "10" "July" "1994" "17:20:35" "+0200" "Lucio Chiappetti" "lucio at ifctr.mi.cnr.it" "" "64" "Re: TSORTKEY and continuation proposals" "^From:" nil nil "7" "1994071015:20:35" "TSORTKEY and continuation proposals" (number " " mark " Lucio Chiappetti Jul 10 64/3029 " thread-indent "\"Re: TSORTKEY and continuation proposals\"\n") "<1994Jul9.054726.18859 at noao.edu>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11369; Sun, 10 Jul 94 11:14:44 EDT Return-Path: Organization: Istituto di Fisica Cosmica e Tecnologie Relative In-Reply-To: <1994Jul9.054726.18859 at noao.edu> Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII From: Lucio Chiappetti Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: TSORTKEY and continuation proposals Date: Sun, 10 Jul 1994 17:20:35 +0200 (MET DST) On Sat, 9 Jul 1994, Doug Tody wrote: > In article , prep at yarrow.wt.uwa.edu.au (Paul Repacholi) writes: > > > > In the interest of the golden rule "once FITS, always FITS" it seems > > that preserving order should become a REQUIRMENT in FITS. > > > > This is easy to say but it is not that simple. The possibility of > reordering header keywords occurs when the FITS header is edited. Often the ... > keywords and insert the new WCS keywords. It is possible to insert or > append the new group of keywords as a sequential block, but there is no > guarantee that all the original WCS keywords will be consecutive. Hence one > deletes all the old WCS keywords wherever they may appear in the header, and > inserts or appends the new keywords somewhere else, the logical place being ... The point is that a FITS header is somewhat an awkward structure to be handled in memory, and on disk, since it is an almost fixed (once written) number of card images in a fixed number of 2880-blocks. Extending the header will require writing a new file (a reason not to use FITS as working format, see below). Hence the point is : either read all the header in memory, and REPLACE the values of changed kewyords, or EXTEND the header allocating more memory and finally FLUSH the header to disk or read "wanted" keywords by name from the input file and write them to the output file eventually with a modified value. What to do then with "unwanted" or "unexpected" keywords ? Ignore them ? or do a further pass and write out those not already written (which implies they are flagged) ? > The situation can get even worse. It may be that the header is ambiguous, > containing multiple copies of the same keyword, e.g. two WCS > specifications. Which is correct? If the WCS is modified do we delete both Is this a "legal" FITS file ? I'd assumed that, besides COMMENTS and HISTORY, duplication of keywords was not allowed or at least discouraged. > Another example would be combining two images and writing a single new > output image. How do we combine the two headers without reordering at least > some of the keywords, and probably removing some duplicate keywords? > Yes indeed > Note that this problem only arises when using FITS as a runtime format where > headers are subject to dynamic updates. Since FITS is intended as a static > archival and exchange format issues such as dynamic updates or reordering ^^^^^^^^^^^^^^^^^^^^^ YES INDEED ---------------------------------------------------------------------------- Lucio Chiappetti - IFCTR/CNR | Ma te' vugl' da' quost avis a ti' Orsign via Bassini 15 - I-20133 Milano | Buttet rabios intant te se' pisnign Internet: LUCIO at IFCTR.MI.CNR.IT | Decnet: IFCTR::LUCIO | (Rabisch, II 46, 119-120) ---------------------------------------------------------------------------- From fitsbits-request Sun Jul 10 16:50:38 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["424" "" "10" "July" "1994" "20:43:50" "GMT" "Andrew T. Chambers" "atc at waikato.ac.nz" "<2vpmi6$1nbb at thebes.waikato.ac.nz>" "16" "Visual Basic VBX and DLL file for FITS?" "^From:" nil nil "7" "1994071020:43:50" "Visual Basic VBX and DLL file for FITS?" (number " " mark " Andrew T. Chamber Jul 10 16/424 " thread-indent "\"Visual Basic VBX and DLL file for FITS?\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11535; Sun, 10 Jul 94 16:50:38 EDT Return-Path: Message-Id: <2vpmi6$1nbb at thebes.waikato.ac.nz> Organization: The University of Waikato Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!newsxfer.itd.umich.edu!gumby!wupost!waikato!atc.ijk.waikato.ac.nz!atc From: Andrew T. Chambers Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Visual Basic VBX and DLL file for FITS? Date: 10 Jul 1994 20:43:50 GMT Does anyone know of an VBX or DLL files for Visual BASIC that are designed to help one implement a FITS viewer or image manipulation program? I woul like to design such a program but its practically impossible without extra VBX or DLL's specially written for the job. I have looked at a list of commercial packages but they don't support FITS. Any help appreciated Andrew Chambers The University of Waikato New Zealand From fitsbits-request Mon Jul 11 02:36:54 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["651" "" "11" "July" "1994" "06:26:04" "GMT" "Bruce Cogan" "cogan at merlin.anu.edu.au" "<2vqolt$me2 at manuel.anu.edu.au>" "17" "Handling many small images" "^From:" nil nil "7" "1994071106:26:04" "Handling many small images" (number " " mark " Bruce Cogan Jul 11 17/651 " thread-indent "\"Handling many small images\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA11836; Mon, 11 Jul 94 02:36:54 EDT Return-Path: Message-Id: <2vqolt$me2 at manuel.anu.edu.au> Organization: Mt. Stromlo and Siding Spring Observatories Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!swrinde!ihnp4.ucsd.edu!munnari.oz.au!newshost.anu.edu.au!merlin.anu.edu.au!cogan From: cogan at merlin.anu.edu.au (Bruce Cogan) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Handling many small images Date: 11 Jul 1994 06:26:04 GMT One of our astronomers has a project in which he will be generating a large number (100's) of small (~10x10 pixel) images. He would like to archive these in FITS format, but dosn't want the overhead of a separate file and its header for each image. I would appreciate suggestions on the best way of storing these in FITS files, such that they would be readily accessible by standard reduction packages. -- Bruce Cogan cogan at mso.anu.edu.au Mt Stromlo Observatory Tel. +61 6 249 0232 Australian National University Fax +61 6 249 0233 Private Bag Weston Creek P.O. , ACT 2611, Australia From fitsbits-request Mon Jul 11 06:22:58 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2355" "Mon" "11" "July" "1994" "10:03:10" "+0000" "Clive Page" "cgp at star.le.ac.uk" "" "44" "FITS continuation problem" "^From:" nil nil "7" "1994071110:03:10" "FITS continuation problem" (number " " mark " Clive Page Jul 11 44/2355 " thread-indent "\"FITS continuation problem\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA13126; Mon, 11 Jul 94 06:22:58 EDT Return-Path: Message-Id: Mime-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Content-Length: 2354 From: Clive Page Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: FITS continuation problem Date: Mon, 11 Jul 1994 10:03:10 +0000 (GMT) I take the point that in-band signalling is undesirable - as characters like \ & - might sometimes occur as normal contents of the value string. But what about putting the continuation marker and count in the comment field? Something like this: LONGSTRG= 'This is a very long string value which will need to' / &1 = ' be continued because it clearly will not fit in its' / &2 = ' entirety on a single line' The presence of an ampersand (or whatever character turns out to be most acceptible) as the first non-blank character of a comment, followed by a line counter, will signal to suitable FITS readers that a set of lines that form a single long character-string value is coming its way. This should not upset existing FITS readers (they will just lost the remainder of the long string). Of course it doesn't solve the problem, as pointed out by Doug Tody, that programs that read and then write information in FITS files may be forced to re-order keywords in some circumstances. One way of getting around this might be to use an even more elaborate comment field, for example: LONGSTRG= 'This is a very long string value which will need to' / &LONGSTRG1 = ' be continued because it clearly will not fit in its' /&LONGSTRG2 = ' entirety on a single line' /&LONGSTRG3 With this format a clever FITS reader could reconstruct the original long string even if somewhere in the processing chain the order of the lines had been randomly scrambled. Unfortunately it leaves even less space for the value string, so even more of them will need to be continued on to two or more lines. This is, of course, not a fully developed proposal, just an idea that would seem to serve the purpose without violating the FITS Standard. No doubt it could be improved to simplifiy FITS readers, e.g. by flagging the last line of the continuation sequence in some way. ------------------------------------------------------------------------- Clive Page, Internet: cgp at star.le.ac.uk Dept of Physics & Astronomy, University of Leicester, Until Apr 95: Phone +44 533 523551 Leicester, LE1 7RH, Fax +44 533 550182 UK. After Aug 94: Phone +44 116 252 3551 Fax: +44 116 255 0182 From fitsbits-request Mon Jul 11 12:36:24 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1904" "Mon" "11" "July" "1994" "12:36:11" "-0400" "William Pence" "pence at tetra.gsfc.nasa.gov" "<199407111636.MAA26424 at tetra.gsfc.nasa.gov>" "41" "Re: Handling many small images" "^From:" nil nil "7" "1994071116:36:11" "Handling many small images" (number " " mark " William Pence Jul 11 41/1904 " thread-indent "\"Re: Handling many small images\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA13641; Mon, 11 Jul 94 12:36:24 EDT Return-Path: Message-Id: <199407111636.MAA26424 at tetra.gsfc.nasa.gov> From: William Pence Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Cc: pence at tetra.gsfc.nasa.gov Subject: Re: Handling many small images Date: Mon, 11 Jul 1994 12:36:11 -0400 Bruce Cogan (cogan at mso.anu.edu.au) wrote: > One of our astronomers has a project in which he will be generating > a large number (100's) of small (~10x10 pixel) images. He would like > to archive these in FITS format, but dosn't want the overhead of > a separate file and its header for each image. > > I would appreciate suggestions on the best way of storing these in > FITS files, such that they would be readily accessible by standard > reduction packages. > The most accessible way to store the images is to just put each image into a separate FITS file with a 10x10 primary array. Every reduction package will be able to read them, and the total size of even a 1000 files, at 5760 bytes each (the minimum size for a FITS header + data unit), is less than 6 MBytes which is not much to worry about these days on most systems. But this is not very efficient in terms of disk space and in I/O time if a program has to open and read many different files. The most efficient way to store the images would be to store tham all in a single FITS binary table extension. (Or the logically related sets of images could be grouped together in a small number of binary tables). Each row of the table could hold one image. The column that stores the images would need to be defined as a vector (e.g., TFORMn = '100I' to hold a 10x10 array of 16 bit integers), and the TDIMn keyword convention would need to be used to specify the dimentionality of the array (e.g., TDIMn = '(10,10) '). You could define as many other columns in the table as desired to store other descriptive parameters about the image, such as the coordinates, or date, or filter. We use this format a lot for our X-ray data files, but many reduction packages cannot readily read these images without preprocessing the FITS files to convert them into a standard primary array image or IMAGE extension. Bill Pence NASA/GSFC From fitsbits-request Mon Jul 11 16:39:13 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["654" "" "11" "July" "1994" "18:44:16" "GMT" "Ron Watkins" "ron at argus.lpl.arizona.edu" "<2vs3u0$a7e at news.CCIT.Arizona.EDU>" "12" "Re: Handling many small images" "^From:" nil nil "7" "1994071118:44:16" "Handling many small images" (number " " mark " Ron Watkins Jul 11 12/654 " thread-indent "\"Re: Handling many small images\"\n") "<199407111636.MAA26424 at tetra.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA13993; Mon, 11 Jul 94 16:39:13 EDT Return-Path: Message-Id: <2vs3u0$a7e at news.CCIT.Arizona.EDU> Organization: Lunar and Planetary Lab, U of AZ Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!howland.reston.ans.net!europa.eng.gtefsd.com!MathWorks.Com!yeshua.marcam.com!charnel.ecst.csuchico.edu!nic-nac.CSU.net!news.Cerritos.edu!news.Arizona.EDU!argus.lpl.Arizona.EDU!ron References: <199407111636.MAA26424 at tetra.gsfc.nasa.gov> From: ron at argus.lpl.Arizona.EDU (Ron Watkins) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: Handling many small images Date: 11 Jul 1994 18:44:16 GMT I would have thought the most efficient way would be to use a 3 dimensional primary array. This has the advantage of having a single header and there wont be any padding of the data space for EVERY array, rather only one padding at the end of all the arrays. If the arrays are all the same size, then a 3-D array might be the best storage mechanism. Ron W. -- Ron Watkins [ron at argus.lpl.arizona.edu] / /~~~~) / 931 Gould-Simpson / /____/ / University of Arizona / / / Tucson AZ. 85721 -- (602) 621-8606 (____ unar & / lanetary (____ ab. From fitsbits-request Mon Jul 11 20:05:21 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1190" "Mon" "11" "July" "1994" "17:39:22" "GMT" "Doug Tody" "tody at lepus.tuc.noao.edu" "<1994Jul11.173922.10619 at noao.edu>" "26" "Re: Handling many small images" "^From:" nil nil "7" "1994071117:39:22" "Handling many small images" (number " " mark " Doug Tody Jul 11 26/1190 " thread-indent "\"Re: Handling many small images\"\n") "<2vqolt$me2 at manuel.anu.edu.au>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA14261; Mon, 11 Jul 94 20:05:21 EDT Return-Path: Message-Id: <1994Jul11.173922.10619 at noao.edu> Organization: National Optical Astronomy Observatories Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!gatech!ncar!noao!lepus.tuc.noao.edu!tody References: <2vqolt$me2 at manuel.anu.edu.au> Reply-To: tody at noao.edu From: tody at lepus.tuc.noao.edu (Doug Tody) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: Handling many small images Date: Mon, 11 Jul 1994 17:39:22 GMT In article <2vqolt$me2 at manuel.anu.edu.au>, cogan at merlin.anu.edu.au (Bruce Cogan) writes: > One of our astronomers has a project in which he will be generating > a large number (100's) of small (~10x10 pixel) images. He would like > to archive these in FITS format, but dosn't want the overhead of > a separate file and its header for each image. > > I would appreciate suggestions on the best way of storing these in > FITS files, such that they would be readily accessible by standard > reduction packages. Hi Bruce, My recommendation would be to keep it simple. A few hundred images is not that many - the separate file approach should not be ruled out even if it is all overhead for such a small image. If greater efficiency is desired I would consider either using the IMAGE extension (if each image requires a separate header) or storing the small images as bands of a 3D image (if all can share the same header). The most general and efficient approach is to use a table, but this is overkill for such a simple dataset. - Doug -- Doug Tody National Optical Astronomy Observatories IRAF project tody at noao.edu P.O. Box 26732, Tucson, Arizona, 85726 iraf at noao.edu From fitsbits-request Mon Jul 11 22:07:44 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["868" "" "11" "July" "1994" "19:56:22" "GMT" "Timothy Kimball" "kimball at stsci.edu" "<2vs856$7if at marvel.stsci.edu>" "18" "Re: Handling many small images" "^From:" nil nil "7" "1994071119:56:22" "Handling many small images" (number " " mark " Timothy Kimball Jul 11 18/868 " thread-indent "\"Re: Handling many small images\"\n") "<2vs3u0$a7e at news.CCIT.Arizona.EDU>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA14319; Mon, 11 Jul 94 22:07:44 EDT Return-Path: Message-Id: <2vs856$7if at marvel.stsci.edu> Organization: Space Telescope Science Institute Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!caen!sol.ctr.columbia.edu!newsxfer.itd.umich.edu!gatech!swrinde!ihnp4.ucsd.edu!ames!ncar!noao!stsci!kimball References: <199407111636.MAA26424 at tetra.gsfc.nasa.gov> <2vs3u0$a7e at news.CCIT.Arizona.EDU> From: kimball at stsci.edu (Timothy Kimball) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: Handling many small images Date: 11 Jul 1994 19:56:22 GMT In article <2vs3u0$a7e at news.CCIT.Arizona.EDU>, ron at argus.lpl.Arizona.EDU (Ron Watkins) writes: |> I would have thought the most efficient way would be to use a 3 dimensional |> primary array. This has the advantage of having a single header and there |> wont be any padding of the data space for EVERY array, rather only one padding |> at the end of all the arrays. If the arrays are all the same size, then a |> 3-D array might be the best storage mechanism. |> Ron W. Yes, except for the keyword values that will be different for each image. You would need to handle this with: 1) a whole lot of individual keywords in the common header, or 2) a table extension listing keyword values for each plane of the primary array. So it's not as strightforward as just putting another axis in the image, unless the keyword values are common to all the images. -- tdk From fitsbits-request Tue Jul 12 16:17:23 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["6867" "Tue" "12" "July" "1994" "16:17:16" "-0400" "William Pence" "pence at tetra.gsfc.nasa.gov" "<199407122017.QAA16849 at tetra.gsfc.nasa.gov>" "147" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994071220:17:16" "FITS continuation keywords" (number " " mark " William Pence Jul 12 147/6867 " thread-indent "\"Re: FITS continuation keywords\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA16383; Tue, 12 Jul 94 16:17:23 EDT Return-Path: Message-Id: <199407122017.QAA16849 at tetra.gsfc.nasa.gov> From: William Pence Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Cc: pence at tetra.gsfc.nasa.gov Subject: Re: FITS continuation keywords Date: Tue, 12 Jul 1994 16:17:16 -0400 We've been following the discussion of the continuation proposal with interest and we appreciate the comments. Many variations on our proposed convention have been suggested, (both recently, and during the prior discusion of this issue in May 1993) but it appears to us that there is still no consensus that any of them is significantly better than our original proposal, So, at this point we are inclined to stick with our simple convention using blank continuation lines (although we could be persuaded to use the CONTINUE= alternative; see point 3 below). To be specific, this convention for encoding long string values is: a. Insert a backslash character (\) immediately before the closing quote character in the string value that is to be continued. b. Continue the string value, enclosed in single quotes, on the next keyword in the header c. The continuation keyword must contain ASCII blanks in columns 1-10 (i.e., the keyword name must be blank and there must be no equal sign in column 9). FITS readers that do not understand this continuation convention will simply treat the continuation lines as COMMENT keywords. A somewhat frivolous example of this usage is: LONGSTR = 'This is a very long string keyword value that \' 'is continued over three\' ' keywords in the FITS header.' or to give a more realistic example of the 'array descriptor' syntax to be used in the XTE FITS files: TDDES2 = 'E(X1L|X1R|X2L|X2R|X3L|X3R) & C("TDDMAC1(0:14)") & \' 'T([" at TIME",1024,0.00390625])' We don't wish to needlessly prolong the debate on this, but would like to comment on a few issues that have been raised: 1. Is the order of keywords in a FITS header significant? The only statement made in the NOST FITS Standard about keyword order is: "Except where specifically stated otherwise in this standard, keywords may appear in any order." and the only places that the order is specifically stated is for the required keywords at the beginning of each Header-Data Unit. We interpret this to mean that the creators of a FITS file are free to decide on the most appropriate order for the keywords, but it does not mean, in our view, that readers may come along later and arbitrarily rearrange the order of the keywords without risking the loss of some information. Clearly, if comment keywords have been inserted in the header at strategic places to explain the meaning of the preceding or following keywords, or if history keywords are inserted to document the order of the data processing operations, then rearranging the header keywords will lead to loss of information. Also, keywords may be put in a specific order to make them easier for humans to understand, or to make it more efficient for software to read them. So we believe that our convention that requires that the order of the continuation lines be preserved is not unprecedented nor violates the FITS Standard. 2. On the use of backslash (\) as a continuation character. We prefer to use the backslash because it is a rarely used ASCII character and is thus unlikely to be confused with a literal backslash within the string value. One can still write string keyword values that end in a backslash, and these will not be confused with our continuation convention as long as one of the 3 conditions listed above is violated (e.g., insert a space character between the backslash and the closing quote character, or make sure that the following keyword does not have a blank name). Some people objected to having any sort of continuation character at the end of the line, but we think it is important to have some sort of indicator to software that it should check the following keyword to see if it is a continuation. Otherwise the software would have to always check the next keyword every time it reads a string valued keyword. As was stated by several other people, the fact that the backslash has a different meaning in other contexts (e.g., C source code and Unix shell scripts) we believe is not a real source of confusion here. Some people suggested putting the continuation character outside the value field in the comment area at the end of the keyword. But this would require imposing new conventions on the format of the comment field in keywords that we would rather avoid. Also, some FITS readers simply ignore the comment field entirely, so putting significant information there could cause difficulties for them. If the continuation character is put in the value field itself then every FITS reader will be able to get at it easily. 3. Should the continuation keywords have a blank name, or some other name. This in our minds is the biggest open issue about this proposal. The blank keyword names will simply be treated as comments by FITS readers that don't understand this convention which we felt was advantagous (i.e. is the least harmful way to deal with an unknown convention). But if certain data systems strip comments from the header and segregate them from the other keyword=value keywords, then this is not so desirable. This is why we suggested, as an alternative, using a special keyword name for the continuation keywords (such as CONTINUE=). While a number of people did to prefer this alternative, the votes were not overwhelming in favor. IF ANYONE ELSE HAS A STRONG PREFERENCE BETWEEN THESE 2 OPTIONS, PLEASE LET US KNOW. We, on the other hand, do not like the idea of repeating the original keyword name on all the continuation cards. This could cause unpredicable results for FITS readers that do not understand this convention (would they return the value of the first keyword, or of the last?), and could be confusing if a FITS reader tried to delete the keyword, but didn't delete all the continuations. Under our convention, if a FITS reader just deleted the first keyword of a continuation sequence then the header would just be left with some orphaned continuation lines that are interpreted as comments and do no harm. 4. Why not use a more complicated 'keyword referencing' scheme to point to the next keyword in the continued string? Some of the alternative proposals are completely independent of the keyword order and still allow unlimited continuation lines. We had suggested such a scheme last year: KEYNAMEA= 'This is a very long keyword string value which \KEYWORDB\' KEYWORDC= ' of the FITS header.' KEYWORDB= 'continues over several lines\KEYWORDC\' Some other variations have been offered in the past week. But no one we've talked to has much interest in implementing such a complicated scheme, and neither do we. Our convention is much simplier and still serves our needs, so why introduce an even more complicated convention into FITS that very few people are interested in using? Regards, Bill Pence Arnold Rots From fitsbits-request Wed Jul 13 03:05:38 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2612" "Wed" "13" "July" "1994" "04:53:49" "GMT" "Doug Tody" "tody at lepus.tuc.noao.edu" "<1994Jul13.045349.5076 at noao.edu>" "48" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994071304:53:49" "FITS continuation keywords" (number " " mark " Doug Tody Jul 13 48/2612 " thread-indent "\"Re: FITS continuation keywords\"\n") "<199407122017.QAA16849 at tetra.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA16920; Wed, 13 Jul 94 03:05:38 EDT Return-Path: Message-Id: <1994Jul13.045349.5076 at noao.edu> Organization: National Optical Astronomy Observatories Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!concert!inxs.concert.net!taco.cc.ncsu.edu!gatech!ncar!noao!lepus.tuc.noao.edu!tody References: <199407122017.QAA16849 at tetra.gsfc.nasa.gov> Reply-To: tody at noao.edu From: tody at lepus.tuc.noao.edu (Doug Tody) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Wed, 13 Jul 1994 04:53:49 GMT In article <199407122017.QAA16849 at tetra.gsfc.nasa.gov>, William Pence writes: > 4. Why not use a more complicated 'keyword referencing' scheme to point > to the next keyword in the continued string? > > Some of the alternative proposals are completely independent of the > keyword order and still allow unlimited continuation lines. We had > suggested such a scheme last year: > > KEYNAMEA= 'This is a very long keyword string value which \KEYWORDB\' > KEYWORDC= ' of the FITS header.' > KEYWORDB= 'continues over several lines\KEYWORDC\' > > Some other variations have been offered in the past week. But no one > we've talked to has much interest in implementing such a complicated > scheme, and neither do we. Our convention is much simplier and still > serves our needs, so why introduce an even more complicated convention > into FITS that very few people are interested in using? Bill - The above is more or less the convention used in IRAF, except that we use numbers instead of A B C. It works fine and is independent of the keyword order. The only disadvantage is that it is hard to squeeze both the root keyword name and sequence number into an 8 character keyword. But we don't find we need to do this very often (there are only a few cases in all of IRAF), so it is not a big problem. Note that in IRAF, only certain applications are expected to understand this convention; generic FITS programs see only what appear to be separate keywords. Hence if we want to delete a "multiline keyword" we do something like delete keywords "wsv*" to delete all the wsvNNNN keywords with a generic image or FITS tool. The same operation applied to an image using your convention will delete only the initial header card, leaving ambiguous, nameless continuation cards in the header. Ditto for any other operation such as copying or printing a keyword. To properly deal with your headers, all general FITS tools would have to understand your special convention. This is not the case with the keyword based convention we use. You are free to use any convention you choose for your data so long as it is legal FITS - so long as you don't expect everyone else's FITS code to understand it. Anyhow, the point I wanted to make is that we don't agree that your convention is "much simpler", or that the numbered continuation approach is "something more complicated which very few people are interested in using". - Doug -- Doug Tody National Optical Astronomy Observatories IRAF project tody at noao.edu P.O. Box 26732, Tucson, Arizona, 85726 iraf at noao.edu From fitsbits-request Wed Jul 13 04:50:28 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["887" "Wed" "13" "July" "1994" "07:02:48" "GMT" "Nick Szabo" "szabo at netcom.com" "" "21" "Customized data compression needs" "^From:" nil nil "7" "1994071307:02:48" "Customized data compression needs" (number " " mark " Nick Szabo Jul 13 21/887 " thread-indent "\"Customized data compression needs\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA16976; Wed, 13 Jul 94 04:50:28 EDT Return-Path: Message-Id: Organization: NETCOM On-line Communication Services (408 261-4700 guest) Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!newsxfer.itd.umich.edu!zip.eecs.umich.edu!yeshua.marcam.com!charnel.ecst.csuchico.edu!csusac!csus.edu!netcom.com!szabo From: szabo at netcom.com (Nick Szabo) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Customized data compression needs Date: Wed, 13 Jul 1994 07:02:48 GMT I'm embarking on a project, one of the applications of which is developing data compression algorithmns optimized for specific kinds of data. By taking advantage of domain specific pattern knowledge (eg redundancy in the genetic code for DNA, patterns in astronomical phenomenon, etc.) customized compression algorithms can in some cases provide a substantial improvement over general-purpose algorithms like Lempel-Ziv. I'd like to find out what data formats in the scientific community would most benefit from customized compression. Specifically, * What formats account for the largest volume of data transmitted over networks? * What kinds of data have the greatest amount of redundancy (especially domain-specific redundancy that wouldn't be caught by a normal compression algorithm)???? much thanks, -- N1ck Szab0 szabo at netcom.com Big Brother is parsing you. From fitsbits-request Wed Jul 13 07:14:11 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["2014" "Wed" "13" "July" "1994" "07:14:03" "-0400" "Arnold Rots" "arots at xebec.gsfc.nasa.gov" "<199407131114.HAA09807 at xebec.gsfc.nasa.gov>" "40" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994071311:14:03" "FITS continuation keywords" (number " " mark " Arnold Rots Jul 13 40/2014 " thread-indent "\"Re: FITS continuation keywords\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA18221; Wed, 13 Jul 94 07:14:11 EDT Return-Path: Message-Id: <199407131114.HAA09807 at xebec.gsfc.nasa.gov> From: Arnold Rots Sender: fitsbits-request at fits.CV.NRAO.EDU To: tody at noao.edu Cc: fitsbits at NRAO.EDU Subject: Re: FITS continuation keywords Date: Wed, 13 Jul 1994 07:14:03 -0400 Doug Tody wrote: > In article <199407122017.QAA16849 at tetra.gsfc.nasa.gov>, William Pence writes: > > 4. Why not use a more complicated 'keyword referencing' scheme to point > > to the next keyword in the continued string? > > > > Some of the alternative proposals are completely independent of the > > keyword order and still allow unlimited continuation lines. We had > > suggested such a scheme last year: > > > > KEYNAMEA= 'This is a very long keyword string value which \KEYWORDB\' > > KEYWORDC= ' of the FITS header.' > > KEYWORDB= 'continues over several lines\KEYWORDC\' > > > > Some other variations have been offered in the past week. But no one > > we've talked to has much interest in implementing such a complicated > > scheme, and neither do we. Our convention is much simplier and still > > serves our needs, so why introduce an even more complicated convention > > into FITS that very few people are interested in using? > Bill - The above is more or less the convention used in IRAF, except that > we use numbers instead of A B C. It works fine and is independent of the > keyword order. The only disadvantage is that it is hard to squeeze both > the root keyword name and sequence number into an 8 character keyword. > But we don't find we need to do this very often (there are only a few cases > in all of IRAF), so it is not a big problem. The big problem with using numbers (and, hence, the general applicability of your scheme) is, as pointed out in an earlier message, that it does not work for column-related keywords - or any other keyword that *has* to end in a digit. This, by the way, is a situation where one really gets into a crunch: a mandatory/customary "T", three digits for the column number, at least one character for a sequence indicator - and there is very little left to indicate what the keyword is really about, especially if it follows some kind of convention that prescribes yet another mandatory character. - Arnold Rots From fitsbits-request Wed Jul 13 09:46:41 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1604" "Wed" "13" "July" "1994" "13:14:01" "GMT" "Doug Tody" "tody at lepus.tuc.noao.edu" "<1994Jul13.131401.18344 at noao.edu>" "30" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994071313:14:01" "FITS continuation keywords" (number " " mark " Doug Tody Jul 13 30/1604 " thread-indent "\"Re: FITS continuation keywords\"\n") "<199407131114.HAA09807 at xebec.gsfc.nasa.gov>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA18356; Wed, 13 Jul 94 09:46:41 EDT Return-Path: Message-Id: <1994Jul13.131401.18344 at noao.edu> Organization: National Optical Astronomy Observatories Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!newsxfer.itd.umich.edu!gatech!ncar!noao!lepus.tuc.noao.edu!tody References: <199407131114.HAA09807 at xebec.gsfc.nasa.gov> Reply-To: tody at noao.edu From: tody at lepus.tuc.noao.edu (Doug Tody) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: Wed, 13 Jul 1994 13:14:01 GMT In article <199407131114.HAA09807 at xebec.gsfc.nasa.gov>, Arnold Rots writes: > > The big problem with using numbers (and, hence, the general > applicability of your scheme) is, as pointed out in an earlier > message, that it does not work for column-related keywords - or any > other keyword that *has* to end in a digit. This, by the way, is a > situation where one really gets into a crunch: a mandatory/customary > "T", three digits for the column number, at least one character for a > sequence indicator - and there is very little left to indicate what > the keyword is really about, especially if it follows some kind of > convention that prescribes yet another mandatory character. > > - Arnold Rots Arnold, this is a good point. I was not proposing this as a fully general mechanism however. Extending FITS to allow any string keyword to be continued arbitrarily is a pretty major change and I was not proposing anything of the sort. We use this mechanism in IRAF only for a few special cases and in these cases the 8 character keyword limit, while a nusiance, is not a serious problem. Frankly, in the cases where we do this there can be a lot of data and we would be better off using a table - we use the multiple keyword technique only because we don't do this often enough to warrant the more complex and general approach (a table). For most normal header strings we live within the 68 character limit. - Doug Doug Tody National Optical Astronomy Observatories IRAF project tody at noao.edu P.O. Box 26732, Tucson, Arizona, 85726 iraf at noao.edu From fitsbits-request Wed Jul 13 10:33:29 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["5719" "Wed" "13" "July" "1994" "10:33:27" "-0400" "Ian M. George" "GEORGE at lheavx.gsfc.nasa.gov" "<940713103327.20207952 at lheavx.gsfc.nasa.gov>" "112" "OFWG Recommendation R11 - Exposure-time related keywords in OGIP FITS files" "^From:" nil nil "7" "1994071314:33:27" "OFWG Recommendation R11 - Exposure-time related keywords in OGIP FITS files" (number " " mark " Ian M. George Jul 13 112/5719 " thread-indent "\"OFWG Recommendation R11 - Exposure-time related keywords in OGIP FITS files\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA18425; Wed, 13 Jul 94 10:33:29 EDT Return-Path: Message-Id: <940713103327.20207952 at lheavx.gsfc.nasa.gov> From: "Ian M George, Code 668, NASA/GSFC, USA (Usque ad mortem bibendum)" Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Cc: GEORGE at lheavx.gsfc.nasa.gov Subject: OFWG Recommendation R11 - Exposure-time related keywords in OGIP FITS files Date: Wed, 13 Jul 1994 10:33:27 -0400 (EDT) At a meeting on 1994 Jul 06, the OGIP FITS Working Group (OFWG) raised its earlier preliminarily recommendation p20.1 regarding exposure-time related keywords for OGIP FITS files to the lofty status of a full recommendation: --------------------------------------------------------------------------- OFWG Recommendation R11 ------------------------------------------------------ | EXPOSURE-TIME RELATED KEYWORDS FOR OGIP FITS FILES | ------------------------------------------------------ Version: 1994 May 11 Below are two sets of FITS keywords & definitions: Table 1 lists the keywords which should be used in OGIP files to store various 'exposure times' associated with a FITS dataset; Table 2 lists the related keywords which should be used to store various correction factors to these times. This set of keywords & definitions was constructed from the most common usage within X- & Gamma-ray FITS files. The lack of a consistent naming convention most probably reflects the fact that most of these keywords were devised by different people in different places at different times. The OFWG strongly recommends that these keywords (only) be used to store the various quantities within OGIP FITS files. It should be noted that only those keywords/quantities considered necessary must be included in the header of a given FITS dataset. However the OFWG suggests that where possible all know quantities are included in the header. TABLE 1 - "Exposure-time" related keywords ****************************************** Keyword Meaning & Notes ------- --------------- TELAPSE is the time interval (in seconds) obtained as difference between the start and stop times of an observation. Any gaps due to Earth occultation, or high background counts and/or other anomalies, are included. ONTIME is the total "good" time (in seconds) on 'source'. If a 'Good Time Interval' (GTI) table is provided, ONTIME should be calculated as the sum of those intervals. Corrections for instrumental 'dead time' effect are NOT included. LIVETIME is the total time (in seconds) on source, corrected for the 'total' instrumental dead time effect. The ratio LIVETIME/ONTIME therefore gives the dead time correction value (which hence lies in the range 0.0 to 1.0). EXPOSURE is the total time (in seconds) on source, corrected for any relevant quantity used to calculate the corrected count rate. The value can include correction which are not directly related with time (e.g. collimation efficiency or vignetting). This keyword used in the header of FITS file, is a mean value when appropriate. TABLE 2 - Keywords for "Exposure-time" correction factors ********************************************************* Keyword Meaning & Notes ------- --------------- DEADC is the total correction factor for any dead time effect (ie LIVETIME/ONTIME), and lies in the range 0.0 to 1.0. Thus the multiplication of this value by the ONTIME value gives the 'effective' integration time or LIVETIME (TIMEDEL in the case of a light curve). Since the total dead time of a given dataset can be the result of a multitude of instrumental/processing effects (especially related to the particular experiment, and/or processing by an onboard computer, and/or spacecraft operations), it is recommended that as well as including the total correction factor in the DEADC keyword, instrument-specific keywords are used to keep a record of the original value for the individual correction factors. ERRCOR Corrections are usually applied to the stored counts and the error on those counts. However, in some cases the errors can need an extra correction, different to that applied to the counts. The ERRCOR keyword contains this extra value. The value is unitless. Es. If the dead time is related to the sampling of the on board computer, it has been found that different correction are needed for counts (count/s) and errors. This can be generalized to all cases in which the error needs to be treated differently compared to the counts. VIGNET is the correction factor for the collimator response or vignetting function (depending upon the instrument) such as to enable the scientific dataset (eg lightcurve) to be rescaled to that which would have been obtained had the source been observed at the angle of peak collimator response (for collimated instruments) or on-axis (for instruments employing imaging optics). In both cases the value of VIGNET should lie in the range 0.0 - 1.0, and thus be the factor by which the scientific dataset should be divided in order to obtain the equivalent on-axis value. In both cases (but especially in the case of imaging optics) the correction factor can be a function of energy, and thus can only be used if a useful mean value can be defined. In the case of imaging optics, VIGNET should contain the the total correction factor due to vignetting and any (energy-independent) reduction in collecting area resulting from obscuration by the near-side of the mirror, support structures etc. It should be noted that historically these correction factors have often [naturally] been referred to by different names, and sometimes as the reciprical of the value defined above. ----------------------------------- END ---------------------------------- Ian M George NASA/GSFC OFWG 1994 Jul 11 From fitsbits-request Wed Jul 13 10:31:00 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1243" "Wed" "13" "July" "1994" "10:31:00" "-0400" "Ian M. George" "GEORGE at lheavx.gsfc.nasa.gov" "<940713103100.20207952 at lheavx.gsfc.nasa.gov>" "28" "OFWG Recommendation R9 - The OGIP data quality flag convention" "^From:" nil nil "7" "1994071314:31:00" "OFWG Recommendation R9 - The OGIP data quality flag convention" (number " " mark " Ian M. George Jul 13 28/1243 " thread-indent "\"OFWG Recommendation R9 - The OGIP data quality flag convention\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA18413; Wed, 13 Jul 94 10:31:00 EDT Return-Path: Message-Id: <940713103100.20207952 at lheavx.gsfc.nasa.gov> From: "Ian M George, Code 668, NASA/GSFC, USA (Usque ad mortem bibendum)" Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Cc: GEORGE at lheavx.gsfc.nasa.gov Subject: OFWG Recommendation R9 - The OGIP data quality flag convention Date: Wed, 13 Jul 1994 10:31:00 -0400 (EDT) At a meeting on 1994 Jul 06, the OGIP FITS Working Group (OFWG) raised its earlier preliminarily recommendation p20.3 regarding the values used to flag good & bad data within FITS datasets produced by the OGIP to the lofty status of a full recommendation: --------------------------------------------------------------------------- OFWG Recommendation R9 ----------------------------------------- | THE OGIP DATA QUALITY FLAG CONVENTION | ----------------------------------------- Version: 1994 Jul 06 The meaning and use of quality flags is often specific to a given instrument and/or data type. However, when devising quality flags for new FITS files, the OFWG recommends that the integer value 0 (zero) be used to indicate that the data to which the flag refers is of GOOD quality, and is to be automatically used by downstream software. An instrument/dataset/application-specific scale of non-zero quality flag values can be used to indicate why the data is considered of bad quality, the degree of 'badness', and data flagged to be ignored in downstream applications by the user. ----------------------------------- END ---------------------------------- Ian M George NASA/GSFC OFWG 1994 Jul 11 From fitsbits-request Wed Jul 13 17:17:57 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["1550" "" "13" "July" "1994" "16:59" "EDT" "BARRY M. SCHLESINGER" "bschlesinger at nssdca.gsfc.nasa.gov" "<13JUL199416595914 at nssdca.gsfc.nasa.gov>" "36" "Re: FITS continuation keywords" "^From:" nil nil "7" "1994071320:59:00" "FITS continuation keywords" (number " " mark " BARRY M. SCHLESIN Jul 13 36/1550 " thread-indent "\"Re: FITS continuation keywords\"\n") "<2vl48b$k7k at explorer.clark.net>"] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA19292; Wed, 13 Jul 94 17:17:57 EDT Return-Path: Message-Id: <13JUL199416595914 at nssdca.gsfc.nasa.gov> Organization: NASA - Goddard Space Flight Center Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!news.umbc.edu!haven.umd.edu!ames!newsfeed.gsfc.nasa.gov!nssdca.gsfc.nasa.gov!bschlesinger References: <199407051645.MAA00598 at tetra.gsfc.nasa.gov> <2vl48b$k7k at explorer.clark.net> From: bschlesinger at nssdca.gsfc.nasa.gov (BARRY M. SCHLESINGER) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: Re: FITS continuation keywords Date: 13 Jul 1994 16:59 EDT In article <2vl48b$k7k at explorer.clark.net>, leb at clark.net (Lee E. Brotzman) writes... >When one defines a new keyword, one must also define the keyword's value, either >integer, floating point, string, etc. Therefore, to have "long strings" one >simply defines the keyword syntax as a string and that the keyword can appear >more than once, like COMMENT. There is no specific prohibition against >repeated occurences of a keyword, so as far as I can tell, this is a legal FITS >solution to the "problem" (which, of course, I still do not believe really >exists). > There is no specific prohibition against repeated occurrences of a keyword. >It may even be better to define the keyword without the '=', thereby creating a >new form of COMMENT/HISTORY. I'll leave that to Barry to see if it is legal It is permitted. The meaning is that columns 9-80 of the card image are comment. Users may define their own keywords that have only comment. The controversial HIERARCH is an example. >(frankly, his message talking about 'no value' for keywords without '=' was a >little confusing to me, just what does 'no value' mean?). > The operational definition is that a value field, following an "= " in columns 9-10 must be of one of the specified types and follow the format prescriptions for the type, for example, character strings enclosed in single right quotes. Comment fields are not limited. A conceptual definition has not been stated and would require some careful thought. Barry Schlesinger NOST FITS Support Office From fitsbits-request Wed Jul 13 18:24:44 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["151" "Wed" "13" "July" "1994" "13:20:28" "-0500" "Steve Mazuk" "Mazuk at courier2.aero.org" "" "4" "FTP Site for FAQ archive?" "^From:" nil nil "7" "1994071318:20:28" "FTP Site for FAQ archive?" (number " " mark " Steve Mazuk Jul 13 4/151 " thread-indent "\"FTP Site for FAQ archive?\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA19430; Wed, 13 Jul 94 18:24:44 EDT Return-Path: Message-Id: Organization: The Aerospace Corporation Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!europa.eng.gtefsd.com!howland.reston.ans.net!swrinde!elroy.jpl.nasa.gov!news.aero.org!news2.aero.org!sheila.aero.org!user From: Mazuk at Courier2.Aero.Org (Steve Mazuk) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: FTP Site for FAQ archive? Date: Wed, 13 Jul 1994 13:20:28 -0500 Where is the FTP archive for the FAQ for this group? My site doesn't have the FAQ and the SCI.ASTRO site doesn't appear to have this group's FAQ. From fitsbits-request Thu Jul 14 12:07:01 1994 Status: RO X-VM-v5-Data: ([nil nil nil nil nil nil nil nil nil] ["664" "Thu" "14" "July" "1994" "14:38:24" "GMT" "Rob Hewett" "hewett at cfabuh.harvard.edu" "" "24" "How to mount Digitized Sky Survey CDROM???" "^From:" nil nil "7" "1994071414:38:24" "How to mount Digitized Sky Survey CDROM???" (number " " mark " Rob Hewett Jul 14 24/664 " thread-indent "\"How to mount Digitized Sky Survey CDROM???\"\n") nil] nil) Received: by fits.cv.nrao.edu (4.1/DDN-DLB/1.5) id AA21365; Thu, 14 Jul 94 12:07:01 EDT Return-Path: Message-Id: Organization: Smithsonian Astrophysical Observatory, Cambridge, MA, USA Path: saips.cv.nrao.edu!hearst.acc.Virginia.EDU!darwin.sura.net!howland.reston.ans.net!europa.eng.gtefsd.com!MathWorks.Com!news.duke.edu!godot.cc.duq.edu!newsfeed.pitt.edu!ctc.com!news.cs.umb.edu!hsdndev!cfanews!cfabuh!hewett From: hewett at cfabuh.harvard.edu (Rob Hewett) Sender: fitsbits-request at fits.CV.NRAO.EDU To: fitsbits at fits.CV.NRAO.EDU Subject: How to mount Digitized Sky Survey CDROM??? Date: Thu, 14 Jul 1994 14:38:24 GMT This question is not a FITS question but I thought some s.a.f reader might know: How do you mount a CDROM from STSI's Digitized Sky Survey on a Sun running either SunOS 4 or 5? This did not work on OS