Date: Tue, 02 Sep 2008 13:55:11 -0400 From: William Pence To: FITSBITS Subject: Start of the FITS Region File Public Comment Period This is to announce the start of the 30-day Public Comment Period on the FITS Region File convention which has been submitted for inclusion in the Registry of FITS Conventions. This is the 12th in a growing series of conventions submitted to the Registry which is maintained by the IAU FITS Working Group. Detailed information about this convention and a sample FITS file that uses it are available for public review and comment from the FITS registry web page at http://fits.gsfc.nasa.gov/fits_registry.html A FITS Region file is a FITS binary table that defines a spatial region of a 2-dimensional image. The region file is often used to define an area that is to be included or excluded from certain data processing operations on the image. The region is specified as the union or intersection of geometric shapes, such as 'circle' or 'rectangle'. The REGION table is the FITS equivalent of the ASCII text region file that has long been used by the ds9 image processing program and its predecessor, SAOimage. Comments may be posted here on the FITSBITS mail exploder or the sci.astro.fits newsgroup. Minor typographical issues may be sent directly to the authors of the convention. Bill Pence (on behalf of the IAU FITS Working Group) ===================================================================== From: Craig Markwardt Date: 03 Sep 2008 10:43:00 -0400 To: fitsbits@donar.cv.nrao.edu Subject: Re: [fitsbits] Start of the FITS Region File Public Comment Period William Pence writes: > This is to announce the start of the 30-day Public Comment Period on > the FITS Region File convention which has been submitted for inclusion > in the Registry of FITS Conventions. This is the 12th in a growing > series of conventions submitted to the Registry which is maintained by > the IAU FITS Working Group. Glad to see this is being documented! Section 2 * "The names of the columns (TTYPEi) indicate the coordinate system in use, in the usual way, and these coordinate names are to be identified in the MFORM1 keyword (MFORM1='X,Y'); the mandatory keyword MTYPE1 provides a name for the coordinate pair." Comment: I'm not sure what the "usual way" refers to. I think it means, "using the CTYPEi coordinate labels of the source image." I'm also unsure of what MFORM1 is meant to mean (this is a Chandra-specific convention). Is 'X,Y' a placeholder or literal? What are the allowed values of MTYPE1, or can it be anything? * "The following columns are optional..." Comment: Is there any convention for null values? It seems there could be many 'unused' fields in a region file. For example, if there is a region composed of the union of a circular and rectangle, the "R" column will be used for the circle but not the rectangle. Also, the rectangle will require X and Y to be 4-vectors, but the circle will only use one X and Y value each. Are the extraneous values simply ignored? General comment: Region files are image and projection dependent. For example, if someone lays down a circular region on a tangent plane projection image in DS9, the same region will not be circular in another projection. The situation is even more complicated for non-circular regions. This means that region files are not globally interchangeable, but rather specific to a particular analysis with a known image and known projection. I think this should at least be noted. Craig Markwardt ===================================================================== Date: Thu, 4 Sep 2008 20:44:19 +0100 (BST) From: "Malcolm J. Currie" To: FITSBITS Subject: Re: [fitsbits] Start of the FITS Region File Public Comment Period I'd rearrange the Definition to say earlier what a REGION table is and offers, for anyone else who might wish to adopt this convention. At the moment the introduction tells how it's implemented before saying what is it. It doesn't state to which data the region is applicable. Is the convention limited to the primary data array or does it include any IMAGE extensions? If not, is there a way to association a region with a specific array? Can there be more than one REGION table in a FITS file? It assumes more prior knowledge of the reader than I would expect. For example, what's GTI? It would help to have a reference or URL to GTI, the IVOA STC Region syntax, OGIP/94-006, ACIS. Please can "in the usual way" in the first bullet of Section 2 be made explicit. It wasn't clear which if any keywords are defined by the convention, such as MFORM1. In the example file I noticed an MFORM2 keyword. Required keywords from other conventions like HDU* and MFORMn should be referenced. Aren't the long-string convention headers (even all the optional headers) in Section 3 a distraction from the essence of the convention? How should the unused values in the co-ordinate vectors be filled, NaN, or doesn't it matter as you can work out how many to read given the shape? The sense and origin of ROTANG isn't always stated. A factorisation definition appearing Section 2 ROTANG rather than the imprecise "rotation angle" would help. It would be prudent to state the units, although a FITS user might assume degrees. Could the purpose and use of COMPONENT be elaborated. The appears to be some duplicated text in Section 4. "The value of SHAPE...". In the polygon definition can you use the same notation as in Table 1, e.g. X subscript 0 rather than X[0]. Also indexing from 0 looks odd for FITS. Last sentence of Section 4. Should "need to be" be "must be". Could the example in Section 5 have some floating-point values too? Minor gripes: Yuk to "soccer". You mean association football or perhaps you should use tennis. (-: Missing hyphens in Section 5: "30-degree sector" and "40-degree sector", "left-hand side" and "right-hand side". Now a little off topic as we're only supposed to discuss the description not the actual convention. However, I did wonder why MFORM1 was "X,Y" rather than a list of axis numbers. That would make it easier to extend to more dimensions. Also have you considered other operators like XOR or EQV? I wondered if it were possible to export a Starlink Ascii Region Definition (ARD) using this convention. Most could export but not all, as ARD allows more than two dimensions and has extra operators. Hobbyhorse: I'd have used EXTTYPE = 'REGION', and EXTNAME for the specific instances if you have more than one set of regions. Malcolm Currie ===================================================================== Date: Mon, 8 Sep 2008 13:56:52 +0200 (CEST) From: "LC's NoSpam Newsreading account" To: FITSBITS Subject: Re: [fitsbits] Start of the FITS Region File Public Comment Period > Comments may be posted here on the FITSBITS mail exploder or the > sci.astro.fits newsgroup. 1) General. The document submitted to the registry is rather informal and/or project specific. It could be accepted as supplementary information demonstrating the usage, but a more formal description should be submitted as primary document for the registry 2) rotation angle The direction and units of the rotation angle are either defined too late with respect to the first mention, or not defined (I cannot find a spec whether the angle shall be in degrees or can be in any units specified in TUNITn 3) definition of header keywords This is one place whether more formality is necessary not only as a matter of writing style. One should define ONLY the kwds really specific of this convention, more than give a complete header dump. Most of the keywords not flagged "r" or "o" (hence mandatory) looks to me NOT specific of this convention but just mandated by the binary table FITS standard. Also columns like e.g. ORIGIN DATE CREATOR seems for mere documentation purposes (less clear for CONTENT which is not a FITS standard reserved kwd) and I doubt that their presence or absence might invalidate the semantics of the region files. Similarly for CHECKSUM and DATASUM. I presume that these kwds, as most of the other "optional" ones (with the exception of the LONGSTRN family, which is referring to a *DIFFERENT* registered convention) might be required in the specific project implementation (Chandra ?) but are completely irrelevant for a general purpose image file Similarly the reference to the SOURCE and GRATING kwds appears totally project specific and should disappear from a general purpose document (saying instead that additional project specific columns might be associated to a region ?) The issue of "WCS tie to the REGION" which is marked as merely recommended is instead worth of a detailed description. The function of the MTYPE and MFORM mandatory kwds is not explained. 4) redundant shapes It looks to me that some shape families are redundant, aren't they just a different name or a different way to refer to the same thing. This seems to add confusion in a standard. What's the difference between a diamond and rhombus (or their rot variants) ? Just the name ? What's the difference between a box and rectangle (or their rot variantes) ? They can be mapped into each other. Also the diamond and rhombus could probably be mapped to a 45 deg rotated square box, can't they ? 5) definition of polygon seems unclear. I presume m cannot be < 3 (should be said) Also do I interpret correctly that, if one uses a column depth such that e.g. the polygon with the largest number of sides is an hexagon, a triangle will be defined with X0 X1 X2 NaN NaN NaN OR X0 X1 X2 X0 X0 X0 ? 6) example (tab. 2) the role of "components" is unclear to me 7) example file region.fits I have displayed the file with ds9 and I'm rather puzzled. I see three regions (a circle, a rotated ellipse and an UNROTATED rectangle). I see that the image is non-zero only in 3 areas, the inside of the circle and ellipse and a different ROTATED rectangle partially overlapping the rectangular region. It is not clear to me whether the image is intrinsically non-zero only in such area already in the HDU, or whether I'm being shown the image filtered through the region. And in general (see next item) whether I should expect a REGION extension always to be tied to a primary image in the same FITS file, or whether a FITS file with no primary HDU and just a REGION extension could be applied to potentially any image. 8) scope and name of the convention The comments above (with the exception of 2/4/5/6) may partly go beyond the formal comments of the sort "is the document complete and understandable enough to be included in the registry", but may reflect my unability to understand whether the convention proposed is a general or project-specific one, and how should be used. Now formally "usage comments" are not relevant for inclusion in the registry, so I try to separate such comments in the next point. However, my impression is that the convention is rather project-specific (or context-specific) more than general. Now it is perfectly legitimate to include in the registry a project-specific convention. However one should not give a false impression that such a convention is general, so I suggest to use a different name (e.g. "Chandra spatial region file" convention or any other choice at discretion of the authors) 9) usage of FITS Region binary table So far I have used only, though extensively, ds9 region files in ASCII form. And perhaps mainly in a non-customary way (not for "data filtering" but for "display purposes"). I've also used the region concept outside of ds9 (e.g. in some applets talking with a database server). For these reasons I fail to see whether the proposed convention is general enough to cope with my way of using regions. - a first comment is that using a FITS file for a region file with respect to an ASCII file may not always be convenient : - an ASCII file can be easily created manually with an editor - a FITS file is convenient when it is more compact than an ASCII file, which occurs (only) when the size of the file is dominated by the data and not by the overhead of header kwds - a FITS extension with the region could be attached to a specific image (or family of image extension) into a single file - an ASCII file can be loaded onto several images (is this possible in ds9 with FITS region files without primary data array ?) - a second comment is that I tend to use much more region files in celestial coordinates than in pixels. And to overlay such files onto DIFFERENT images taking advantage of the WCS of the images. Will this be possible with the convention under discussion ? Is it just a matter of saying MFORM1='RA,DEC' and having the shape reference points in two columns named RA,DEC and all other info in appropriate TUNITs ? - the other comment is more a question of the sort : could my current usage model (described below) be mapped into the convention under discussion ? What I often do is the following : - I have a database with a primary table (say a list of X-ray sources with id,ra,dec etc. ... let's call them Xid,Xra,Xdec) - the same database contains other tables (say a list of optical sources with Oid,Ora,Odec, a list of radio sources with Rid,RRa,Rdec, etc. etc. - then I have a "glorified correlation table" tying all this together. Such a table may look like row n Xid=1 Oid=22 Rid=123 ... row n+1 Xid=1 Oid=24 Rid=null ... row n+2 Xid=2 Oid=77 Rid=456 ... i.e. X-ray source 2 has a single "counterpart set" associated to it, while X-ray source 1 has here two "counterpart sets". A counterpart set may have non-null entries in some of the member tables only - finally I have some data products which may vary from X-ray images to various optical thumbnails or radio mini-maps in the surrounding of the X-ray target. - what I want to do is to display one thumbnail image, and seeing all counterpart sets overlaid onto it e.g. - a single red thick circle at the X-ray position with radius the X-ray error radius (this applies to the unique common Xid, one for all counterpart sets of the same object) - a green rotated ellipse for each extended radio source (Rid not null and flagged as extended in its db table) with ellipse parameters being radio observation parameters stored in the table - a green box for the pointlike radio sources, with sizes given by ra, dec errors from the radio db table - a blue circle for sources in a given optical catalogue, with radius function of the magnitude in the I band (smaller when fainter) - a white circle for sources in an IR catalogue with radius function of flux etc. etc. - and of course I would need to tell that a particular white circle is associated to a particular blue circle or green box (their sources belong to the same "counterpart set") I originally used to generate ASCII region files with tags and using ds9 "region groups" for this purpose. Now that was a little clumsy. It did not allow for instance easy reverse interaction (clicking on a region on the image and telling which object it is, or clicking on an entry in a tabular listing and hilighting the region on the image). I currently use an applet as client of a server, and pass "logical regions" along a socket using a custom protocol. Note that I do not need instead any custom protocol for the images. The applet receives the URL of a FITS image file, opens a stream to it, and implements a minimal image reader. So you can see that I'd appreciate the existence of a GENERAL FITS convention for region files. I'd welcomed such a thing ! But I fail to see whether the convention under examination is general enough (I suspect not). Note that in my usage some aspects of the shapes (the kind of shape itself, or display items like the colour and thickess) are associated to a particular database table (i.e. band or kind of source), and some other spatial parameters of the shape CAN be NOT associated to an intrinsic spatial characteristics but map some other source property (e.g. magnitude or flux). Lucio Chiappetti ===================================================================== From: Craig Markwardt Date: 08 Sep 2008 12:22:03 -0400 To: fitsbits@donar.cv.nrao.edu Subject: Re: [fitsbits] Start of the FITS Region File Public Comment Period William Pence writes: > A FITS Region file is a FITS binary table that defines a spatial region > of a 2-dimensional image. The region file is often used to define an > area that is to be included or excluded from certain data processing > operations on the image. The region is specified as the union or > intersection of geometric shapes, such as 'circle' or 'rectangle'. The > REGION table is the FITS equivalent of the ASCII text region file that > has long been used by the ds9 image processing program and its > predecessor, SAOimage. Another comment: The sample fits header lists TCRPXn, TCRVLn and TCDLTn as required keywords for RA and DEC columns. Are these really required? Does that mean that one cannot simple have a floating point (1E) column describing the RA and Dec? I note that X, Y and R are all in pixel space as well, which means that one cannot define the position and radius in physical units? Craig Markwardt ===================================================================== Date: Tue, 9 Sep 2008 04:04:31 +0200 (CEST) From: Thierry Forveille To: "LC's NoSpam Newsreading account" Cc: FITSBITS Subject: Re: [fitsbits] Start of the FITS Region File Public Comment Period I agree with Lucio that this convention poses a policy/strategy problem: all conventions we have examined to date either solved a general problem for good (to quasi-unanimous satisfaction), or (mostly) were clearly of historical or very specialized interest. Here we instead have something which attacks a problem of very broad interest for the first time, but (quite naturally) has not yet converged to a generally acceptable solution. My strong preference would be for a small working group to start work on a future standard for the general problem of specifying regions within FITS (no, I am not volunteering ;-)), but I don't know what to do with the existing convention in the mean time. On the one hand such data files might be out in the wild and documenting them can only be a good thing, but on the other hand officializing the convention would spread usage of what most likely will be an interim stage of the actual standard.