Date: Thu, 18 Sep 2008 14:00:07 -0400 From: Norbert Zacharias To: fitsbits@donar.cv.nrao.edu Subject: [fitsbits] comment on SIP, optical distortion Dear Bill Pence and FITS community, reading about the proposed SIP conventions I would like to draw your attention to the most commonly seen, pure optical distortion (3rd order) term. This could be expressed with the SIP mechanism, however typically a single coefficient is applicable for many telescopes to control "distortion" (besides the knowledge of the "center" i.e. the location of the optical axis on the detector, which I see is already handled). Let D3 be the coefficient of (3rd order) optical distortion, and dx, dy the corrections to be applied to the x, y coordinates (e.g. in pixel unit), then dx = D3 * x * r**2 dy = D3 * y * r**2 with r**2 = x**2 + y**2 The SIP mechanism is more general but has the disadvantage that users (or people working on the calibrations of a detector) often try to solve for a full 3rd order polynomial, while the data might be better represented with just that single D parameter in order to keep the error propagation low. The D3 parameter is used since at least a century in the astrometric community, and having a FITS keywork mechanism for it as alternative option to the SIP for many applications would be good to have. Norbert ------------------------------------------------ Norbert Zacharias, Ph.D. Chief of Cataloging Division Astrometry Department US Naval Observatory voice: 202 762 1423 3450 Mass. Ave. NW fax: 202 762 1516 Washington, DC 20392 e-mail: nz@usno.navy.mil ------------------------------------------------ ************************************************************************ From: "Dick Shaw" To: fitsbits@donar.cv.nrao.edu Date: Thu, 18 Sep 2008 14:15:19 -0700 Subject: [fitsbits] Comments on image distortion conventions Regarding the proposed Simple Imaging Polynomial (SIP) convention for representing non-linear image distortions: 1. The document should specify default values for any coefficients that are absent from the header, but might be expected based upon the value of A_ORDER or B_ORDER. Presumably either the values would be taken to be zero or the convention should require that all keywords be present. Similarly, are A_ORDER and B_ORDER restricted to be non-negative? 2. It is unfortunate that the transformation goes directly from raw array pixels to world coordinates, without defining an intermediate pixel coordinate system. This approach carries some liabilities, one of which follows: 3. The convention would appear to mis-use the CTYPEi reserved keyword in a way that at least technically violates the FITS standard. Quoting from the definition of this term in Sect. 8.2 of the FITS V3.0 Standard: "All non-linear coordinate system names _must_ be expressed in "4-3" form: the first four characters specify the coordinate type, the fifth character is a hyphen, and the remaning three characters specify an algorithm code for computing the world coordinate value...." -Dick Shaw ************************************************************************ Date: Wed, 24 Sep 2008 13:27:21 -0400 From: Doug Mink To: fitsbits@donar.cv.nrao.edu Subject: Re: [fitsbits] Comments on image distortion conventions Dick Shaw wrote: > Regarding the proposed Simple Imaging Polynomial (SIP) convention for > representing non-linear image distortions: > > 1. The document should specify default values for any coefficients that are > absent from the header, but might be expected based upon the value of A_ORDER > or B_ORDER. Presumably either the values would be taken to be zero or the > convention should require that all keywords be present. Similarly, are A_ORDER > and B_ORDER restricted to be non-negative? This should all be included in the document. > 2. It is unfortunate that the transformation goes directly from raw array > pixels to world coordinates, without defining an intermediate pixel coordinate > system. This approach carries some liabilities, one of which follows: > > 3. The convention would appear to mis-use the CTYPEi reserved keyword in a way > that at least technically violates the FITS standard. Quoting from the > definition of this term in Sect. 8.2 of the FITS V3.0 Standard: "All > non-linear coordinate system names _must_ be expressed in "4-3" form: the > first four characters specify the coordinate type, the fifth character is a > hyphen, and the remaining three characters specify an algorithm code for > computing the world coordinate value...." First I should say that my WCSTools package can read both SIP and TNX distortion parameters. The SIP implementation is based on code from IPAC and the TNX implementation is based on my translation of IRAF SPP code into C. The Harvard DASCH plate digitization project is using TNX with great success for very wide field plates taken with small telescopes (though they have found a few bugs in my subroutines). SIP's 4-3-3 usage is an artifact of a time in the development of the FITS WCS distortion standard when the 3-character suffix was proposed as a standard. I think that there is an advantage in following the plate solution tradition of including distortion in the pixel <-> world coordinate transformation directly. If the appropriate keywords are provided so that a reasonable WCS is obtained using the standard keywords and the first 8 characters of the CTYPEi, there shouldn't be too much of a problem. TNX adds a melange of nested multi-line/multi-value keywords to FITS, but it works and has been used for lots of images. There is also a ZPX variation on ZPN, but that seems not to have been used as often. -Doug Mink ************************************************************************ Date: Tue, 30 Sep 2008 14:26:41 -0400 From: "David W Hogg" To: fitsbits@donar.cv.nrao.edu Subject: [fitsbits] Start of the Public Comment Period on 2 Image Distortion Conventions (If you respond to this, please CC me, because I am *not* on FITSBITS.) Here are a few cents of mine; I will ask Dustin Lang to follow up as well. I warn you that I am writing this *without* having read the documents, but I figure if I wait until I read the documents, I will never get my comments in in time. ----- We have been using SIP at http://astrometry.net/ for a few years now, and have working C and Python code that manipulates, creates, writes, and reads SIP. Overall, we like SIP, for the main reason: (0) SIP is very clear and simple, to read, implement, adjust, and analyze. We adopted SIP over TNX at the time for two reasons: (1) SIP was in image coordinates, TNX was in sky coordinates (at the time, this may have changed), and we expect *most* (not all) fixed distortions to be fixed in camera and not sky coordinates. (2) SIP made use of the usual FITS keywords, and did not require a string keyword manipulator to extract distortion coefficients, as TNX did at the start (don't know if it does still). My main problems or issues with SIP are as follows: (3) The inverse transformation is just a separate transformation and there is no restriction that the inverse *actually be* the inverse or any approximation to it. Indeed, it can't be an exact inverse. However, computers are smart and fast, so the inverse *could* have been encoded as the Newton's method inverse of the forward, and only one transformation would be stored in the header at all! I consider this an issue for a number of reasons: (3.1) is it wrong to invert the transformation by iterating the forward transformation? If so, how do we prevent that? (3.2) there are many ways to approximate the reverse transformation with a polynomial, and different investigators will make different choices (least squares on the calibration stars? least squares on arbitrary control points? use errors how?). This is not standardized, I suspect, in the standard. If it *were* standardized, would we force SIP readers to check that? (4) The restriction to 9th order may not seem restrictive, but if you know your camera has repeatable radial distortions, and you have millions of images, then you can easily fit radial terms beyond 9th order. Up to 9, these can be encoded as a SIP with many terms set to exactly zero. After 9, no dice. (5) The standard does not prevent an investigator from inputting a distortion map that is multi-valued, folded, and non-invertible, in forward or in reverse. In fact, once you get to high enough order, it is very hard to write a SIP header that does *not* have these problems somewhere in the "focal plane". There is one over-arching problem with all of these standards, which perhaps deserves addressing: (6) If you read the definitional WCS papers, the authors were clearly thinking of WCS to describe the mapping of a processed, output map. They were not thinking of using WCS to describe obtained, natural images. They were imagining that the investigator deals with all the natural images in some idiosyncratic way, produces a distortion-free map, and specifies the properties of that map using the WCS convention. That is *not* the way WCS is used in most applications, and leads to some of the compromises encountered in TNX and SIP. Finally, one last irrelevant comment: (7) One extreme WCS standard that we *could* put forward, and which *would* work in all circumstances is the following: (7.1) A list of ra, dec values, and a matched list of x,y values. Interpolate as you wish! (7.2) This standard would allow distortions of any kind, automatically includes the forwards and reverse transformations, does not need to piggy-back on the existing WCS TAN (or any other) undistorted standard, permits distortions of any order or character, permits the investigator to make the distortion finer in some parts of the image than others, etc. etc. It is not fully specified, of course, but a full specification would not be hard to write. (7.3) Implemented standards are better than non-implemented standards, so for now we still prefer SIP. David -- David W. Hogg - associate professor, NYU - http://cosmo.nyu.edu/hogg/ ************************************************************************ From: "LC's No-Spam Newsreading account" Date: Thu, 2 Oct 2008 12:05:18 +0200 To: fitsbits@donar.cv.nrao.edu Subject: [fitsbits] Comments on SIP distortion FITS convention I have looked at the submitted document. I had also a quick glance to WCS Paper IV (by the way, why is it still in draft status since such a long time ?) The document on the SIP convention fulfills the requirements for inclusion in the registry (the document is clear enough and the convention has been in real usage and is already supported by commonly used software). Although I'm not an expert at all on distortions, my impression is that the SIP convention is definitely simpler, while WCS Paper IV is more general. So my concern would be about the coexistence of two partially overlapping conventions ... of course this depends on whether WCS Paper IV progresses toward publication (and if it does, it is likely it will become part of the standard as the other WCS papers). What are the plans? Could it incorporate the SIP convention in some compatibility mode ?