|
|
Thread Tools | Display Modes |
|
#1
|
|||
|
|||
[fitsbits] BLANK keyword misinterpretation
We are currently testing of the new CCD mosaic which is being readied
for shipment to Keck to replace the old detector in the HIRES spectrograph. The laboratory setup permits illumination patterns which are hard to duplicate in the instrument after deployment. In the course of examining the images we discovered that I had made a misinterpretation of the FITS standard. I believe that the FITS standard is unambiguous. Nevertheless, slight changes to the wording of the standard could be very useful in preventing other such misinterpretations. Until it becomes possible to make such changes other implementors should be cautious when using the BLANK keyword. It is also possible that changes to the APIs that are used for writing FITS images would help. Our data originate via a 16-bit AtoD conversion which produces bits corresponding to C "unsigned short" on modern big-endian processors. The pixel values are always offset such that all values are positive. We store the bits directly in the image arrays as unsigned short integers using the traditional trick of offsetting the physical data values using the keywords BITPIX = 16 BZERO = 32768 BSCALE = 1 The complete HDU structure of the mosaic FITS files is created at the beginning of the CCD readout, and at this time the values in the data arrays of all IMAGE extensions are initialized to be all zero bits. During the readout the FITS file is stored in shared memory where it can be accessed by FITS viewer to provide real-time display of the incoming image. In our systems the pixels may not arrive in sequence, so the images being constructed may have "holes" during the readout. The "holes" in the image during the readout confound the automatic contrast setting algorithm in the FITS viewer. However, the FITS viewer that we use understands the usage of the BLANK keyword. In order to instruct it ignore pixels that do not yet have data I modified the FITS headers to include the keyword BLANK = 0 Whereas this had the intended effect during real-time readout, this was a mistake for the archival data. FITS is defined by NOST 100-2.0 as printed in http://adsabs.harvard.edu/cgi-bin/np...6A...376..359H The interpretation of BITPIX is shown by Table 5.2 in section 5.4.1.1. I believe it would be helpful to many implementors if the text where slightly enhanced such that the entries 16-bit twos-complement binary integer 32-bit twos-complement binary integer were changed to read 16-bit twos-complement (signed) binary integer 32-bit twos-complement (signed) binary integer The definition of the data representation for BITPIX = 16 is given by section 6.2.2. It is completely unambiguous. The definition of the BLANK keyword is given in section 5.4.2.5. Its relevant sentence reads: The value field shall contain an integer that specifies the representation of array values whose physical values are undefined. This is unambiguous, but I misinterpreted it. I would have got the interpretation right if there had been one extra sentence akin to this: This number corresponds to the data array values before applying any transformation indicated by BSCALE and BZERO. The problem is that with these values of BZERO and BSCALE the signed FITS array value of 0 corresponds to the unsigned CCD data value of 32768. The result of my misinterpretation is that images with a gentle ramp of CCD data values in the vicinity of 32768 have sprinkles of single pixels where a conforming FITS viewer flags the pixels as NULL or undefined. In our software the fix for this is relatively straightforward. The image in shared memory during readout is currently used only for real-time display. There is a final pass over each HDU where keyword are modified before the image is written to disk for archiving. In that pass the software will reset the keywords such that BLANK = -32768 This change will have no significant impact on real-time performance. We are using CFITSIO as the API for creating these FITS images. This problem could also be alleviated if the API provided an explicitly defined means of indicating BLANK along with other parameters of the image. -- Steve Allen UCO/Lick Observatory Santa Cruz, CA 95064 Voice: +1 831 459 3046 http://www.ucolick.org/~sla PGP: 1024/E46978C5 F6 78 D1 10 62 94 8F 2E 49 89 0E FE 26 B4 14 93 |
#2
|
|||
|
|||
[fitsbits] BLANK keyword misinterpretation
On Fri 2003-11-21T09:09:29 -0500, Arnold Rots hath writ:
Shouldn't you also flip the sign bit? That's part of the traditional trick. Yes. Without that the data value in every pixel would be wrong. With the BLANK value wrong it was only a few in 10000, and randomly scattered, in most non-lab images. -- Steve Allen UCO/Lick Observatory Santa Cruz, CA 95064 Voice: +1 831 459 3046 http://www.ucolick.org/~sla PGP: 1024/E46978C5 F6 78 D1 10 62 94 8F 2E 49 89 0E FE 26 B4 14 93 |
#3
|
|||
|
|||
[fitsbits] BLANK keyword misinterpretation
Shouldn't you also flip the sign bit?
That's part of the traditional trick. - Arnold Steve Allen wrote: ... Our data originate via a 16-bit AtoD conversion which produces bits corresponding to C "unsigned short" on modern big-endian processors. The pixel values are always offset such that all values are positive. We store the bits directly in the image arrays as unsigned short integers using the traditional trick of offsetting the physical data values using the keywords BITPIX = 16 BZERO = 32768 BSCALE = 1 ... -- Steve Allen UCO/Lick Observatory Santa Cruz, CA 95064 Voice: +1 831 459 3046 http://www.ucolick.org/~sla PGP: 1024/E46978C5 F6 78 D1 10 62 94 8F 2E 49 89 0E FE 26 B4 14 93 _______________________________________________ fitsbits mailing list http://listmgr.cv.nrao.edu/mailman/listinfo/fitsbits -------------------------------------------------------------------------- Arnold H. Rots Chandra X-ray Science Center Smithsonian Astrophysical Observatory tel: +1 617 496 7701 60 Garden Street, MS 67 fax: +1 617 495 7356 Cambridge, MA 02138 USA http://hea-www.harvard.edu/~arots/ -------------------------------------------------------------------------- |
#4
|
|||
|
|||
[fitsbits] BLANK keyword misinterpretation
On Fri 2003-11-21T09:09:29 -0500, Arnold Rots hath writ:
Shouldn't you also flip the sign bit? That's part of the traditional trick. Yes. Without that the data value in every pixel would be wrong. With the BLANK value wrong it was only a few in 10000, and randomly scattered, in most non-lab images. -- Steve Allen UCO/Lick Observatory Santa Cruz, CA 95064 Voice: +1 831 459 3046 http://www.ucolick.org/~sla PGP: 1024/E46978C5 F6 78 D1 10 62 94 8F 2E 49 89 0E FE 26 B4 14 93 |
#5
|
|||
|
|||
[fitsbits] BLANK keyword misinterpretation
Shouldn't you also flip the sign bit?
That's part of the traditional trick. - Arnold Steve Allen wrote: ... Our data originate via a 16-bit AtoD conversion which produces bits corresponding to C "unsigned short" on modern big-endian processors. The pixel values are always offset such that all values are positive. We store the bits directly in the image arrays as unsigned short integers using the traditional trick of offsetting the physical data values using the keywords BITPIX = 16 BZERO = 32768 BSCALE = 1 ... -- Steve Allen UCO/Lick Observatory Santa Cruz, CA 95064 Voice: +1 831 459 3046 http://www.ucolick.org/~sla PGP: 1024/E46978C5 F6 78 D1 10 62 94 8F 2E 49 89 0E FE 26 B4 14 93 _______________________________________________ fitsbits mailing list http://listmgr.cv.nrao.edu/mailman/listinfo/fitsbits -------------------------------------------------------------------------- Arnold H. Rots Chandra X-ray Science Center Smithsonian Astrophysical Observatory tel: +1 617 496 7701 60 Garden Street, MS 67 fax: +1 617 495 7356 Cambridge, MA 02138 USA http://hea-www.harvard.edu/~arots/ -------------------------------------------------------------------------- |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
S&T Bulletins .... the REAL numbers | Anthony Ayiomamitis | Amateur Astronomy | 9 | January 3rd 04 09:22 PM |
Electric Gravity&Instantaneous Light | ralph sansbury | Astronomy Misc | 8 | August 31st 03 02:53 AM |