Reading floating point FITS files

On Wed 2003/11/12 23:34:27 -0800, Steve Allen wrote
in a message to: FITSbits

It is not obvious how physical data values would be preserved and
documented when doing this. Most pretty pictures have been clipped,
stretched, mapped, and/or scaled in ways that obliterate physical
meaning in the data values.

Preserve physical data values? This image may help to clarify matters -
http://antwrp.gsfc.nasa.gov/apod/ap020920.html

At best this seems an awful lot like reinventing TIFF in FITS.
There should be strong justifications before doing this.

Can TIFF put a celestial coordinate grid on the above image?

Mark Calabretta
ATNF

On Fri 2003-11-14T13:38:04 +1100, Mark Calabretta hath writ:
Preserve physical data values? This image may help to clarify matters -
http://antwrp.gsfc.nasa.gov/apod/ap020920.html

Yes. I suspect that most original flux data have been eradicated.
In an astronomical image of this sort I can imagine wanting to
compare relative fluxes and change the mapping in ways that
this JPEG does not allow.

At best this seems an awful lot like reinventing TIFF in FITS.
There should be strong justifications before doing this.

Can TIFF put a celestial coordinate grid on the above image?

Yes, if you admit that geographic coordinates have properties
sufficiently similar to celestial coordinates, via this formalism:
http://www.remotesensing.org/geotiff/geotiff.html

There are a plethora of real estate-related needs for Geographical
Information Systems (GIS) which understand GeoTIFF. Of the population
which has viewers which can display images with coordinates, I suspect
that more people have GeoTIFF viewers than FITS WCS viewers.

My impression is that the standardization of FITS WCS Paper II is
going to result in FITS viewers that adopt the image manipulation and
overlaying technologies that have long been in GIS.

Nevertheless I would be happy if the folks who are producing pretty
pictures such as this demonstrate that there is a need for the extra
un-ambiguity permitted by FITS coordinates using the formalisms of WCS
Paper II. Then I'll vote for Mark's notions to go into FITS.

--
Steve Allen UCO/Lick Observatory Santa Cruz, CA 95064
Voice: +1 831 459 3046 http://www.ucolick.org/~sla
PGP: 1024/E46978C5 F6 78 D1 10 62 94 8F 2E 49 89 0E FE 26 B4 14 93

On Fri 2003-11-14T13:38:04 +1100, Mark Calabretta hath writ:
Preserve physical data values? This image may help to clarify matters -
http://antwrp.gsfc.nasa.gov/apod/ap020920.html

Yes. I suspect that most original flux data have been eradicated.
In an astronomical image of this sort I can imagine wanting to
compare relative fluxes and change the mapping in ways that
this JPEG does not allow.

At best this seems an awful lot like reinventing TIFF in FITS.
There should be strong justifications before doing this.

Can TIFF put a celestial coordinate grid on the above image?

Yes, if you admit that geographic coordinates have properties
sufficiently similar to celestial coordinates, via this formalism:
http://www.remotesensing.org/geotiff/geotiff.html

There are a plethora of real estate-related needs for Geographical
Information Systems (GIS) which understand GeoTIFF. Of the population
which has viewers which can display images with coordinates, I suspect
that more people have GeoTIFF viewers than FITS WCS viewers.

My impression is that the standardization of FITS WCS Paper II is
going to result in FITS viewers that adopt the image manipulation and
overlaying technologies that have long been in GIS.

Nevertheless I would be happy if the folks who are producing pretty
pictures such as this demonstrate that there is a need for the extra
un-ambiguity permitted by FITS coordinates using the formalisms of WCS
Paper II. Then I'll vote for Mark's notions to go into FITS.

--
Steve Allen UCO/Lick Observatory Santa Cruz, CA 95064
Voice: +1 831 459 3046 http://www.ucolick.org/~sla
PGP: 1024/E46978C5 F6 78 D1 10 62 94 8F 2E 49 89 0E FE 26 B4 14 93

On Thu 2003/11/13 19:43:11 -0800, Steve Allen wrote
in a message to: FITSbits

Yes. I suspect that most original flux data have been eradicated.
In an astronomical image of this sort I can imagine wanting to
compare relative fluxes and change the mapping in ways that
this JPEG does not allow.

It's not hard to think of with ways to handle this (e.g. using the
PVi_ma keywords on an 'RGB' axis for scaling). However, the
constituent images can always be stored separately, even in the same
FITS file. The aim is simply to store a colour image in a standard
way, something which FITS currently cannot do!

Yes, if you admit that geographic coordinates have properties
sufficiently similar to celestial coordinates, via this formalism:
http://www.remotesensing.org/geotiff/geotiff.html

Even assuming that GeoTIFF can map north up and east to the left
(I don't know), and that it can handle galactic and other celestial
coordinate systems (possibly together in one image), it still doesn't
make much sense to me that as the last step in producing a composite
image, after regridding each input FITS image onto a common coordinate
system, you have to resort to writing it out in GeoTIFF simply because
FITS can't handle colour.

Mark Calabretta
ATNF

On Thu 2003/11/13 19:43:11 -0800, Steve Allen wrote
in a message to: FITSbits

Yes. I suspect that most original flux data have been eradicated.
In an astronomical image of this sort I can imagine wanting to
compare relative fluxes and change the mapping in ways that
this JPEG does not allow.

It's not hard to think of with ways to handle this (e.g. using the
PVi_ma keywords on an 'RGB' axis for scaling). However, the
constituent images can always be stored separately, even in the same
FITS file. The aim is simply to store a colour image in a standard
way, something which FITS currently cannot do!

Yes, if you admit that geographic coordinates have properties
sufficiently similar to celestial coordinates, via this formalism:
http://www.remotesensing.org/geotiff/geotiff.html

Even assuming that GeoTIFF can map north up and east to the left
(I don't know), and that it can handle galactic and other celestial
coordinate systems (possibly together in one image), it still doesn't
make much sense to me that as the last step in producing a composite
image, after regridding each input FITS image onto a common coordinate
system, you have to resort to writing it out in GeoTIFF simply because
FITS can't handle colour.

Mark Calabretta
ATNF

On Fri 2003/11/14 17:32:49 +1100, Mark Calabretta wrote
in a message to: Steve Allen
and copied to: FITSbits

Yes. I suspect that most original flux data have been eradicated.
In an astronomical image of this sort I can imagine wanting to
compare relative fluxes and change the mapping in ways that
this JPEG does not allow.

Well you could always twiddle the R, G, and B balance on your monitor!

It's not hard to think of with ways to handle this (e.g. using the
PVi_ma keywords on an 'RGB' axis for scaling). However, the
constituent images can always be stored separately, even in the same
FITS file. The aim is simply to store a colour image in a standard
way, something which FITS currently cannot do!

Alternatively, store the three images in a data cube with primary WCS
defined by 'WAVE-TAB' (or 'FREQ-TAB', etc.). This WCS would record the
wavelength (or frequency, etc.) of the radio, optical, and X-ray images
(in the Crab Nebula example). Pixels values, (BUNIT, BSCALE, BZERO)
would be stored in any desired units (e.g. flux density or even
magnitudes). Naturally, these units apply to all three planes.

So far we have an "ordinary" data cube. Now add colour via a secondary
WCS; the 'RGB' coordinate type would have six PVi_ma; an offset and
scale (default values 0.0 and 1.0) for each of the three planes (R, G,
and B). Voila!

(Getting metaphysical for a moment, it would appear that this sort of
WCS stretches the envelope in applying to pixel values, not just pixel
coordinates. However, it is essentially the same as the 'COMPLEX' and
'STOKES' "conventional" types, provided that you interpret the 'R' in
'RGB' as meaning "a red colourspace coordinate where the range defined
by PVi_ma maps to 0.0 to 1.0 in a standard colourspace". This is a
general feature of the conventional coordinate types; they differ
fundamentally in qualifying the value of a pixel, rather than simply
locating it in some coordinate space. This relates to the fact that
such axes have to be treated specially, principally because they are
not interpolable.)

Mark Calabretta
ATNF

On Fri 2003/11/14 17:32:49 +1100, Mark Calabretta wrote
in a message to: Steve Allen
and copied to: FITSbits

Yes. I suspect that most original flux data have been eradicated.
In an astronomical image of this sort I can imagine wanting to
compare relative fluxes and change the mapping in ways that
this JPEG does not allow.

Well you could always twiddle the R, G, and B balance on your monitor!

It's not hard to think of with ways to handle this (e.g. using the
PVi_ma keywords on an 'RGB' axis for scaling). However, the
constituent images can always be stored separately, even in the same
FITS file. The aim is simply to store a colour image in a standard
way, something which FITS currently cannot do!

Alternatively, store the three images in a data cube with primary WCS
defined by 'WAVE-TAB' (or 'FREQ-TAB', etc.). This WCS would record the
wavelength (or frequency, etc.) of the radio, optical, and X-ray images
(in the Crab Nebula example). Pixels values, (BUNIT, BSCALE, BZERO)
would be stored in any desired units (e.g. flux density or even
magnitudes). Naturally, these units apply to all three planes.

So far we have an "ordinary" data cube. Now add colour via a secondary
WCS; the 'RGB' coordinate type would have six PVi_ma; an offset and
scale (default values 0.0 and 1.0) for each of the three planes (R, G,
and B). Voila!

(Getting metaphysical for a moment, it would appear that this sort of
WCS stretches the envelope in applying to pixel values, not just pixel
coordinates. However, it is essentially the same as the 'COMPLEX' and
'STOKES' "conventional" types, provided that you interpret the 'R' in
'RGB' as meaning "a red colourspace coordinate where the range defined
by PVi_ma maps to 0.0 to 1.0 in a standard colourspace". This is a
general feature of the conventional coordinate types; they differ
fundamentally in qualifying the value of a pixel, rather than simply
locating it in some coordinate space. This relates to the fact that
such axes have to be treated specially, principally because they are
not interpolable.)

Mark Calabretta
ATNF

Mark, Steve, etal,

Mark Calabretta writes:
.. The aim is simply to store a colour image in a standard way,
something which FITS currently cannot do! ..

The prior discussion in this thread was concerned with the possibility
of defining WCS-related conventions for encoding color images in FITS
primary HDUs and IMAGE extensions. There is another way that we could
support inclusion of color imagery in FITS files: we could agree to
permit XTENSION='JPEG', XTENSION='TIFF', XTENSION='GeoTiff',
XTENSION='PNG', etc. The XTENSION mechanism enables FITS to wrap up
any data object[*] and declare its type. Data types which have
standard MIME codes are sufficiently well defined that they can be
regarded as suitable for long-term archival storage.

-Don Wells
[*] The XTENSION mechanism will even permit us to agree to use
XTENSION='FITS', as Perry Greenfield pointed out two years ago.
--
Donald C. Wells Scientist - GBT-PTCS Project
http://www.cv.nrao.edu/~dwells
National Radio Astronomy Observatory +1-434-296-0277
520 Edgemont Road, Charlottesville, Virginia 22903-2475 USA

Mark, Steve, etal,

Mark Calabretta writes:
.. The aim is simply to store a colour image in a standard way,
something which FITS currently cannot do! ..

The prior discussion in this thread was concerned with the possibility
of defining WCS-related conventions for encoding color images in FITS
primary HDUs and IMAGE extensions. There is another way that we could
support inclusion of color imagery in FITS files: we could agree to
permit XTENSION='JPEG', XTENSION='TIFF', XTENSION='GeoTiff',
XTENSION='PNG', etc. The XTENSION mechanism enables FITS to wrap up
any data object[*] and declare its type. Data types which have
standard MIME codes are sufficiently well defined that they can be
regarded as suitable for long-term archival storage.

-Don Wells
[*] The XTENSION mechanism will even permit us to agree to use
XTENSION='FITS', as Perry Greenfield pointed out two years ago.
--
Donald C. Wells Scientist - GBT-PTCS Project
http://www.cv.nrao.edu/~dwells
National Radio Astronomy Observatory +1-434-296-0277
520 Edgemont Road, Charlottesville, Virginia 22903-2475 USA

On Wed 2003-11-19T16:05:24 +0000, Rob Seaman hath writ:
It is possible to imagine extending FITS in both directions - to
implement color support within FITS or to permit foreign formats to be
accessed from within FITS. Both directions would require software to
be developed - and to be widely adopted - before the extensions would
be useful. Before we put resources into what would be a daunting task,
there should be a motivation stronger than purely a sense of esthetics.

My impression is that the protocols within WCS Papers I and II along
with some notions from Paper III might result in a conventional
multi-HDU representation for multi-wavelength data. With that in
place it is conceiveable that developers of plug-ins for tools such as
Adobe Photoshop (TM) might be motivated to write FITS import filters
which would provide handles for producing pretty pictures.

I see the current discussion as a roadmap which might be followed by
practitioners of multi-wavelength astronomy. Thanks for pointing
out the importance of having a demonstrated need before inventing
a new scheme and burdening every FITS implementor with handling it.

--
Steve Allen UCO/Lick Observatory Santa Cruz, CA 95064
Voice: +1 831 459 3046 http://www.ucolick.org/~sla
PGP: 1024/E46978C5 F6 78 D1 10 62 94 8F 2E 49 89 0E FE 26 B4 14 93