Question(s) regarding development of proprietary FITS manipulation software. . .

Thanks Maren. One question I suppose I've just forgotten to ask: Is
there anything particularly special about the actual graphics content
of a FITS file? Or is it perhaps simply a glorified TIFF or JPEG of
some sort?

Regards,
Michael

On Aug 4, 11:09 pm, Maren Purves wrote:
On Fri, 3 Aug 2007, gberz3 wrote:
So far, I've really only looked at the CFITSIO items and it's very
well documented (really too well) but the function names don't seem
very intuitive. I'm sure they mean a lot more to someone familiar
with FITS, but I'm absolutely not. It's basically like someone in
1989 asking you to write a JPEG file manipulation program. Sure the
information exists, but with 0 familiarity you'll need a lot of input.

there used to be a file called longnames.h ...

Aloha,
Maren

Michael,

it's not graphics, it's data.
Well described reducible data.
You can't get much physics out of just images.

Aloha,
Maren

gberz3 wrote:
Thanks Maren. One question I suppose I've just forgotten to ask: Is
there anything particularly special about the actual graphics content
of a FITS file? Or is it perhaps simply a glorified TIFF or JPEG of
some sort?

Regards,
Michael

On Aug 4, 11:09 pm, Maren Purves wrote:
On Fri, 3 Aug 2007, gberz3 wrote:
So far, I've really only looked at the CFITSIO items and it's very
well documented (really too well) but the function names don't seem
very intuitive. I'm sure they mean a lot more to someone familiar
with FITS, but I'm absolutely not. It's basically like someone in
1989 asking you to write a JPEG file manipulation program. Sure the
information exists, but with 0 familiarity you'll need a lot of input.
there used to be a file called longnames.h ...

On Mon 2007-08-20T14:13:19 -0700, gberz3 hath writ:
Thanks Maren. One question I suppose I've just forgotten to ask: Is
there anything particularly special about the actual graphics content
of a FITS file? Or is it perhaps simply a glorified TIFF or JPEG of
some sort?

Have you read RFC 4047?

That might help if you're starting from a modern internet perspective
rather than a 1970s astronomy perspective.

--
Steve Allen WGS-84 (GPS)
UCO/Lick Observatory Natural Sciences II, Room 165 Lat +36.99855
University of California Voice: +1 831 459 3046 Lng -122.06015
Santa Cruz, CA 95064 http://www.ucolick.org/~sla/ Hgt +250 m

Maren,

I understand that. But many of the sample FITS files we've been
given have ASCII data in a header, then literally an accompanying
"picture"; unless, of course, it's simply a matter of the programs
we're using automatically interpreting the "picture". Is this
incorrect? Our concern is how to properly display the "picture" and
what relationship it has to other "picture" formats.

If it's not a picture, then what is it *exactly*? Simple plots of
data? How is it to be interpreted? We've got a TON of data
regarding what FITS is supposed to be, but nothing regarding the
actual interpreting of data.

Any suggestions you have would be greatly appreciated.

Regards,
Michael



On Aug 20, 2007, at 5:49 PM, Maren Purves wrote:

Michael,

it's not graphics, it's data.
Well described reducible data.
You can't get much physics out of just images.

Aloha,
Maren

gberz3 wrote:
Thanks Maren. One question I suppose I've just forgotten to ask: Is
there anything particularly special about the actual graphics content
of a FITS file? Or is it perhaps simply a glorified TIFF or JPEG of
some sort?
Regards,
Michael
On Aug 4, 11:09 pm, Maren Purves wrote:
On Fri, 3 Aug 2007, gberz3 wrote:
So far, I've really only looked at the CFITSIO items and it's very
well documented (really too well) but the function names don't seem
very intuitive. I'm sure they mean a lot more to someone familiar
with FITS, but I'm absolutely not. It's basically like someone in
1989 asking you to write a JPEG file manipulation program. Sure
the
information exists, but with 0 familiarity you'll need a lot of
input.
there used to be a file called longnames.h ...

Michael:

Formats like GIF and JPEG are designed such that the images are scaled into the
8-bit 0-255 range used by computer displays, and are generally accompanied by
color information, such as color tables or separate red-green-blue images. As
such, they are already prepared for direct imaging.

The data in FITS files, however, are generally not preprocessed for viewing.
They are in the units needed for scientific analysis. Such images usually need
some additional processing in order to be easily viewed. The simplest way to
process the image is to rescale it, mapping the minimum value to 0 and the
maximum value to 255. That often works, but usually one has to tweak it to get
a good image. These are some examples of the kind of operations that a general
FITS viewer would need to be able to do:

1. Adjust the mapping from image values to displayed intensity, to zero in on
the most significant range of values.

2. Apply a variety of color tables to the data, and adjust the gamma of those
color tables.

3. Display the logarithm of the data. I've also found it useful to display the
Nth root.

Unfortunately, there's no right answer that covers all situations. It really
depends on the data.

Bill Thompson


Michael Williams wrote:
Maren,

I understand that. But many of the sample FITS files we've been
given have ASCII data in a header, then literally an accompanying
"picture"; unless, of course, it's simply a matter of the programs
we're using automatically interpreting the "picture". Is this
incorrect? Our concern is how to properly display the "picture" and
what relationship it has to other "picture" formats.

If it's not a picture, then what is it *exactly*? Simple plots of
data? How is it to be interpreted? We've got a TON of data
regarding what FITS is supposed to be, but nothing regarding the
actual interpreting of data.

Any suggestions you have would be greatly appreciated.

Regards,
Michael



On Aug 20, 2007, at 5:49 PM, Maren Purves wrote:


Michael,

it's not graphics, it's data.
Well described reducible data.
You can't get much physics out of just images.

Aloha,
Maren

gberz3 wrote:

Thanks Maren. One question I suppose I've just forgotten to ask: Is
there anything particularly special about the actual graphics content
of a FITS file? Or is it perhaps simply a glorified TIFF or JPEG of
some sort?
Regards,
Michael
On Aug 4, 11:09 pm, Maren Purves wrote:

On Fri, 3 Aug 2007, gberz3 wrote:

So far, I've really only looked at the CFITSIO items and it's very
well documented (really too well) but the function names don't seem
very intuitive. I'm sure they mean a lot more to someone familiar
with FITS, but I'm absolutely not. It's basically like someone in
1989 asking you to write a JPEG file manipulation program. Sure
the
information exists, but with 0 familiarity you'll need a lot of
input.

there used to be a file called longnames.h ...


_______________________________________________
fitsbits mailing list

http://listmgr.cv.nrao.edu/mailman/listinfo/fitsbits



--
William Thompson
NASA Goddard Space Flight Center
Code 671
Greenbelt, MD 20771
USA

301-286-2040

Michael,

as you asked me explicitly: I have very little to add to what Bill
(thanks, Bill) wrote.

Maybe a bit more concise: images in GIFs and JPEGs are made to be
looked at by people. You generally can't get much quantitative science
out of them. FITS files are (or at least should be) fully described,
so that when processed correctly you can derive science from them.

(yes, you could measure e.g. lines out of a spectrum from an image,
but it's much easier if you have the underlying numbers, and a lot
of science has been done with Palomar Observatory Sky Survey prints,
but usually by digitizing them).

Aloha,
Maren

William Thompson wrote:
Michael:

Formats like GIF and JPEG are designed such that the images are scaled into the
8-bit 0-255 range used by computer displays, and are generally accompanied by
color information, such as color tables or separate red-green-blue images. As
such, they are already prepared for direct imaging.

The data in FITS files, however, are generally not preprocessed for viewing.
They are in the units needed for scientific analysis. Such images usually need
some additional processing in order to be easily viewed. The simplest way to
process the image is to rescale it, mapping the minimum value to 0 and the
maximum value to 255. That often works, but usually one has to tweak it to get
a good image. These are some examples of the kind of operations that a general
FITS viewer would need to be able to do:

1. Adjust the mapping from image values to displayed intensity, to zero in on
the most significant range of values.

2. Apply a variety of color tables to the data, and adjust the gamma of those
color tables.

3. Display the logarithm of the data. I've also found it useful to display the
Nth root.

Unfortunately, there's no right answer that covers all situations. It really
depends on the data.

Bill Thompson


Michael Williams wrote:
Maren,

I understand that. But many of the sample FITS files we've been
given have ASCII data in a header, then literally an accompanying
"picture"; unless, of course, it's simply a matter of the programs
we're using automatically interpreting the "picture". Is this
incorrect? Our concern is how to properly display the "picture" and
what relationship it has to other "picture" formats.

If it's not a picture, then what is it *exactly*? Simple plots of
data? How is it to be interpreted? We've got a TON of data
regarding what FITS is supposed to be, but nothing regarding the
actual interpreting of data.

Any suggestions you have would be greatly appreciated.

Regards,
Michael

On Aug 20, 2007, at 5:49 PM, Maren Purves wrote:
Michael,

it's not graphics, it's data.
Well described reducible data.
You can't get much physics out of just images.

Alright, I've looked at RFC 4047, and I'm not much better off. I
have about 6 (and I'm sure that's not even the tip of the iceberg)
valid FITS files all formatted completely differently. The only
thing that's consistent among them are the top of the headers. Some
choose their own "keywords" others have a million "COMMENTS". I
suppose, for time's sake, what I need now, are the most common
"formats". Can someone send me (offline or otherwise) some sample
data with accompanying comments/markup regarding what each piece of
data means? A few examples of what I currently have a

FITS compliant header followed by. . .

1) binary data
2) non-compliant header-like text
3) array's of numbers

.. . .and the list goes on. Any help would be greatly appreciated.
And thanks again for your help thus far.


Regards,
Michael


On Aug 21, 2007, at 10:36 AM, William Thompson wrote:

Michael:

Formats like GIF and JPEG are designed such that the images are
scaled into the 8-bit 0-255 range used by computer displays, and
are generally accompanied by color information, such as color
tables or separate red-green-blue images. As such, they are
already prepared for direct imaging.

The data in FITS files, however, are generally not preprocessed for
viewing. They are in the units needed for scientific analysis.
Such images usually need some additional processing in order to be
easily viewed. The simplest way to process the image is to rescale
it, mapping the minimum value to 0 and the maximum value to 255.
That often works, but usually one has to tweak it to get a good
image. These are some examples of the kind of operations that a
general FITS viewer would need to be able to do:

1. Adjust the mapping from image values to displayed intensity, to
zero in on the most significant range of values.

2. Apply a variety of color tables to the data, and adjust the
gamma of those color tables.

3. Display the logarithm of the data. I've also found it useful
to display the Nth root.

Unfortunately, there's no right answer that covers all situations.
It really depends on the data.

Bill Thompson


Michael Williams wrote:
Maren,
I understand that. But many of the sample FITS files we've been
given have ASCII data in a header, then literally an accompanying
"picture"; unless, of course, it's simply a matter of the
programs we're using automatically interpreting the "picture".
Is this incorrect? Our concern is how to properly display the
"picture" and what relationship it has to other "picture" formats.
If it's not a picture, then what is it *exactly*? Simple plots
of data? How is it to be interpreted? We've got a TON of data
regarding what FITS is supposed to be, but nothing regarding the
actual interpreting of data.
Any suggestions you have would be greatly appreciated.
Regards,
Michael
On Aug 20, 2007, at 5:49 PM, Maren Purves wrote:
Michael,

it's not graphics, it's data.
Well described reducible data.
You can't get much physics out of just images.

Aloha,
Maren

gberz3 wrote:

Thanks Maren. One question I suppose I've just forgotten to
ask: Is
there anything particularly special about the actual graphics
content
of a FITS file? Or is it perhaps simply a glorified TIFF or
JPEG of
some sort?
Regards,
Michael
On Aug 4, 11:09 pm, Maren Purves wrote:

On Fri, 3 Aug 2007, gberz3 wrote:

So far, I've really only looked at the CFITSIO items and it's
very
well documented (really too well) but the function names don't
seem
very intuitive. I'm sure they mean a lot more to someone
familiar
with FITS, but I'm absolutely not. It's basically like
someone in
1989 asking you to write a JPEG file manipulation program.
Sure the
information exists, but with 0 familiarity you'll need a lot
of input.

there used to be a file called longnames.h ...
_______________________________________________
fitsbits mailing list

http://listmgr.cv.nrao.edu/mailman/listinfo/fitsbits

--
William Thompson
NASA Goddard Space Flight Center
Code 671
Greenbelt, MD 20771
USA

301-286-2040

On Fri 2007-08-24T14:40:50 -0400, Michael Williams hath writ:
Alright, I've looked at RFC 4047, and I'm not much better off.

Yes.

I am very reticent to write this, but in the absence of a lot
better notion of what exactly is the goal here my reaction is
a lot like the Dilbert strip from 1995-11-17:

PHB: I want to build a relational database.
Dilbert: Ok. What color do you want?
PHB: I think mauve has the most RAM.

FITS guarantees that the data will be accessible to posterity -- syntax.
Aside from the most basic structures about storage of arrays and tables,
FITS guarantees almost nothing about semantics.

--
Steve Allen WGS-84 (GPS)
UCO/Lick Observatory Natural Sciences II, Room 165 Lat +36.99855
University of California Voice: +1 831 459 3046 Lng -122.06015
Santa Cruz, CA 95064 http://www.ucolick.org/~sla/ Hgt +250 m

Ha! Only Dilbert could explain it so well. Alright, now we're
getting somewhere. I suppose I should go back and ask a more
pertinent question. Why in the world are interpretive images even
relevant? Why would *anyone* present FITS data as images if they are
not image data? Why not represent it as sound?

I guess that's what I'm getting at. What relevance does an image have
to actual FITS data if there is no "attached" image, and what is the
proper means by which to display said image?

Michael

On Aug 24, 4:18 pm, Steve Allen wrote:
On Fri 2007-08-24T14:40:50 -0400, Michael Williams hath writ:

Alright, I've looked at RFC 4047, and I'm not much better off.

Yes.

I am very reticent to write this, but in the absence of a lot
better notion of what exactly is the goal here my reaction is
a lot like the Dilbert strip from 1995-11-17:

PHB: I want to build a relational database.
Dilbert: Ok. What color do you want?
PHB: I think mauve has the most RAM.

FITS guarantees that the data will be accessible to posterity -- syntax.
Aside from the most basic structures about storage of arrays and tables,
FITS guarantees almost nothing about semantics.

--
Steve Allen WGS-84 (GPS)
UCO/Lick Observatory Natural Sciences II, Room 165 Lat +36.99855
University of California Voice: +1 831 459 3046 Lng -122.06015
Santa Cruz, CA 95064 http://www.ucolick.org/~sla/ Hgt +250 m

Michael Williams writes:

Why in the world are interpretive images even relevant? Why would
*anyone* present FITS data as images if they are not image data?
Why not represent it as sound?

People have been "imaging" the sky since the camera (and film) were
invented. Galileo drew by hand what he saw through his telescope;
Tycho and many earlier astronomers sketched what they had seen with
their naked eyes. FITS files are often presented as pictures because
people think of the heavens that way. The movie "Contact", however,
offers one example of astronomical data presented, as you suggest, as
sound. The presentation of other data objects that can be
represented with FITS - say spectra, time series, or catalogs - may
indeed require the skills of Edward Tufte more than Eadweard
Muybridge or Edward Elgar.

Astronomical data are rich in implications - the birth of the
universe, the death of the stars, their rebirth in the heavy elements
that make life and this computer possible. The representation of
rich data requires dense display techniques. Astronomers look at
data as pictures because contour maps and line plots can hide the
lurking science.

The sky is also just plain pretty on all scales we've looked to
date. Should presentation graphics only occur as some add-on step
relying on GIFs and JPEGs, not FITS?

I guess that's what I'm getting at. What relevance does an image
have to actual FITS data if there is no "attached" image, and what
is the proper means by which to display said image?

Consider HTML - the content (from the server) is distinct from the
presentation (left to the client). Same with FITS. The image is in
the content, not the format - and, of course, in the eye of the
beholder.

For the proper means, you might download a copy of Ximtool or DS9 to
reverse engineer the common techniques, the representational language
(perhaps start with "window and stretch") and the graphical tools
(histogram vectors are more expressive than scalar gammas, for
instance).

Rob Seaman
NOAO