Effective Date?

We have a large number of coadded frames to archive.
Many of these are coadds of dithered Mosaic camera sets.
Some are coadds of images from diverse epochs separated
by days, weeks or months. For the dither sequences, it makes
sense to simply construct a DATE-OBS from the weighted
average of the individual dates (and times). For the coadds
covering much longer timescales, this is less desirable.

Is there a de facto FITS standard for a keyword (something
like DATE-EFF) which reflects a date and time of mid-epoch?

Comments about the desirability of such?

Rob Seaman
NOAO Science Archive

Rob Seaman wrote:

We have a large number of coadded frames to archive.
Many of these are coadds of dithered Mosaic camera sets.
Some are coadds of images from diverse epochs separated
by days, weeks or months. For the dither sequences, it makes
sense to simply construct a DATE-OBS from the weighted
average of the individual dates (and times). For the coadds
covering much longer timescales, this is less desirable.

Is there a de facto FITS standard for a keyword (something
like DATE-EFF) which reflects a date and time of mid-epoch?

Comments about the desirability of such?

Rob Seaman
NOAO Science Archive


I would say that an effective date would be meaningless; consider
a supernova that was peaking during the first frame, it might show
up in the final mosaic but not be related to DATE-EFF. The only
sure way would be to include a table of all the components; but
in the case of normal single-exposure frames, in many observing
systems DATE-OBS refers to the moment the shutter opens, and
hence in a composite could logically be set to the DATE-OBS
of the first component frame. With a careful explanatory comment...

Pete.

Peter Bunclark writes:

I would say that an effective date would be meaningless; consider
a supernova that was peaking during the first frame, it might show
up in the final mosaic but not be related to DATE-EFF.

I think "meaningless" is a bit strong. Many surveys and targeted
projects (at least in the optical/IR) produced data products - images,
spectra or catalogs - that are composited in some fashion from multiple
separate observations. Astronomers have traditionally found these to
be useful for many purposes, and some of these purposes have retained
some notion of an epoch and of time variability.

Let me rephrase the question. Is there a de jure or de facto FITS
standard for conveying a notion of an epoch? A few years back we
deprecated the EPOCH keyword in favor of EQUINOX, since that was the
information actually being conveyed. Where did that leave us in regards
to conveying information about the actual epoch of an observation (or
set of observations)?

For instance, one could imagine a careful project wanting to address
Peter's concern. They might place a single scalar representation of
the "average" epoch into a FITS image header. Excursions from that
average might be conveyed in an attached FITS extension as a mask
represented as some binary table format. Where does that leave us?
In the same relationship to temporal accuracy as a wideband filter
attempting to represent field stars of many differing colors. It is
true that the effective epoch of observation would still vary between
all the variable objects in the field - but this is true of *any* finite
length observation.

The only sure way would be to include a table of all the components;
but in the case of normal single-exposure frames, in many observing
systems DATE-OBS refers to the moment the shutter opens, and hence
in a composite could logically be set to the DATE-OBS of the first
component frame. With a careful explanatory comment...

And this initial shutter open DATE-OBS is what we produce for dither
sequences of NOAO Mosaic images, for instance. A complication here is
that the DATE-OBS plus the EXPTIME no longer accurately conveys
information about the date/time of the shutter closing. (Assuming the
instrument in question has a shutter :-) This is an issue, for example,
for generating an effective airmass or time of mid-UT for an exposure.

Perhaps an appropriate compromise between utility and precision would
be to convey both the shutter open time and the shutter close time,
something like OBSSTART and OBSSTOP? I'm aware of certain instruments
that do precisely this, of course. In the case of a random coadded
sequence of images, the fact that OBSSTOP - OBSSTART EXPTIME would
be sufficient indication that care in interpretation was warranted.

The larger issue is that as we step forward into the glorious VO future,
the pressure on FITS to represent coherent astronomical semantics will
*increase*, not decrease. We can't simply rely on VOTables and UCDs
and such to save the day.

A question to ponder. Given a general astronomical data product -
raw or reduced or calibration - image, spectra, catalog or "etc." - what
is the minimal set of coherent scientific metadata required to interpret
the context of that data product? How best might that minimal scientific
metadata be represented in FITS data structures for each type of data
product?

Rob Seaman
NOAO Science Data Systems

Rob Seaman wrote:
[...] Many surveys and targeted
projects (at least in the optical/IR) produced data products - images,
spectra or catalogs - that are composited in some fashion from multiple
separate observations. Astronomers have traditionally found these to
be useful for many purposes, and some of these purposes have retained
some notion of an epoch and of time variability.
[...]

I'm not sure about some of the larger issues here, but I've had a
need to know the observation ending time, so for years I've had
a 'UTEND' keyword for that.


--
Bill Wyatt ) "remove this" for email
Smithsonian Astrophysical Observatory (Cambridge, MA, USA)

Rob Seaman wrote:
[...] Many surveys and targeted
projects (at least in the optical/IR) produced data products - images,
spectra or catalogs - that are composited in some fashion from multiple
separate observations. Astronomers have traditionally found these to
be useful for many purposes, and some of these purposes have retained
some notion of an epoch and of time variability.
[...]

I'm not sure about some of the larger issues here, but I've had a
need to know the observation ending time, so for years I've had
a 'UTEND' keyword for that.


--
Bill Wyatt ) "remove this" for email
Smithsonian Astrophysical Observatory (Cambridge, MA, USA)

Peter Bunclark writes:

I would say that an effective date would be meaningless; consider
a supernova that was peaking during the first frame, it might show
up in the final mosaic but not be related to DATE-EFF.

I think "meaningless" is a bit strong. Many surveys and targeted
projects (at least in the optical/IR) produced data products - images,
spectra or catalogs - that are composited in some fashion from multiple
separate observations. Astronomers have traditionally found these to
be useful for many purposes, and some of these purposes have retained
some notion of an epoch and of time variability.

Let me rephrase the question. Is there a de jure or de facto FITS
standard for conveying a notion of an epoch? A few years back we
deprecated the EPOCH keyword in favor of EQUINOX, since that was the
information actually being conveyed. Where did that leave us in regards
to conveying information about the actual epoch of an observation (or
set of observations)?

For instance, one could imagine a careful project wanting to address
Peter's concern. They might place a single scalar representation of
the "average" epoch into a FITS image header. Excursions from that
average might be conveyed in an attached FITS extension as a mask
represented as some binary table format. Where does that leave us?
In the same relationship to temporal accuracy as a wideband filter
attempting to represent field stars of many differing colors. It is
true that the effective epoch of observation would still vary between
all the variable objects in the field - but this is true of *any* finite
length observation.

The only sure way would be to include a table of all the components;
but in the case of normal single-exposure frames, in many observing
systems DATE-OBS refers to the moment the shutter opens, and hence
in a composite could logically be set to the DATE-OBS of the first
component frame. With a careful explanatory comment...

And this initial shutter open DATE-OBS is what we produce for dither
sequences of NOAO Mosaic images, for instance. A complication here is
that the DATE-OBS plus the EXPTIME no longer accurately conveys
information about the date/time of the shutter closing. (Assuming the
instrument in question has a shutter :-) This is an issue, for example,
for generating an effective airmass or time of mid-UT for an exposure.

Perhaps an appropriate compromise between utility and precision would
be to convey both the shutter open time and the shutter close time,
something like OBSSTART and OBSSTOP? I'm aware of certain instruments
that do precisely this, of course. In the case of a random coadded
sequence of images, the fact that OBSSTOP - OBSSTART EXPTIME would
be sufficient indication that care in interpretation was warranted.

The larger issue is that as we step forward into the glorious VO future,
the pressure on FITS to represent coherent astronomical semantics will
*increase*, not decrease. We can't simply rely on VOTables and UCDs
and such to save the day.

A question to ponder. Given a general astronomical data product -
raw or reduced or calibration - image, spectra, catalog or "etc." - what
is the minimal set of coherent scientific metadata required to interpret
the context of that data product? How best might that minimal scientific
metadata be represented in FITS data structures for each type of data
product?

Rob Seaman
NOAO Science Data Systems

Rob Seaman wrote:

We have a large number of coadded frames to archive.
Many of these are coadds of dithered Mosaic camera sets.
Some are coadds of images from diverse epochs separated
by days, weeks or months. For the dither sequences, it makes
sense to simply construct a DATE-OBS from the weighted
average of the individual dates (and times). For the coadds
covering much longer timescales, this is less desirable.

Is there a de facto FITS standard for a keyword (something
like DATE-EFF) which reflects a date and time of mid-epoch?

Comments about the desirability of such?

Rob Seaman
NOAO Science Archive


I would say that an effective date would be meaningless; consider
a supernova that was peaking during the first frame, it might show
up in the final mosaic but not be related to DATE-EFF. The only
sure way would be to include a table of all the components; but
in the case of normal single-exposure frames, in many observing
systems DATE-OBS refers to the moment the shutter opens, and
hence in a composite could logically be set to the DATE-OBS
of the first component frame. With a careful explanatory comment...

Pete.