[fitsbits] versioning vs. schema -- where does it stop?

On Thu 2005-06-16T18:32:25 +0200, Francois Ochsenbein hath writ:

There was also some suggestion about having a versioning mechanism in
FITS -- and it would be a great benefit, I think, to know right from the
beginning of a FITS input stream that it has (or may have), far beyond the
beginning of the file, some extensions which may NOT be recognized by an
old software. I feel this modification would be a good opportunity to
include this versioning mechanism in FITS.

Doesn't a versioning mechanism effectively require creating and
approving a list of UCDs for various FITS concepts?

I am left wondering whether it is politically possible in the current
environment to create a versioning mechanism for FITS which does not
extend most of the way toward requiring an XML schema definition.

A full XML schema which could be used as part of toolsets which could
- validate the structure and keyword content of FITS files
- give complete semantic descriptions of the content of FITS files
- transform FITS files to and from other representations
All of this is almost certainly a Good Thing.

If the FITS standardization effort starts down the road toward
versioning, can it stop without effectively merging with the
technologies underlying the VO efforts?

--
Steve Allen WGS-84 (GPS)
UCO/Lick Observatory Natural Sciences II, Room 165 Lat +36.99858
University of California Voice: +1 831 459 3046 Lng -122.06014
Santa Cruz, CA 95064 http://www.ucolick.org/~sla/ Hgt +250 m

There was also some suggestion about having a versioning mechanism in
FITS -- and it would be a great benefit, I think, to know right from the
beginning of a FITS input stream that it has (or may have), far beyond the
beginning of the file, some extensions which may NOT be recognized by an
old software. I feel this modification would be a good opportunity to
include this versioning mechanism in FITS.

Doesn't a versioning mechanism effectively require creating and
approving a list of UCDs for various FITS concepts?

I am left wondering whether it is politically possible in the current
environment to create a versioning mechanism for FITS which does not
extend most of the way toward requiring an XML schema definition.


=== I was in fact thinking on a really light-weight mechanism --
for instance just a recommendation to include a version
number of smething similar in comment part of the SIMPLE
keyword if the FITS file potentially makes use of the 64 bit.

[...]


--Francois
================================================== ==============================
Francois Ochsenbein ------ Observatoire Astronomique de Strasbourg
11, rue de l'Universite F-67000 STRASBOURG Phone: +33-(0)390 24 24 29
Email: (France) Fax: +33-(0)390 24 24 32
================================================== ==============================

Francois Ochsenbein wrote:
There was also some suggestion about having a versioning mechanism in
FITS -- and it would be a great benefit, I think, to know right from the
beginning of a FITS input stream that it has (or may have), far beyond the
beginning of the file, some extensions which may NOT be recognized by an
old software. I feel this modification would be a good opportunity to
include this versioning mechanism in FITS.

Doesn't a versioning mechanism effectively require creating and
approving a list of UCDs for various FITS concepts?

I am left wondering whether it is politically possible in the current
environment to create a versioning mechanism for FITS which does not
extend most of the way toward requiring an XML schema definition.



=== I was in fact thinking on a really light-weight mechanism --
for instance just a recommendation to include a version
number of smething similar in comment part of the SIMPLE
keyword if the FITS file potentially makes use of the 64 bit.

if you want to distinguish it, why not put in a VERSION (or
something of the sort) keyword?
Whoever doesn't use it can ignore it, but hiding information
that may be necessary for processing in the comments doesn't
sound like a good idea to me.

Aloha,

Maren Purves
UKIRT

Francois Ochsenbein writes:
I am left wondering whether it is politically possible in the current
environment to create a versioning mechanism for FITS which does not
extend most of the way toward requiring an XML schema definition.


=== I was in fact thinking on a really light-weight mechanism --
for instance just a recommendation to include a version
number of smething similar in comment part of the SIMPLE
keyword if the FITS file potentially makes use of the 64 bit.

I am afraid that the utility would be limited, with different people
attaching slightly different meanings to the same version number. Or
we'd need a central authority keeping track of what is what, and that
may be more trouble/work that the issue justifies.

On Thu, 16 Jun 2005, Thierry Forveille wrote:
Francois Ochsenbein writes:

=== I was in fact thinking on a really light-weight mechanism --
for instance just a recommendation to include a version

I do appreciate a light-weight mechanism. However if that is *only* a
recommendation, we should define what should be default implied if the
version indicator is missing (lowest version ?)

number of smething similar in comment part of the SIMPLE

However I do not think the comment field of an existing keyword is the
right place. It should be some dedicated keyword, like e.g. the LONGSTRN
keyword used to flag files adhering to the "long string convention".

keyword if the FITS file potentially makes use of the 64 bit.

Also I'm not sure the version flag is really The Necessary Thing To Do
concerning the "documentation of presence" of the (future) 64-bit
proposal/s. It is already documented per se (by the presence of BITPIX
64, TFORM K or Q pointers). A piece of s/w can simply declare them
"unsupported".

More tricky is the issue of more subtle implications on header keyword
(since they are not strongly typed). For instance if usage of 64-bit
quantities requires a NAXISn keyword to be a string longer than the
number of digits in a 32-bit integer ... what will be the implication on
existing readers ?

This is a thing which some header mechanism shall forewarn the reader
(but since is not forbidden by the existing standard nor explicitly
required by the new proposals, probably won't be covered by a versioning
mechanism).

we'd need a central authority keeping track of what is what

Sure.
If we'd do it, how would we do it ?

- consider version 0 (or default) what actually exist in the standard
AT THE PRESENT TIME
- increment version number at each new bunch of changes to the standard
(which would suggest to vote new changes together)

or try to reconstruct past history, e.g.

- version 0 would be the original basic FITS (1981)
- version 1 generalized extensions and ascii tables (1988)
- version 2 image extensions (1994)
- version 3 binary tables (1995)
- version 4 the recent incorporation of conventions (april 2005)
- version 5 the 64-bit stuff

... not sure whether and where all the WCS stuff fits in there

or something intermediate, e.g. 0 will be the 2001 A&A paper which
is comprehensive of 0-3 above, and we increment from there ?

Note that the above does not cope with widespread "conventions" which
are not part of the standard (LONGSTR, HIERARCH etc.)

Should we instead use (somewhere in the primary header to be
encountered soon by readers) a keyword marking a list of "conventions"
or "features" adhered to by the file ? something like

CONVENTN='BINTABLE,64BIT,LONGSTRN,WCSII'

What is the likelihood that readers will actually use it ?

--
----------------------------------------------------------------------
is a newsreading account used by more persons to
avoid unwanted spam. Any mail returning to this address will be rejected.
Users can disclose their e-mail address in the article if they wish so.