[fitsbits] INHERIT and Hierarchical Grouping

On Tue 2007-04-17T15:25:24 -0400, William Pence hath writ:
the Hierarchical Grouping convention (now also open for public comment)
provides a more general mechanism for specifying the relationship
between multiple HDUs that may be in different files

Does anyone have any comments about the Hierarchical Grouping convention
itself? Are there any deficiencies or limitations in this convention
that have not been considered? Are there alternative ways of
accomplishing the same thing that might be simpler or offer more
features than this convention?

My initial impression was that the Hierarchical Grouping Convention
(HGC) was initially a way of handling interconversions between FITS
files and HDF files. That made it seem like "feature envy", and also
like a solution in search of a problem.

I think that in general the FITS community has not tried hard enough
to acknowledge the cases where interoperability is hindered because
one team chose one way of representing complex data structures. In
most cases everyone else simply uses exactly the same data reduction
system for those sorts of FITS files. The problems are solved because
everything that has to know how somehow just knows how.

Doug Mink's WCSTools code for FITS has all sorts of heuristics about
how it tries to make sense of the zoo of different coordinate
conventions which evolved before the WCS papers were approved and
adopted. Bill Joye's DS9 viewer has a raft of GUI buttons to push so
that the user can select which set of heuristics should be used when
trying to decide how to present the data in a multi-HDU FITS file.

HGC adopted a mentality that no existing HDU would have to be
modified. That means that no single-HDU-minded application has any
way of knowing that a given HDU belongs to a group. In some cases
that's okay because the individual HDUs have significant meaning even
when they stand alone.

But in other cases the individual HDUs are small parts of a normalized
data scheme where the picture only makes sense when all parts are
considered. Those cases more nearly resemble the sorts of activity
that goes on inside relational databases. What HGC does not provide
merely within its descriptive document is a strongly motivating
example of such things or MUST/SHOULD/MAY advisories which would guide
the implementation of some sort of integrity checking mechanism.

My impression is that I would want to have Bill Pence's questions
answered by someone who has a deep knowledge of the sorts of
problems that arise with highly structured data and the sorts of
mechanisms that are used in relational databases.

--
Steve Allen WGS-84 (GPS)
UCO/Lick Observatory Natural Sciences II, Room 165 Lat +36.99845
University of California Voice: +1 831 459 3046 Lng -122.06025
Santa Cruz, CA 95064 http://www.ucolick.org/~sla/ Hgt +250 m