A Space & astronomy forum. SpaceBanter.com

Go Back   Home » SpaceBanter.com forum » Astronomy and Astrophysics » FITS
Site Map Home Authors List Search Today's Posts Mark Forums Read Web Partners

[fitsbits] Start of Public Comment Period on FITS Binary TableProposals



 
 
Thread Tools Display Modes
  #1  
Old October 14th 04, 05:56 PM
William Pence
external usenet poster
 
Posts: n/a
Default [fitsbits] Start of Public Comment Period on FITS Binary TableProposals

ANNOUNCEMENT: START OF FORMAL PUBLIC COMMENT PERIOD


This is to announce the official start of the 4-week formal Public Comment
Period on 2 proposals concerning conventions used within FITS binary tables.
These conventions are currently defined in the unofficial Appendix B of the
FITS Standard document. The purpose of the current 2 proposals is to
officially approve these conventions and to incorporate them into the FITS
Standard.

The 2 proposals are attached to this message below for your review and
comment. The full text of Appendix B of the NOST FITS Standard (on page 57
of the printed document) may be viewed in HTML, PDF, or Postscript formats at:

http://archive.stsci.edu/fits/fits_standard/
ftp://legacy.gsfc.nasa.gov/fits_info...s_standard.pdf
ftp://legacy.gsfc.nasa.gov/fits_info...standard.ps.gz

Note that only sections B.1 and B.2 of the appendix are covered by these
proposals. Appendix B.3 is not being considered for approval at this time.

Please review these proposals carefully and post any comments, criticisms,
or suggestions here on the sci.astro.fits newsgroup, or on the related
FITSBITS moderated mail list. (You can subscribe to FITSBITS at
http://listmgr.cv.nrao.edu/mailman/listinfo/fitsbits).

Under the "Rules and Procedures" of the IAU FITS Working Group, the first
step in the official approval process of any FITS proposal is a formal
Public Comment Period to take place on the sci.astro.fits newsgroup
and the moderated FITSBITS mail list. At the end of this public comment
period the regional FITS committees (North America, Europe, Japan, and
Australia/New Zealand) will then vote on the proposal. Following that the
IAU FITS Working Group will then conduct a final vote to approve or
disapprove the proposal. These rules and procedures are more fully
described at http://fits.gsfc.nasa.gov/iaufwg/iaufwg_rules.html.

William Pence
Chairman, IAU FITS Working Group

--------------------------------------------------------------------

Proposal 1:

Recommendation to Incorporate the "Variable Length Array"
Facility in Binary Tables into the FITS Standard


The "Variable Length Array" FITS data format that is described in Appendix
B.1 of the FITS Standard is now used by a number of major data analysis
packages. Examples of current usage include the "Redistribution Matrix
Files" (RMF), widely used in X-ray astronomy to store calibration
information, and the pixel mask files used in the IRAF system. Many of
the general software libraries for reading and writing FITS files that
are documented on the FITS Support Office web site currently support
this variable length array format in FITS binary tables.

Appendix B.1 is not an official part of the FITS Standard and is included in
that document for informational purposes only. Because the implementations
described above have now fully demonstrated the usefulness and importance
of the Variable Length Array Facility, it is recommended that Appendix B.1
be officially approved and incorporated into the FITS Standard in its
entirety, with the following proviso:

"Because some software systems may not support FITS tables that use the
variable length array format, the producers of FITS data products should
consider the capabilities of the likely recipients of their files when
deciding whether or not to use this format, and as a general rule should use
it only in cases where it provides significant advantages over the simpler
fixed length array format."


-------------------------------------------------------------------------

Proposal 2:

Recommendation to Incorporate the "Multidimensional Array"
Convention in Binary Tables into the FITS Standard

Appendix B.2 in the FITS Standard describes a "Multidimensional array"
convention that uses the TDIMn keyword to define the dimensionality of
arrays that are contained in binary table columns. Many FITS files now use
this convention and contain the TDIMn keyword. Appendix B.2 is currently
not an official part of the FITS Standard and is included in that document
for informational purposes only. The widespread usage of the TDIMn keyword
demonstrates the usefulness and importance of this convention, and thus it
is recommended that the "Multidimensional array" convention described in
Appendix B.2 be officially approved and incorporated into the FITS Standard
with the following 2 modifications:

1. When the IAU FITS Working Group voted to approve the NOST FITS Standard
in October 2000, it did so with the following modification to the text in
Appendix B.2:

It is noted that the following sentence in B.2, "The size implied
by the TDIMn keyword will equal the element count specified in the
TFORMn keyword." is not valid in the case of variable length array
columns. This sentence should be replaced with wording similar to
the following: "The total number of elements in the array equals
the product of the dimensions specified in the TDIMn keyword.
This size must be equal to the repeat count on the TFORMn keyword,
or, in the case of columns which have a "P'' TFORMn datatype,
equal to the array length specified in the variable length array
descriptor (see Appendix B.1). In the special case where the
variable length array descriptor has a size of zero, then the
TDIMn keyword is not applicable.

2. Delete the last 2 sentences of the 3rd paragraph that mention other
conventions for storing variable-length character strings and
one-dimensional arrays of fixed-length strings.
  #2  
Old October 18th 04, 02:11 PM
Tom McGlynn
external usenet poster
 
Posts: n/a
Default

While I'm in favor of both of these proposals, it seems to me
that they need a formal document giving the precise words
which are to be added to the FITS standard. Indicating that
these are currently appendices B.1 and B.2 is not sufficient.
These have phrases like: "This facility is still undergoing
trials ..." and similar caveats that need to be deleted. The
first few words of B.2 are fine for a proposal for some
new feature, but seem quaint in light of the fact that FITS files
have been using this feature for years (decades?).

B.2 has the phrase "This convention is optional." which is very
unclear to me. If this phrase were to be retained would it mean
that someone could use TDIMnn = '(2,3,4)' to mean something other
than what B.2 currently specifies.

Ideally this document would indicate precisely the new words
in the standard and where these
new conventions are to be inserted into the existing standard.

Finally and more substantively, Bill and I have discussed the
limitation of the variable length records convention to 4-byte integer
sizes and offsets. This may begin to chafe fairly soon. While 2 GB will
usually be OK for the sizes for individual cells of a variable length
column, it is already very easy to build tables where the offsets
need to be more than 2 GB. While we may not want to include
8 byte pointers in the current proposal it might be useful
to have some discussion of this now.

Regards,
Tom McGlynn

William Pence wrote:

ANNOUNCEMENT: START OF FORMAL PUBLIC COMMENT PERIOD


  #3  
Old October 19th 04, 02:31 PM
external usenet poster
 
Posts: n/a
Default

column, it is already very easy to build tables where the offsets
need to be more than 2 GB. While we may not want to include
8 byte pointers in the current proposal it might be useful
to have some discussion of this now.


We (Arecibo Observatory) routinely write tables that are really much
larger than 2GB. There are more such limits than just fits, so we chop
them into multiple files.

Since there are already have many integer types, it's hard to see the
problem with going ahead with 8-byte integers. Any software that can't
deal with them can't deal with the large table either, and getting the
convention set lets people adapt to it sooner, before fits becomes the
limiting factor.

-Mike

---------------------------
Mike Nolan
+1 787 878 2612x334 Fax: +1 787 878 1861
Arecibo Observatory / Cornell University
HC03 Box 53995, Arecibo, PR 00612 USA
 




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[fitsbits] Start of the FITS MIME type Public Comment Period William Pence FITS 8 June 17th 04 06:08 AM
[fitsbits] problems with fits readers Eric Greisen FITS 0 June 4th 04 08:15 PM
[fitsbits] Happy Birthday, FITS! Don Wells FITS 0 March 28th 04 01:58 PM
Reading floating point FITS files John Green FITS 34 November 29th 03 01:31 AM
[fitsbits] BLANK keyword misinterpretation Steve Allen FITS 4 November 21st 03 05:42 PM


All times are GMT +1. The time now is 03:30 PM.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 SpaceBanter.com.
The comments are property of their posters.