|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
[fitsbits] Start of Public Comment Period on FITS Binary TableProposals
ANNOUNCEMENT: START OF FORMAL PUBLIC COMMENT PERIOD
This is to announce the official start of the 4-week formal Public Comment Period on 2 proposals concerning conventions used within FITS binary tables. These conventions are currently defined in the unofficial Appendix B of the FITS Standard document. The purpose of the current 2 proposals is to officially approve these conventions and to incorporate them into the FITS Standard. The 2 proposals are attached to this message below for your review and comment. The full text of Appendix B of the NOST FITS Standard (on page 57 of the printed document) may be viewed in HTML, PDF, or Postscript formats at: http://archive.stsci.edu/fits/fits_standard/ ftp://legacy.gsfc.nasa.gov/fits_info...s_standard.pdf ftp://legacy.gsfc.nasa.gov/fits_info...standard.ps.gz Note that only sections B.1 and B.2 of the appendix are covered by these proposals. Appendix B.3 is not being considered for approval at this time. Please review these proposals carefully and post any comments, criticisms, or suggestions here on the sci.astro.fits newsgroup, or on the related FITSBITS moderated mail list. (You can subscribe to FITSBITS at http://listmgr.cv.nrao.edu/mailman/listinfo/fitsbits). Under the "Rules and Procedures" of the IAU FITS Working Group, the first step in the official approval process of any FITS proposal is a formal Public Comment Period to take place on the sci.astro.fits newsgroup and the moderated FITSBITS mail list. At the end of this public comment period the regional FITS committees (North America, Europe, Japan, and Australia/New Zealand) will then vote on the proposal. Following that the IAU FITS Working Group will then conduct a final vote to approve or disapprove the proposal. These rules and procedures are more fully described at http://fits.gsfc.nasa.gov/iaufwg/iaufwg_rules.html. William Pence Chairman, IAU FITS Working Group -------------------------------------------------------------------- Proposal 1: Recommendation to Incorporate the "Variable Length Array" Facility in Binary Tables into the FITS Standard The "Variable Length Array" FITS data format that is described in Appendix B.1 of the FITS Standard is now used by a number of major data analysis packages. Examples of current usage include the "Redistribution Matrix Files" (RMF), widely used in X-ray astronomy to store calibration information, and the pixel mask files used in the IRAF system. Many of the general software libraries for reading and writing FITS files that are documented on the FITS Support Office web site currently support this variable length array format in FITS binary tables. Appendix B.1 is not an official part of the FITS Standard and is included in that document for informational purposes only. Because the implementations described above have now fully demonstrated the usefulness and importance of the Variable Length Array Facility, it is recommended that Appendix B.1 be officially approved and incorporated into the FITS Standard in its entirety, with the following proviso: "Because some software systems may not support FITS tables that use the variable length array format, the producers of FITS data products should consider the capabilities of the likely recipients of their files when deciding whether or not to use this format, and as a general rule should use it only in cases where it provides significant advantages over the simpler fixed length array format." ------------------------------------------------------------------------- Proposal 2: Recommendation to Incorporate the "Multidimensional Array" Convention in Binary Tables into the FITS Standard Appendix B.2 in the FITS Standard describes a "Multidimensional array" convention that uses the TDIMn keyword to define the dimensionality of arrays that are contained in binary table columns. Many FITS files now use this convention and contain the TDIMn keyword. Appendix B.2 is currently not an official part of the FITS Standard and is included in that document for informational purposes only. The widespread usage of the TDIMn keyword demonstrates the usefulness and importance of this convention, and thus it is recommended that the "Multidimensional array" convention described in Appendix B.2 be officially approved and incorporated into the FITS Standard with the following 2 modifications: 1. When the IAU FITS Working Group voted to approve the NOST FITS Standard in October 2000, it did so with the following modification to the text in Appendix B.2: It is noted that the following sentence in B.2, "The size implied by the TDIMn keyword will equal the element count specified in the TFORMn keyword." is not valid in the case of variable length array columns. This sentence should be replaced with wording similar to the following: "The total number of elements in the array equals the product of the dimensions specified in the TDIMn keyword. This size must be equal to the repeat count on the TFORMn keyword, or, in the case of columns which have a "P'' TFORMn datatype, equal to the array length specified in the variable length array descriptor (see Appendix B.1). In the special case where the variable length array descriptor has a size of zero, then the TDIMn keyword is not applicable. 2. Delete the last 2 sentences of the 3rd paragraph that mention other conventions for storing variable-length character strings and one-dimensional arrays of fixed-length strings. |
#2
|
|||
|
|||
While I'm in favor of both of these proposals, it seems to me
that they need a formal document giving the precise words which are to be added to the FITS standard. Indicating that these are currently appendices B.1 and B.2 is not sufficient. These have phrases like: "This facility is still undergoing trials ..." and similar caveats that need to be deleted. The first few words of B.2 are fine for a proposal for some new feature, but seem quaint in light of the fact that FITS files have been using this feature for years (decades?). B.2 has the phrase "This convention is optional." which is very unclear to me. If this phrase were to be retained would it mean that someone could use TDIMnn = '(2,3,4)' to mean something other than what B.2 currently specifies. Ideally this document would indicate precisely the new words in the standard and where these new conventions are to be inserted into the existing standard. Finally and more substantively, Bill and I have discussed the limitation of the variable length records convention to 4-byte integer sizes and offsets. This may begin to chafe fairly soon. While 2 GB will usually be OK for the sizes for individual cells of a variable length column, it is already very easy to build tables where the offsets need to be more than 2 GB. While we may not want to include 8 byte pointers in the current proposal it might be useful to have some discussion of this now. Regards, Tom McGlynn William Pence wrote: ANNOUNCEMENT: START OF FORMAL PUBLIC COMMENT PERIOD |
#3
|
|||
|
|||
column, it is already very easy to build tables where the offsets
need to be more than 2 GB. While we may not want to include 8 byte pointers in the current proposal it might be useful to have some discussion of this now. We (Arecibo Observatory) routinely write tables that are really much larger than 2GB. There are more such limits than just fits, so we chop them into multiple files. Since there are already have many integer types, it's hard to see the problem with going ahead with 8-byte integers. Any software that can't deal with them can't deal with the large table either, and getting the convention set lets people adapt to it sooner, before fits becomes the limiting factor. -Mike --------------------------- Mike Nolan +1 787 878 2612x334 Fax: +1 787 878 1861 Arecibo Observatory / Cornell University HC03 Box 53995, Arecibo, PR 00612 USA |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
[fitsbits] Start of the FITS MIME type Public Comment Period | William Pence | FITS | 8 | June 17th 04 06:08 AM |
[fitsbits] problems with fits readers | Eric Greisen | FITS | 0 | June 4th 04 08:15 PM |
[fitsbits] Happy Birthday, FITS! | Don Wells | FITS | 0 | March 28th 04 01:58 PM |
Reading floating point FITS files | John Green | FITS | 34 | November 29th 03 12:31 AM |
[fitsbits] BLANK keyword misinterpretation | Steve Allen | FITS | 4 | November 21st 03 04:42 PM |