|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
[fitsbits] ADASS FITS BoF Minutes
Minutes of the ADASS FITS Birds-of-a-Feather Session
---------------------------------------------------- The FITS Birds-of-a-Feather (BoF) session, held each year in conjunction with the ADASS conference, took place in Pasadena, California on 24 October 2004. The purpose of the FITS BoF sessions is to encourage members of the FITS user community to discuss current issues related to the FITS data format. About 70 people attended this 90 minute session which was led by the chairman of the IAU FITS Working Group, William Pence. Here are some brief minutes of the main items that were discussed: 1. The meeting opened with a note of congratulations to Don Wells (in absentia) on his recent retirement from NRAO and appreciation for all his contributions to the FITS community over the past 3 decades. Even though Don is officially retired, it is good to know that he will be continuing his involvement in FITS-related activities. 2. Recent changes to the FITS Committees The IAU FITS Working Group (serving under IAU Commission 5) was reorganized at the end of last year and now has 22 members: 11 from North America, 9 from Europe, and 1 each from Japan and Australia. See http://fits.gsfc.nasa.gov/iaufwg/iaufwg.html for the complete list. A new Regional FITS committee was also formed to represent FITS users in Australia and New Zealand. This new committee will be chaired by Mark Calabretta, and it joins the 3 previously existing regional committees for North America, Europe, and Japan which all provide advice and guidance to the the IAU FITS Working Group. 3. New FITS Approval Procedures The rules by which the FITS committees approve new FITS proposals have been clarified and are documented in detail at http://fits.gsfc.nasa.gov/iaufwg/iaufwg_rules.html. The main steps in the review process can be summarized as: 1. A formal Public Comment Period is held on the FITSBITS mail list (and the related sci.astro.fits newsgroup). 2. The regional FITS committees vote on the proposal with a 2/3 majority in each committee required for approval. 3. The IAU FITS Working Group votes with a 3/4 majority required for approval. One of the comments made during the discussion of this topic was that these formal procedures should also include the traditional requirement of 'interoperability testing' of the proposal, at least in cases where it is relevant. 4. Status of the FITS MIME type proposal Steve Allen reported that the proposal for 2 new Internet MIME types ("image/fits" and "application/fits"), which had previously been approved by the FITS committees, is now in the final stages of approval by the relevant Internet standards committee. [Final approval was subsequently granted on 1-Nov-2004]. Congratulations and thanks are due to Steve Allen and Don Wells for drafting and shepherding this proposal through the long approval process! 5. Status of the WCS Paper III on Spectral Coordinate Systems Several of the authors of this paper were in attendance (Eric Greisen, Frank Valdes, and Steve Allen) and led a brief discussion. This paper is now in the Public Comment Period phase of the formal approval process, so everyone was encouraged to carefully review the paper and submit comments on FITSBITS by 12 November. Much of the discussion concerned what, if anything, should be required in the way of interoperability testing before the FITS committees vote on the proposal. It was noted that Mark Calabretta's wcslib software library supports most of the WCS conventions defined in the paper but there is little in the way of other software to enable independent testing. There was no clear consensus on what interoperability testing should be performed, but there was general agreement that a set of FITS files that use the various WCS conventions should be created and made available for testing purposes. Pence agreed to collect any suitable files and make them available from the FITS Support Office web site (fits.gsfc.nasa.gov). 6. The SIP convention for representing image distortions David Shupe and Richard Hook gave a presentation on the "SIP" convention for representing image distortions that is being used in the images produced by the Spitzer Science Center and may possibly be adopted for some HST images. When the SIP convention was developed several years ago it more or less conformed to the recommendations given in an early draft of the WCS paper on representations of distortions in FITS world coordinate systems (the 4th in the series of WCS papers). Since then however the draft WCS paper has evolved and recommends a convention that differs from the SIP convention. There was some discussion on how this divergence could or should be rectified. No conclusions were reached, but Shupe agree to continue this discussion on the fitswcs email list in the near future. 7. Status of the TDIMn and Variable Length Array Proposals These 2 unofficial FITS conventions are currently defined in appendix B of the FITS Standard. A proposal to make them official by incorporating them into the Standard itself is now in the Public Comment Period phase of the formal review process. Everyone was encouraged to carefully review the proposals and submit any comments by 12 November. Francois Ochsenbein pointed out that the variable length array convention makes it more difficult to stream FITS format files from one application to another because in general the entire 'heap' of data (which could be very large) must be transmitted before the receiving application can process a single row of the table. This is especially relevant for VO applications where steaming of data is the norm. It was agreed that some cautionary statements about the possible adverse consequences of using the variable length array convention should be added to the proposal that is currently in review. 8. The FITS User's Guide The FITS User's Guide, last updated in 1997, is now in great need of revision to reflect the many changes that have taken place of the past 7 years. It currently contains many obsolete sections that would be confusing or misleading to new FITS users. Pence has begun updating some of the sections, and with the help of a few other contributors, hopes to have a new draft available in early 2005. 9. 64-bit Integer support in FITS Whether to add support in FITS for 64-bit integer images and table columns has been a controversial issue for many years. Technically, this would only require a few relatively simple changes to the wording in the FITS Standard: 64-bit integer images would be denoted by "BITPIX = 64", and binary table columns would have a "TFORM" type of 'K'. In addition, a new 64-bit array descriptor with a TFORM type of 'Q' could be defined (analogous to the 32-bit 'P' array descriptor. The controversy about this has mainly centered on the impact that this change would have on software developers and users. Recently it has become apparent that there are now few practical obstacles to prevent developers from writing software (or updating existing software) to support 64-bit integers using most programming languages and computer platforms in common use today. Even Fortran-77 compilers widely support the integer*8 data type. There ensued a general discussion on the need (or lack of need) for 64-bit integers by FITS data providers and on the possible impacts that this change to the FITS Standard would have. Many people expressed support for this proposal in general, but also recommended that some cautionary statements be added to urge users to not abuse or unnecessarily use this new feature if it is approved. At the end of the discussion a straw vote was taken to gauge the amount of support for this issue. Out of the 60 to 70 people in attendance, about 80% voted in favor of seriously considering this proposal now, and only 1 person voted for deferring consideration to some unspecified later date. The remaining 20% or so did not vote one way or the other. Given this strong support, it is likely that this issue will be brought up for further discussion on the FITSBITS email list and possible consideration by the FITS committees in the near future. 10. A Registry of FITS Conventions Pence briefly outlined a few ideas about establishing an official registry of FITS conventions. The general idea is that this would be a way to recognize and document existing important keyword conventions that are in use by at least some subset of the astronomical community. Details on how one could propose a convention to the registry, who would need to approve it, and what information would need to be supplied to go into the registry remain to be determined. There was also some discussion on how formal or detailed the registry should be, and ideas of how XML schema could be used, as in the VO registry of services. This will certainly be a topic of more discussion on FITSBITS in the coming year. 11. FITS and the VO The last topic of the BoF session was on the role of FITS and the FITS committees in the new Virtual Observatory era. No one offered any suggestions for technical changes to the FITS Standard that would make the FITS format more supportive of VO applications, so the discussion quickly moved on to the differences between the FITS and VO committee structures and the standards approval procedures. Some voiced the opinion that sooner or later, there should be some rationalization or merging of the various committees. It was pointed out that IAU Commission 5 is planning to create a VO oversight committee (at the same organizational level as the existing FITS Working Group) which might help to improve communications between developers in the 2 communities. If there are to be any major changes to current FITS committee structure, it is unlikely to occur before the next IAU General Assembly meeting in 2006. |
#2
|
|||
|
|||
Hi Bill,
About the Variable Length Array proposal, I would propose the following modification of the "proviso": "Because the Variable Length Array format disables the capability of consuming FITS files in data streams (all table rows and potentially the whole 'heap' has to be read in order to be able to process the first row), the producers of FITS data products should consider the capabilities of the likely recipients of their files when deciding whether or not to use this format, and as a general rule should use it only in cases where it provides significant advantages over the simpler fixed length array format." Would that be agreeable ? It could be noticed that recommending to store the data in the heap in the same order as the records (i.e. the "pointers" have values increasing with the ordinal row number, except in the case where a "pointer" indicates a value already specified in a previous row if a data array is shared by several rows) would minimize the quantity of data to read and store before the first row can be processed. --Francois ================================================== ==== Francois Ochsenbein CDS, Strasbourg ================================================== ==== |
#3
|
|||
|
|||
On Mon, 8 Nov 2004, William Pence wrote: Minutes of the ADASS FITS Birds-of-a-Feather Session ---------------------------------------------------- .... .... 5. Status of the WCS Paper III on Spectral Coordinate Systems Several of the authors of this paper were in attendance (Eric Greisen, Frank Valdes, and Steve Allen) and led a brief discussion. This paper is now in the Public Comment Period phase of the formal approval process, so everyone was encouraged to carefully review the paper and submit comments on FITSBITS by 12 November. Much of the discussion concerned what, if anything, should be required in the way of interoperability testing before the FITS committees vote on the proposal. It was noted that Mark Calabretta's wcslib software library supports most of the WCS conventions defined in the paper but there is little in the way of other software to enable independent testing. The AST library (http://www.starlink.ac.uk/ast/) also supports most of the paper III conventions with the one main exception that there is as yet no support for -TAB. David |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
FITS long integer support (was [fitsbits] ADASS FITS BoFon Sunday) | William Pence | FITS | 6 | October 22nd 04 08:23 PM |
globular clusters | AstronomyWanaB | Amateur Astronomy | 42 | August 19th 04 11:51 PM |
Messenger Launch SLC-17B 06:15:56 UTC 3 Aug 2004 | Sam Wormley | Amateur Astronomy | 4 | August 3rd 04 02:17 PM |
[fitsbits] Start of the FITS MIME type Public Comment Period | William Pence | FITS | 8 | June 17th 04 06:08 AM |
[fitsbits] Happy Birthday, FITS! | Don Wells | FITS | 0 | March 28th 04 01:58 PM |