A Space & astronomy forum. SpaceBanter.com

Go Back   Home » SpaceBanter.com forum » Astronomy and Astrophysics » FITS
Site Map Home Authors List Search Today's Posts Mark Forums Read Web Partners

[fitsbits] Potential new compression method for FITS tables



 
 
Thread Tools Display Modes
  #1  
Old October 30th 10, 12:46 AM posted to sci.astro.fits
Rob Seaman
external usenet poster
 
Posts: 49
Default [fitsbits] Potential new compression method for FITS tables

On Oct 28, 2010, at 1:42 PM, William Pence wrote:

For the past few months, several of us (Rob Seaman, Rick White, and myself) have been experimenting with a new compression method for FITS binary tables that appears to be significantly more effective than the usual method of simply compressing the whole FITS file with gzip. We have produced a document, available at http://fits.gsfc.nasa.gov/tiletable.pdf that describes this proposed convention in more detail;


"Tables" are a broad class of data structure. So far we have focused on general tabular structures such as catalogs that are suitable fodder for lossless compression algorithms.

Another class of table is a spectrum that might associate a flux vector with vectors for wavelength and variance, for instance. As with imaging data, this may well be appropriate for compression via a lossy technique. The initial draft of the convention doesn't focus on such cases, but it's worth investigating the community interest.

On the other hand, the GZIP_2 algorithm that shuffles the input data vector may well be useful for the case of lossless compression of floating-point images in the prior tiled-image convention. Reduced data products such as calibrated spectra or image pipeline output can benefit from the (technically) lossy compression ("efficient representation") of floating-point pixels. However some cameras, especially in the infrared, produce *raw* floating-point images as the result of internal coadditions or sampling techniques via non-destructive reads. Raw data of whatever type is typically required to be losslessly preserved.

One size does not fit all.

Rob Seaman
NOAO


 




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[fitsbits] WCS keywords in tables Steve Allen FITS 0 April 6th 10 06:12 PM
[fitsbits] Update to the FITS Tiled Image Compression Convention William Pence[_2_] FITS 0 September 18th 09 05:08 PM
[fitsbits] BZIP2 FITS image compression feasibility study William Pence[_2_] FITS 0 June 3rd 09 09:20 PM
[fitsbits] FITS Image Compression Paper and Software William Pence[_2_] FITS 0 March 17th 09 08:07 PM
FITS Image compression, is it portable? Carsten A. Arnholm Amateur Astronomy 4 May 31st 06 09:59 PM


All times are GMT +1. The time now is 11:18 PM.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 SpaceBanter.com.
The comments are property of their posters.