|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
[fitsbits] Potential new compression method for FITS tables
On Oct 28, 2010, at 1:42 PM, William Pence wrote:
For the past few months, several of us (Rob Seaman, Rick White, and myself) have been experimenting with a new compression method for FITS binary tables that appears to be significantly more effective than the usual method of simply compressing the whole FITS file with gzip. We have produced a document, available at http://fits.gsfc.nasa.gov/tiletable.pdf that describes this proposed convention in more detail; "Tables" are a broad class of data structure. So far we have focused on general tabular structures such as catalogs that are suitable fodder for lossless compression algorithms. Another class of table is a spectrum that might associate a flux vector with vectors for wavelength and variance, for instance. As with imaging data, this may well be appropriate for compression via a lossy technique. The initial draft of the convention doesn't focus on such cases, but it's worth investigating the community interest. On the other hand, the GZIP_2 algorithm that shuffles the input data vector may well be useful for the case of lossless compression of floating-point images in the prior tiled-image convention. Reduced data products such as calibrated spectra or image pipeline output can benefit from the (technically) lossy compression ("efficient representation") of floating-point pixels. However some cameras, especially in the infrared, produce *raw* floating-point images as the result of internal coadditions or sampling techniques via non-destructive reads. Raw data of whatever type is typically required to be losslessly preserved. One size does not fit all. Rob Seaman NOAO |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
[fitsbits] WCS keywords in tables | Steve Allen | FITS | 0 | April 6th 10 06:12 PM |
[fitsbits] Update to the FITS Tiled Image Compression Convention | William Pence[_2_] | FITS | 0 | September 18th 09 05:08 PM |
[fitsbits] BZIP2 FITS image compression feasibility study | William Pence[_2_] | FITS | 0 | June 3rd 09 09:20 PM |
[fitsbits] FITS Image Compression Paper and Software | William Pence[_2_] | FITS | 0 | March 17th 09 08:07 PM |
FITS Image compression, is it portable? | Carsten A. Arnholm | Amateur Astronomy | 4 | May 31st 06 09:59 PM |