A Space & astronomy forum. SpaceBanter.com

Go Back   Home » SpaceBanter.com forum » Astronomy and Astrophysics » FITS
Site Map Home Authors List Search Today's Posts Mark Forums Read Web Partners

[fitsbits] Potential new compression method for FITS tables



 
 
Thread Tools Display Modes
  #1  
Old December 17th 10, 06:11 PM posted to sci.astro.fits
Rob Seaman
external usenet poster
 
Posts: 49
Default [fitsbits] Potential new compression method for FITS tables

One additional point is that many vendor mass storage technologies can be configured with general purpose compression "built-in". Such a system can't take advantage of algorithms tailored for the particular data in question.

Rob
--

On Dec 17, 2010, at 9:27 AM, William Thompson wrote:

Preben Grosbol wrote:

As a side comments, I participated in a meeting for general archives a few
years ago. There they did not recommend to save compress data since
the effect of single bit errors is more serious than for raw data, not
justifying the gain of disk space.


Data compression addresses three issues:

1. The cost of archiving the data.
2. The rate at which data can be delivered to users
3. The ability of users to locally store data for scientific research

I mentioned before that the Solar Dynamics Observatory (SDO) mission is using
the Tiled Image Compression convention to help them manage their ~2 TB/day of
data volume. This is actually to address points 2 and 3 above than the issue of
archiving the data. The data are not archived in FITS format. Instead, the
data are stored in a database management system, with the images and associated
metadata stored separately. The FITS files are generated on-the-fly as
requested. This is how the same data can be delivered in both uncompressed and
Rice compressed formats.

I'm not sure how the images are stored within the database management system,
whether they're compressed or uncompressed, but I suspect that they're
compressed. I do know that the raw data from the telescopes undergo compression
on board the spacecraft before being telemetered down. At least for the AIA
telescope, this on-board compression is lossy--I'm not sure about HMI.

Bill Thompson


 




Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
[fitsbits] Potential new compression method for FITS tables Preben Grosbol FITS 0 December 17th 10 01:40 PM


All times are GMT +1. The time now is 03:40 PM.


Powered by vBulletin® Version 3.6.4
Copyright ©2000 - 2024, Jelsoft Enterprises Ltd.
Copyright ©2004-2024 SpaceBanter.com.
The comments are property of their posters.