|
|
Thread Tools | Display Modes |
#1
|
|||
|
|||
[fitsbits] Large FITS files
On Mon 2007-07-30T10:08:07 -0400, Robert J. Harris hath writ:
I'm looking to process large ( ~ GB) FITS datacubes. They are large enough not to load into memory. What I would like to do is input the cube, to perform a filtering operation on each plane of the cube, and then output another cube made up of the planes of the original cube after they have been processed. If it weren't for the last part of it, I could use the extended filename syntax of cfitsio, but I do need to put the planes back together, and it's not obvious how I might do that. Any help is appreciated. Thanks in advance! Our 64 Mpix images from the DEIMOS 8k*8k imager aren't that big, but they seemed a bit more of a challenge years ago when we were first preparing to write them. We use the CFITSIO fits_create_memfile() call after using the Unix shared memory routines to get a chunk of shared memory. Then we use the other routines to create the FITS elements and store the offsets in data structures which are used thereafter by code which is unaware that it is actually addressing memory which conforms to FITS. In early tests we experimented with using Unix memory mapped files instead of shared memory. It worked pretty much the same as far as the use of fits_create_memfile() was concerned, but we found that the operating system (with a little tuning of the kernel params) was better at deciding how to do the swapping than our code was at trying the same thing. -- Steve Allen WGS-84 (GPS) UCO/Lick Observatory Natural Sciences II, Room 165 Lat +36.99855 University of California Voice: +1 831 459 3046 Lng -122.06015 Santa Cruz, CA 95064 http://www.ucolick.org/~sla/ Hgt +250 m |
Thread Tools | |
Display Modes | |
|
|
Similar Threads | ||||
Thread | Thread Starter | Forum | Replies | Last Post |
[fitsbits] Large FITS files | Robert J. Harris | FITS | 0 | July 30th 07 03:08 PM |
[fitsbits] HEALPIX fits files | Eric Greisen | FITS | 6 | January 27th 05 01:19 AM |
FITS long integer support (was [fitsbits] ADASS FITS BoF onSunday) | Eric Greisen | FITS | 10 | October 26th 04 08:14 AM |
FITS long integer support (was [fitsbits] ADASS FITS BoFon Sunday) | William Pence | FITS | 6 | October 22nd 04 08:23 PM |
FITS long integer support (was [fitsbits] ADASS FITS BoF onSunday) | Thomas McGlynn | FITS | 0 | October 20th 04 03:18 PM |