SFTfileIO.c File Reference

Author:
R.IO library for reading/writing "Short Fourier transform" (SFT) data files.
More...

#include <sys/types.h>
#include <errno.h>
#include <string.h>
#include <strings.h>
#include <dirent.h>
#include <lal/LALStdio.h>
#include <lal/FileIO.h>
#include <lal/SFTfileIO.h>
#include <lal/StringVector.h>

Include dependency graph for SFTfileIO.c:

Go to the source code of this file.

Data Structures

struct  tagSFTLocator
struct  _SFT_header_v1_t
struct  _SFT_header_v2_t

Defines

#define MIN_SFT_VERSION   1
#define MAX_SFT_VERSION   2
#define TRUE   1
#define FALSE   0
#define BLOCKSIZE   8192 * 8
 blocksize used in SFT-reading for the CRC-checksum computation (has to be multiple of 8 !!)
#define REALLOC_BLOCKSIZE   100
#define GPS2REAL8(gps)   (1.0 * (gps).gpsSeconds + 1.e-9 * (gps).gpsNanoSeconds )
#define GPSEQUAL(gps1, gps2)   (((gps1).gpsSeconds == (gps2).gpsSeconds) && ((gps1).gpsNanoSeconds == (gps2).gpsNanoSeconds))
#define MYROUND(x)   ( floor( (x) + 0.5 ) )
#define FILE_SEPARATOR   ';'
#define DIR_SEPARATOR   '/'
#define POLY64   0xd800000000000000ULL
#define TABLELEN   256
#define NEGATE   '^'

Functions

 NRCSID (SFTFILEIOC,"$Id: SFTfileIO.c,v 1.134 2008/10/08 13:17:38 bema Exp $")
int read_SFTversion_from_fp (UINT4 *version, BOOLEAN *need_swap, FILE *fp)
 Read valid SFT version-number at position fp, and determine if we need to endian-swap the data.
void _LALLoadSFTs (LALStatus *status, SFTVector **sfts,const SFTCatalog *catalog,REAL8 fMin,REAL8 fMax)
 Load the given frequency-band [fMin, fMax] (inclusively) from the SFT-files listed in the SFT-'catalogue' ( returned by LALSFTdataFind() ).
void LALSFTdataFind (LALStatus *status, SFTCatalog **catalog, const CHAR *file_pattern, SFTConstraints *constraints)
 Find the list of SFTs matching the file_pattern and satisfying the given constraints, return an SFTCatalog of the matching SFTs.
void LALSFTtimestampsFromCatalog (LALStatus *status, LIGOTimeGPSVector **timestamps, const SFTCatalog *catalog)
 Extract a timstamps-vector from the given SFTCatalog.
void LALLoadSFTs (LALStatus *status, SFTVector **outsfts, const SFTCatalog *catalog, REAL8 fMin, REAL8 fMax)
void LALLoadMultiSFTs (LALStatus *status, MultiSFTVector **out, const SFTCatalog *inputCatalog, REAL8 fMin, REAL8 fMax)
 Function to load a catalog of SFTs from possibly different detectors.
void LALCheckSFTs (LALStatus *status, INT4 *check_result, const CHAR *file_pattern, SFTConstraints *constraints)
 Function to check validity of SFTs listed in catalog.
void LALCheckSFTCatalog (LALStatus *status, INT4 *check_result, SFTCatalog *catalog)
void LALReadTimestampsFile (LALStatus *status, LIGOTimeGPSVector **timestamps, const CHAR *fname)
 Read timestamps file and returns timestamps vector (alloc'ed in here!).
void LALWriteSFT2file (LALStatus *status, const SFTtype *sft, const CHAR *fname, const CHAR *comment)
 Write the given *v2-normalized* (i.e.
void LALWriteSFTVector2Dir (LALStatus *status, const SFTVector *sftVect, const CHAR *dirname, const CHAR *comment, const CHAR *description)
 Write the given *v2-normalized* (i.e.
void LALWrite_v2SFT_to_v1file (LALStatus *status, const SFTtype *sft, const CHAR *fname)
 For backwards-compatibility: write a *v2-normalized* (ie dt x DFT) SFTtype to a v1-SFT file.
void LALWriteSFTfile (LALStatus *status, const SFTtype *sft, const CHAR *outfname)
 [OBSOLETE] Write a *v1-normalized* (i.e.
void LALDestroySFTCatalog (LALStatus *status, SFTCatalog **catalog)
 Free an 'SFT-catalogue'.
const CHARXLALshowSFTLocator (const struct tagSFTLocator *locator)
 Mostly for *debugging* purposes: provide a user-API to allow inspecting the SFT-locator [which is an OPAQUE entry in the SFTCatalog!].
INT4 XLALCountIFOsInCatalog (const SFTCatalog *catalog)
void LALGetSFTheaders (LALStatus *status, SFTVector **headers, const CHAR *fpattern, const LIGOTimeGPS *startTime, const LIGOTimeGPS *endTime)
 [DEPRECATED] Function to read and return a list of SFT-headers for given filepattern and start/end times.
void LALReadSFTfile (LALStatus *status, SFTtype **sft, REAL8 fMin, REAL8 fMax, const CHAR *fname)
 [DEPRECATED] Basic SFT reading-function.
void LALReadSFTfiles (LALStatus *status, SFTVector **sftvect, REAL8 fMin, REAL8 fMax, UINT4 wingBins, const CHAR *fpattern)
 [DEPRECATED] Higher-level SFT-reading function to read a whole vector of SFT files and return an SFTvector sftvect.
void LALReadSFTheader (LALStatus *status, SFTHeader *header, const CHAR *fname)
 [DEPRECATED]: Low-level function to read only the SFT-header of a given file.
void LALReadSFTdata (LALStatus *status, SFTtype *sft, const CHAR *fname, INT4 fminBinIndex)
 [DEPRECATED] This is a function for low-level SFT data-reading: the SFT-data is read starting from fminBinIndex and filled into the pre-allocate vector sft of length N

Variables

const SFTConstraints empty_SFTConstraints
const SFTCatalog empty_SFTCatalog


Detailed Description

Author:
R.IO library for reading/writing "Short Fourier transform" (SFT) data files.

Prix, B. Machenschalk, A.M. Sintes, B. Krishnan

This implements the SFTv2 standard defined in LIGO-T040164-01-Z A previous non-LAL implementation of this standard is found in the "SFT reference library" gravity.phys.uwm.edu:2402/usr/local/cvs/lscsoft sftlib, Copyright (C) 2004 Bruce Allen

The function calc_crc64() here is based on crc64() in SFTReferenceLibrary.c.

Id
SFTfileIO.c,v 1.134 2008/10/08 13:17:38 bema Exp

Definition in file SFTfileIO.c.


Define Documentation

#define MIN_SFT_VERSION   1

Definition at line 58 of file SFTfileIO.c.

#define MAX_SFT_VERSION   2

Definition at line 59 of file SFTfileIO.c.

#define TRUE   1

Definition at line 61 of file SFTfileIO.c.

#define FALSE   0

Definition at line 62 of file SFTfileIO.c.

#define BLOCKSIZE   8192 * 8

blocksize used in SFT-reading for the CRC-checksum computation (has to be multiple of 8 !!)

Definition at line 65 of file SFTfileIO.c.

#define REALLOC_BLOCKSIZE   100

Definition at line 78 of file SFTfileIO.c.

#define GPS2REAL8 ( gps   )     (1.0 * (gps).gpsSeconds + 1.e-9 * (gps).gpsNanoSeconds )

Definition at line 81 of file SFTfileIO.c.

#define GPSEQUAL ( gps1,
gps2   )     (((gps1).gpsSeconds == (gps2).gpsSeconds) && ((gps1).gpsNanoSeconds == (gps2).gpsNanoSeconds))

Definition at line 83 of file SFTfileIO.c.

#define MYROUND ( x   )     ( floor( (x) + 0.5 ) )

Definition at line 86 of file SFTfileIO.c.

#define FILE_SEPARATOR   ';'

#define DIR_SEPARATOR   '/'

#define POLY64   0xd800000000000000ULL

Definition at line 3610 of file SFTfileIO.c.

#define TABLELEN   256

Definition at line 3611 of file SFTfileIO.c.

#define NEGATE   '^'

Definition at line 3791 of file SFTfileIO.c.


Function Documentation

NRCSID ( SFTFILEIOC  ,
"$Id: SFTfileIO.  c,
v 1.134 2008/10/08 13:17:38 bema Exp $"   
)

int read_SFTversion_from_fp ( UINT4 version,
BOOLEAN need_swap,
FILE *  fp 
)

Read valid SFT version-number at position fp, and determine if we need to endian-swap the data.

Restores filepointer to original position before returning.

RETURN: 0 = OK, -1 = ERROR

Definition at line 3702 of file SFTfileIO.c.

void _LALLoadSFTs ( LALStatus status,
SFTVector **  sfts,
const SFTCatalog catalog,
REAL8  fMin,
REAL8  fMax 
)

Load the given frequency-band [fMin, fMax] (inclusively) from the SFT-files listed in the SFT-'catalogue' ( returned by LALSFTdataFind() ).

Note: fMin (or fMax) is allowed to be set to -1, which means to read in all Frequency-bins from the lowest (or up to the highest) found in the SFT-file.

Note 2: The returned frequency-interval is guaranteed to contain [fMin, fMax], but is allowed to be larger, as it must be an interval of discrete frequency-bins as found in the SFT-file.

Parameters:
sfts  [out] vector of read-in SFTs
catalog  The 'catalogue' of SFTs to load
fMin  minumum requested frequency (-1 = read from lowest)
fMax  maximum requested frequency (-1 = read up to highest)

Definition at line 564 of file SFTfileIO.c.

void LALSFTdataFind ( LALStatus status,
SFTCatalog **  catalog,
const CHAR file_pattern,
SFTConstraints constraints 
)

Find the list of SFTs matching the file_pattern and satisfying the given constraints, return an SFTCatalog of the matching SFTs.

The optional constraints that can be specified are (type SFTConstraints)

==> The returned SFTCatalog can be used directly as input to LALLoadSFTs() to load a single-IFO SFTVector, or LALLoadMultiSFTs() to load a multi-IFO vector of SFTVectors

Except for the 'file_pattern' input, all the other constraints are optional and can be passed as NULL (either globally constraings==NULL, or individually).

Note that the constraints are combined by 'AND' and the resulting full constraint MUST be satisfied (in particular: if 'timestamps' is given, all timestamps within [startTime, endTime] MUST be found!.

The returned SFTs in the catalogue are sorted by increasing GPS-epochs !

Parameters:
catalog  [out] SFT-catalogue of matching SFTs
file_pattern  which SFT-files
constraints  additional constraints for SFT-selection

Definition at line 184 of file SFTfileIO.c.

void LALSFTtimestampsFromCatalog ( LALStatus status,
LIGOTimeGPSVector **  timestamps,
const SFTCatalog catalog 
)

Extract a timstamps-vector from the given SFTCatalog.

Note:
A list of *unique* timestamps is returned, i.e. only a single copy of a timestamp is returned, even if there are multiple occurrances of this timestamp in the catalog, e.g. for multiple IFOs or multiple frequency-bands...
Parameters:
timestamps  [out] extracted timestamps
catalog  input SFT-catalogue

Definition at line 502 of file SFTfileIO.c.

void LALLoadSFTs ( LALStatus status,
SFTVector **  outsfts,
const SFTCatalog catalog,
REAL8  fMin,
REAL8  fMax 
)

Parameters:
outsfts  [out] vector of read-in SFTs
catalog  The 'catalogue' of SFTs to load
fMin  minumum requested frequency (-1 = read from lowest)
fMax  maximum requested frequency (-1 = read up to highest)

Definition at line 642 of file SFTfileIO.c.

void LALLoadMultiSFTs ( LALStatus status,
MultiSFTVector **  out,
const SFTCatalog inputCatalog,
REAL8  fMin,
REAL8  fMax 
)

Function to load a catalog of SFTs from possibly different detectors.

This is similar to LALLoadSFTs except that the input SFT catalog is allowed to contain multiple ifos. The output is the structure MultiSFTVector which is a vector of (pointers to) SFTVectors, one for each ifo found in the catalog. As in LALLoadSFTs, fMin and fMax can be set to -1 to get the full SFT from the lowest to the highest frequency bin found in the SFT.

output SFTvectors are sorted alphabetically by detector-name

Parameters:
out  [out] vector of read-in SFTs -- one sft vector for each ifo found in catalog
inputCatalog  The 'catalogue' of SFTs to load
fMin  minumum requested frequency (-1 = read from lowest)
fMax  maximum requested frequency (-1 = read up to highest)

Definition at line 901 of file SFTfileIO.c.

void LALCheckSFTs ( LALStatus status,
INT4 check_result,
const CHAR file_pattern,
SFTConstraints constraints 
)

Function to check validity of SFTs listed in catalog.

This function simply reads in those SFTs and checks their CRC64 checksum, which is the only check that has not yet been done by the operations up to this point.

Returns the LAL-return code of a failure (or 0 on success) in 'check_result'.

Note:
: because this function has to read the complete SFT-data into memory for the whole set of matching SFTs, it is potentially slow and memory-intensive.
This function will NOT fail if one of the SFT-operations fails, instead it returns the status-code of this failure in 'check_result'. The function will fail, however, on invalid input.
Parameters:
check_result  LAL-status of SFT-operations
file_pattern  where to find the SFTs: normally a path+file-pattern
constraints  additional constraints for SFT-selection

Definition at line 1116 of file SFTfileIO.c.

void LALCheckSFTCatalog ( LALStatus status,
INT4 check_result,
SFTCatalog catalog 
)

Parameters:
check_result  LAL-status of SFT-operations
catalog  catalog of SFTs to check

Definition at line 1159 of file SFTfileIO.c.

void LALReadTimestampsFile ( LALStatus status,
LIGOTimeGPSVector **  timestamps,
const CHAR fname 
)

Read timestamps file and returns timestamps vector (alloc'ed in here!).

Definition at line 1220 of file SFTfileIO.c.

void LALWriteSFT2file ( LALStatus status,
const SFTtype sft,
const CHAR fname,
const CHAR comment 
)

Write the given *v2-normalized* (i.e.

dt x DFT) SFTtype to a v2-SFT file. Add the comment to SFT if comment != NULL.

NOTE: Currently this only supports writing v2-SFTs. If you need to write a v1-SFT, you should use LALWrite_v2SFT_to_v1file()

NOTE2: the comment written into the SFT-file contains the 'sft->name' field concatenated with the user-specified 'comment'

Parameters:
sft  SFT to write to disk
fname  filename
comment  optional comment (for v2 only)

Definition at line 1286 of file SFTfileIO.c.

void LALWriteSFTVector2Dir ( LALStatus status,
const SFTVector sftVect,
const CHAR dirname,
const CHAR comment,
const CHAR description 
)

Write the given *v2-normalized* (i.e.

dt x DFT) SFTVector to a directory. Add the comment to SFT if comment != NULL.

NOTE: Currently this only supports writing v2-SFTs. If you need to write a v1-SFT, you should use LALWriteSFTfile()

Output SFTs have naming convention following LIGO-T040164-01

Parameters:
sftVect  SFT vector to write to disk
dirname  base filename (including directory path)
comment  optional comment (for v2 only)
description  optional sft description to go in the filename

Definition at line 1403 of file SFTfileIO.c.

void LALWrite_v2SFT_to_v1file ( LALStatus status,
const SFTtype sft,
const CHAR fname 
)

For backwards-compatibility: write a *v2-normalized* (ie dt x DFT) SFTtype to a v1-SFT file.

NOTE: the only difference to WriteSFTfile() is that the data-normalization is changed back to v1-type 'DFT', by dividing the dt corresponding to the frequency-band contained in the SFTtype.

Parameters:
sft  SFT to write to disk
fname  filename

Definition at line 1506 of file SFTfileIO.c.

void LALWriteSFTfile ( LALStatus status,
const SFTtype sft,
const CHAR outfname 
)

[OBSOLETE] Write a *v1-normalized* (i.e.

raw DFT) SFTtype to a SFT-v1 file.

Note:
:only SFT-spec v1.0 is supported, and the SFTtype must follow the *obsolete* v1-normalization. => Use LALWriteSFT2file() to write v2 SFTs !
Parameters:
sft  SFT to write to disk
outfname  filename

Definition at line 1564 of file SFTfileIO.c.

void LALDestroySFTCatalog ( LALStatus status,
SFTCatalog **  catalog 
)

Free an 'SFT-catalogue'.

Parameters:
catalog  the 'catalogue' to free

Definition at line 1668 of file SFTfileIO.c.

const CHAR* XLALshowSFTLocator ( const struct tagSFTLocator locator  ) 

Mostly for *debugging* purposes: provide a user-API to allow inspecting the SFT-locator [which is an OPAQUE entry in the SFTCatalog!].

NOTE: this returns a STATIC string, so don't try to FREE it, and make a copy if you need to keep it beyond one call of this function!

Definition at line 1721 of file SFTfileIO.c.

INT4 XLALCountIFOsInCatalog ( const SFTCatalog catalog  ) 

Definition at line 1736 of file SFTfileIO.c.

void LALGetSFTheaders ( LALStatus status,
SFTVector **  headers,
const CHAR fpattern,
const LIGOTimeGPS startTime,
const LIGOTimeGPS endTime 
)

[DEPRECATED] Function to read and return a list of SFT-headers for given filepattern and start/end times.

The startTime and endTime entries are allowed to be NULL, in which case they are ignored.

Note:
We return the headers as an SFTVector, but with empty data-fields.
Parameters:
headers  [out] Vector of SFT-headers
fpattern  path/filepattern
startTime  include only SFTs after this time (can be NULL)
endTime  include only SFTs before this (can be NULL)

Definition at line 1791 of file SFTfileIO.c.

void LALReadSFTfile ( LALStatus status,
SFTtype **  sft,
REAL8  fMin,
REAL8  fMax,
const CHAR fname 
)

[DEPRECATED] Basic SFT reading-function.

Given a filename fname and frequency-limits [fMin, fMax], returns an SFTtype sft containing the SFT-data.

Note:
1) the actual returned frequency-band is [floor(Tsft * fMin), ceil(Tsft * fMax)] / Tsft, i.e. we need to round to an integer frequency-bin within the SFT, but the requested frequency-band is guaranteed to be contained in the output (if present in the SFT-file), but can be slightly larger.
2) The special input fMin=fMax=0 means to read and return the whole frequency-band contained in the SFT-file.

3) Currently only SFTv1 are supported!!

Parameters:
sft  [out] output SFT
fMin  lower frequency-limit
fMax  upper frequency-limit
fname  path+filename

Definition at line 1901 of file SFTfileIO.c.

void LALReadSFTfiles ( LALStatus status,
SFTVector **  sftvect,
REAL8  fMin,
REAL8  fMax,
UINT4  wingBins,
const CHAR fpattern 
)

[DEPRECATED] Higher-level SFT-reading function to read a whole vector of SFT files and return an SFTvector sftvect.

The handling of [fMin, fMax] is identical to LALReadSFTfile().

Note:
1) the file-pattern fpattern can use a wide range of glob-patterns.
2) currently the SFTs matching the pattern are required to have the same number of frequency bins, otherwise an error will be returned. (This might be relaxed in the future).

3) This function does not use glob() and should therefore be safe even under condor.

Parameters:
sftvect  [out] output SFT vector
fMin  lower frequency-limit
fMax  upper frequency-limit
wingBins  number of frequency-bins to be added left and right.
fpattern  path/filepattern

Definition at line 2002 of file SFTfileIO.c.

void LALReadSFTheader ( LALStatus status,
SFTHeader header,
const CHAR fname 
)

[DEPRECATED]: Low-level function to read only the SFT-header of a given file.

NOTE: don't use! This function is obsolete and SFT-v1-specific and only kept for backwards-compatibility with Hough-codes.

Parameters:
header  [out] returned header
fname  path+filename

Definition at line 2213 of file SFTfileIO.c.

void LALReadSFTdata ( LALStatus status,
SFTtype sft,
const CHAR fname,
INT4  fminBinIndex 
)

[DEPRECATED] This is a function for low-level SFT data-reading: the SFT-data is read starting from fminBinIndex and filled into the pre-allocate vector sft of length N

NOTE: !! NO re-normalization is done here!! this remains up to the caller of this function!!

Parameters:
sft  [out] output-SFT: assuming memory is allocated
fname  path+filename
fminBinIndex  minimun frequency-index to read

Definition at line 2347 of file SFTfileIO.c.


Variable Documentation

const SFTConstraints empty_SFTConstraints

Definition at line 128 of file SFTfileIO.c.

const SFTCatalog empty_SFTCatalog

Definition at line 129 of file SFTfileIO.c.


Generated on Tue Oct 14 02:33:21 2008 for LAL by  doxygen 1.5.2