Code Changes in S5 BlockNormal Data-Conditioning
IntroductionThe data-conditioning used in the BlockNormal event generation has been relatively unchanged since its initial implementation in S2. The same steps of down-sampling (low-pass filter), high-pass shaping filter, Kalman line filters, frequency band filters, regression filters and a final whitening filter have remained. The technique of calculating filters from each playground has remained unchanged.
There have been some software changes since S2. A narrative of the S2 code implementation was prepared for the S2 review (S2_BNETG_Implementation.pdf). Only three changes affected the filter calculations. The rest were to make the code more flexible and to support the long S5 run. These changes are detailed below. A narrative of the S5 FirstYear code implementation has now been prepared for the S5 review (S5_BNETG_Implementation.pdf)
The Penn State group uses two revision control systems. A local Subversion repository (PSUBurst) is used for development and testing. The code is published to the `matapps' CVS repository for use in LSC data analysis. This change document covers the BN_2007-07-09 code release. The archived data-conditioning code can be viewed through this web interface
Changes in calculations
Corrected Kalman filter implementation In the S3 analysis, we uncovered a problem with Kalman process noise measurement (Kalman process noise study web page). This was traced to errors in the normalization and incorrect determination of the line-width. The line-width was being used as a tunable parameter, instead of being set be the FWHM of the spectral feature. We corrected Kalman filter creation routines. We also changed the Kalman line determination to set the line width based on the spectral feature. This lead to the automated spectral line finding code Initial S3 Test in Dec 04.
Changed routines: kmat.m, calcW.m
Remove use of noshiftresampleIn the S2 analysis code, we had used a modified version (noshiftresample) for the MATLAB resample function to perform the downsample. This was done because impulse tests in initial development acheived the smallest time-shifts with additional zero-padding of this filter. However, there were problems with the implementation. We did more detailed testing on steps as well as impulses. This S3 study showed that the standard MATLAB 'resample' gave equivalent or better performance on the 1:4 down-sampling. We thus removed the use of this special function.
Changed routines: BNETG, dc, blimit
Deprecated routines: noshiftresample
Change from AS_Q to DARM_ERRFor the S4 and S5 runs, we changed from LSC-AS_Q to LSC-DARM_ERR as the gravitational wave channel. In the per-playground data-conditioning filter creation, we changed the channel passed on GPS time. During event generation, the BlockNormal routine (BNETG) has already read in the time-series data.
Changed routines: FilterDrvr, calc_regress
Changes in implementation
Remove hard-coded filter specificationsThe S2 review identified a problem with the use of hard-coded values for the high-pass shaping, Kalman, regression and whitening filter specifications. This was done by moving all filter specification to ASCII files. The per-IFO high-pass shaping and band filters are defined in one file. The per-IFO Kalman, regression and whitening filter lists are done in separate files. This is shown in the S5 Data-Conditioning page. This required changing all the data-conditioning initialization code. This change also made it possible to automate a number of filter tuning steps, such as band tuning.
Changed routines: dc, dcinit, dc_set_band
New routines: dcsetkalman, dcsetregress, dcsetwhiten, rdbandfilters, rdkalmanfilters, rdregressfilters, rdwhitenfilters, getbandfiltersfile,getshapingfilter, getdcspecfile, getdcfilename, getdcspecfile, wrkalmanfilters
Deprecated routines: dc_set_H1_Kalman, dc_set_H2_Kalman, dc_set_L1_Kalman, dc_set_H1_regress, dc_set_H2_regress, dc_set_L1_regress, dc_set_H1_whiten, dc_set_H2_whiten, dc_set_L1_whiten
Make filter configuation selectableTo enable performance testing and simplify use of BlockNormal on auxiliary channel data, all filter stages (low-pass down-sample, high-pass shaping, Kalman, regression and whitening) were made selectable using Boolean flags in a structure.
Changed routines: dc, dcinit, FilterDrvr, calcW, calc_regress, calc_whiten
New routines: prdcconfig
Character strings for band identifiersTo support the use of different bands for different runs, the frequency bands are now identified by a string. The code support the old numbered bands, but for S3 and onward, bands are dealt with as strings.
Changed routines: dc_set_band, getbandfreq
New routines: band2str
GPS-derived sub-directories for playground filter filesWe calculate filter files for each playground (about once every 6000s). The long duration of the S5 run results in quantities of filter files that are too large for single directories. We implemented a GPS-derived sub-directory scheme for these files.
Changed routines: calcW, calc_regress, calc_whiten, getdcfilename
New routines: getdcsubdir
GPS-dependent filter spec filesThe long duration of the S5 run has meant that we have need to change the mix of Kalman filters over time. To support this, there can now be multiple filter specification files. The names have the embedded GPS ranges they apply to (we adopted this from the frame file spec). This change supports existing functionality. The change is mostly in the `getdcspecfile' routine. The calling routines merely need to pass the desired GPS time to it.
Changed routines: getdcspecfile, dcsetkalman, dcsetregress, dcsetwhiten