Review Committee Meeting Wednesday 23 November 2005 08:30 PST / 11:30 EST
Minutes: Wednesday 23 November 2005 08:30 PST / 11:30 EST
We will continue the review of the "S4 LIGO-Only Untriggered Burst Search"
The technical report is:
New material has been added: there are now efficiency curves. We will start
the meeting by going over this new material.
- Review of existing validation for MDC frame production
- Sanity checks on efficiency curves. Do they make sense?
- Go through the checklist (repeated below) to check off as many
items as is possible at this time.
- Walk-through of the technical documentation (link above).
- Walk-through of GWDAW slides, if available ... otherwise just
look at slides presented at LSC meeting:
- Assign code reviewing tasks.
We will not be able to get through this agenda this week. We will need to
schedule more review telecons next week to get through as much material as we
can prior to the Dec 2 Executive Meeting.
Checklist of things to do for S4 LIGO-Only Untriggered Burst search.
1. THE BOX IS EMPTY: 0 events +- 10%
- Careful bookkeeping of time: make sure that we've not missed
anything (re-run of 22 segments)
- Check hrss reconstruction code changes
- Check changes to CorrPower
- Is there something peculiar about the two high GS WB events?
. Check why they are not in time-slides.
. Try to find origin of events.
- Look at remaining triggers after amplitude cuts (but below
gamma-threshold). Are they gravitational waves?
- Final look at R0 test: send Mike Landry details of HW injection
(channel, actuation function) so that he can determine if the
sign of the injection was correct.
We know these are "wrong" in the sense:
- Data is not treated consistently between foreground and
background -- biases introduced
- Foreground contaminates background
We cannot say that these are write to within 10% but they are
not vital to the analysis -- they are simply reassuring.
3. EFFICIENCIES: hrss50 values +- 10%
- Bookkeeping again: is everything the same?
- Check software vs. hardware sensitivities.
- Check MDC frames and software to produce them. Look at
- Assess effect of calibration errors. Inject MDC frames
with +-10% amplitude into pipeline? Estimate loss from
each stage of pipeline? Also, estimate effect of phase
errors on CorrPower.
For GWDAW: just need to see if 50% hrss levels will be affected
at the 10% level.
- Apply 10% adjustment (systematic) to account for final
calibration, plus whatever "conservative" additional systematic
is deemed necessary.
. Contact calibration team....
- Sanity check: noise curve level vs. hrss50% -- is it the same
ratio as before?
- Sanity check: do white-noise burst sensitivities scale with T-F
- High-hrss fall-off: how to deal with this? (Only 1% effect.)
- Discussion about efficiency for SG22 70Hz waveform.
- Keith Riles raised concerns about 70-Hz Q=100 sine Gaussians
- WB only efficiencies are on
Igor's page. Problem is at the
- Sergey analyzed the
He expects the problem has to do with a combination of the length of
the signal and the nearness to the low-frequency cutoff creating a
train of clusters that are mismatched; errors time estimations cause
the events to be rejected. For larger signals the effect is greater
because there are more clusters in train above threshold.
- Another issue might be that the signals last significantly longer
than 1 second, but only 1 second is injected. This might lead to
start/end transients. It should be possible to inject longer injections.
- Action: Sergey will produce wavelet spectrograms of signal
to illustrate this effect.
- Conclusion: this curve is an extreme case and illustrates the
behavior of the algorithm on the boundary of validity. It might be
worthwhile to show at GWDAW as a technical aside but will require
careful explaination. Not yet clear if it should be shown in a paper.
- Keith Thorne described existing validations of the MDC frames.
- Main S4 BurstMDC web page
- MDCs generated for S4
- MDC Validation Efforts
- These items are also tracked in the Burst Group's e-log under S4 -
- Beam functions have been checked against other independent code
(Patrick Sutton's and VIRGO's). There is a known issue with GEO
(arm-1, arm-2 vs. x-arm, y-arm convention) but this does not impact
the LIGO-only work.
- hrss has been checked by two independent codes: one by Keith Thorne
which computes hrss directly from frames, other by waveburst which gives
good agreement in its estimated hrss.
- Calibration has been checked earlier by comparing results of
analyzing ASQ vs. results using h(t). These comparisons were done with
V01 calibration, but should be valid for the V03 calibration too.
- Last remaining thing to check: do the sky and polarization angles
go over the correct ranges? Action: Keith Thorne will histogram
the random angles to see if they give the correct distribution.
- Concern about the frequency of injections: they are done on average
every 100 seconds. Waveburst does point estimations of power every
60 seconds, and whitens using a linear interpolation. Thus, whitening
might depend on an earlier injections. Based on earlier investigations,
this is not a significant effect. Waveforms are injected in random
order, so earlier injection would not always be the same. However,
amplitude is scaled the same for all injections in waveburst so loud
injections will have loud preceeding injections. Unlikely that this is
responsible for the Q=100 f=70Hz efficiency curve drop-off because we
would expect to see similar effects in other curves.
- Conclusion: we are comfortable with this level of validation of the
MDC frames for GWDAW.
- Bookkeeping: are we sure that the data that should be analyzed has been
analyzed? All waveburst jobs ran successfully. All trigger files exist
(they are produced at the end of a job). Trigger files are intact. Log
shows exactly the same amount of data analyzed as the segment lists.
Keith Riles did a second check of this and is comfortable. Conclusion:
we believe that all the data has been processed and accounted for at least
to the end of waveburst.
- More sanity checks on efficiency curves:
- Do the white noise bursts behave as expected? Would normally expect
that the hrss50 would scale as the fourth-root of the time-frequency
volume. This seems to be generally true but there are a few cases where
it is not. These could be due to other factors.
- Need to see the hrss50 points plotted on an S4 noise curve. Patrick
Sutton might be able to produce average-S4 noise curves. But the best
S4 noise curves should be sufficient for a sanity check.
Action: Igor will see if it is easy to plot hrss50 points on S4
noise curve; Laura might have scripts to do this.
- Calibration errors:
- Peter Shawhan reports the V04 calibration will show that H1 is 10%
worse and that H2 is approximately the same sensitivity. Overall
sensitivity of search therefore might not be much worse.
- Main concern: amplitude cut. Will this lead to a large loss of events?
At low SNR? Will this change hrss50? Igor suggests that cuts are quite
loose, and that studies have shown only a few percent drop in efficiency
when the cuts are tightened significantly. Action: we will look
at the injection amplitude-vs-amplitude scatterplots to see if errors in
amplitude might cause a significant loss of detection.
- How will effect of using V03 calibration rather than V04 calibration
be estimated? We should not just build in a 10% correction since it is
the most sensitive instrument that is changing. One possibility is to do
a new set of MDCs where the injected signals are calibrated with V04
calibration but the analysis is done with V03 calibration as usual.
Action: Keith Thorne will produce V04 MDC files when the V04
calibration is avaliable. In the past, systematic errors (including
calibration) have been applied to hrss50 values quoted, but are not
displayed on efficiency curves. Conclusion: calibration will probably
not affect the search efficiencies at the 10% level (but it could be close)
so this is probably not an issue for GWDAW unless there would be
significant loss of injections due to the amplitude cut.
- Efficiency curve fits. It looks like the fits are strange, having odd
kinks where a more satisfactory fit would not. This is only a small effect
but it is most significant at the 50% point. We need to consider making
better fits for the efficiency curves.
- Other Actions: Everyone should look through the checklist to
identify items that they can address. This might be simply sending around
a link to an existing enote item, or by doing some follow-up studies. The
more that we can do off-line, the more we can get done during the telecons.
$Id: minutes-2005-11-23.html,v 1.5 2005/11/24 22:43:27 jolien Exp $