LSC-Virgo Burst Analysis Working Group

Navigation

Burst Group Home
Wiki
ExtTrig Home
CBC Group Home
LSC, LIGO, Virgo

Documents

'How-to' docs
Agendas/minutes
[2008, 2007, earlier]
Talks/Posters [pre-wiki]
Papers
Paper plans
White papers
ViewCVS

Investigations

Analysis projects
Old notebook [General, S2, S3, S4, S5]
Virgo workarea
External collabs

Review

Main review page
Telecons

S5 LIGO-Only Untriggered Review Bulletins

Comparison of measured and predicted hrss values for cWB

This page details the effort to model the sensitivity of cWB during the first year of S5. The procedure is based on that used for the S4 LIGO-GEO review, though with a more sophisticated and accurate modelling of the (pretty complicated) coherent WaveBurst algorithm. In brief, random sky positions and polarizations are chosen for a large number of simulated GWBs of different hrss amplitudes. For each injection, we generate simulated data corresponding to a single time-frequency pixel of the whitened data stream, including both the signal and the Gaussian noise background. (The background noise is modelled using run-averaged noise spectra provided by Sergei Klimenko.) The various coherent WaveBursts statistics are computed (regularized likelihood, reduced correlated energy, rho_effective, etc.) In particular, the rho_effective for each "injection" is computed and compared to the thresholds used in S5 to estimate the cWB detection effciency versus hrss. Interpolation then gives the prediction for hrss50 and hrss90.

The single-pixel version of the script used is MC_cWB_S5.m. The multiple-pixel version is MC_cWB_S5_broad.m. The helper functions they call are in the X-Pipeline SVN repository.

Status: 2008/11/04

I have implemented a more accurate simulation of the cWB algorithm. In particular, the simulation generates fake data and computes the reduced correlated energy in the principle component frame. It also includes the regularization of the cross polarization, and also computes the null energy and the network correlation coefficient. Currently the only significant aspect of cWB that is not included is the penalty factor; it is not clear if this is important.

For several weeks there was a factor of about sqrt{2} amplitude discrepancy between the measured and predicted points. On the Nov 04 review telecon, Sergey realised there was a factor 2 error in the definition of the Pearson correlation coefficient given in the technical document (T080011-00-Z, Jan 18 2008). Here is the incorrect definition in the technical document:

    \begin{equation}
    r_{ij}  =  \frac{L_{ij}}{2\sqrt{L_{ii} L_{jj}}}
    \end{equation}

    \begin{equation}
    e_{c}  =  \sum_{i \ne j} L_{ij} |r_{ij}|
    \end{equation}
Comparison to the code (netevent.cc, see lines 703-724) shows that what is actually computed in cWB is

    \begin{equation}
    r_{ij}  =  \frac{L_{ij}}{\sqrt{L_{ii} L_{jj}}}
    \end{equation}

    \begin{equation}
    e_{c}  =  \sum_{i \ne j} L_{ij} |r_{ij}|
    \end{equation}
Note the difference by a factor of 2 in r_{ij}. Correcting the matlab script to use the definition of r_{ij} from the code resolved the discrepancy. The new code has very good performance for the sine-Gaussians, both for the 50% and 90% points. For Gaussians and WNBs the performance is not good; however, the present code is not designed to model broad-band signals.

MDC set: SG1 (Q9 sine-Gaussians)

Comparison of measured and predicted hrss values for cWB
    MDC set: SG1
 
                                 hmeas50                       hmeas90 
      freq   hmeas50   hpred50   -------   hmeas90   hpred90   ------- 
     (kHz)   (1e-21)   (1e-21)   hpred50   (1e-21)   (1e-21)   hpred90 
    ------    ------   -------   -------   -------   -------   ------- 
    0.0700    2.4000    2.8352    0.8465   10.5000   11.4917    0.9137
    0.1000    0.9560    1.1491    0.8319    4.2500    4.7437    0.8959
    0.1530    0.5940    0.6971    0.8522    2.6400    2.8525    0.9255
    0.2350    0.5750    0.7089    0.8111    2.4800    2.9908    0.8292
    0.3610    1.0300    1.2315    0.8364    4.4800    5.0685    0.8839
    0.5540    1.1600    1.2863    0.9018    5.0700    5.2578    0.9643
    0.8490    1.7800    1.9626    0.9070    7.9200    7.7297    1.0246
    0.9450    2.0000    2.1142    0.9460    9.2500    8.4544    1.0941
    1.0530    2.3100    2.4661    0.9367   10.2000   10.1883    1.0012
    1.1720    2.5100    2.6106    0.9615   11.5000   10.7718    1.0676
    1.3040    2.8700    2.9448    0.9746   12.7000   12.0390    1.0549
    1.4510    3.2500    3.2002    1.0156   14.4000   12.5798    1.1447
    1.6150    3.4900    3.6962    0.9442   15.7000   15.0907    1.0404
    1.7970    3.9900    4.0452    0.9864   17.9000   16.2641    1.1006
    2.0000    5.0700    4.3983    1.1527   23.0000   18.0253    1.2760
                            mean: 0.93 +/- 0.09                 1.01 +/- 0.12

(Nmc=1e4)

MDC set: SG5 (Q3 sine-Gaussians)

The agreement is similar to the Q9 case.

Comparison of measured and predicted hrss values for cWB
    MDC set: SG5
 
                                 hmeas50                       hmeas90 
      freq   hmeas50   hpred50   -------   hmeas90   hpred90   ------- 
     (kHz)   (1e-21)   (1e-21)   hpred50   (1e-21)   (1e-21)   hpred90 
    ------    ------   -------   -------   -------   -------   ------- 
    0.0700    1.9100    2.8421    0.6720    8.8700   11.3360    0.7825
    0.1000    0.9560    1.1541    0.8283    4.5700    4.7989    0.9523
    0.1530    0.6450    0.7065    0.9130    2.9600    2.8480    1.0393
    0.2350    0.6320    0.7168    0.8817    2.8700    2.9950    0.9583
    0.3610    0.9080    1.2293    0.7386    4.1600    5.0863    0.8179
    0.5540    1.2300    1.3083    0.9402    5.7400    5.2529    1.0927
    0.8490    1.9100    1.9265    0.9914    8.6900    7.6377    1.1378
    0.9450    2.1500    2.1486    1.0006    9.8400    9.0597    1.0861
    1.0530    2.3800    2.4667    0.9648   11.9000   10.1103    1.1770
    1.1720    2.7200    2.6417    1.0296   12.9000   10.9984    1.1729
    1.3040    3.0800    2.8576    1.0778   14.1000   11.7339    1.2016
    1.4510    3.4600    3.2411    1.0676   16.5000   13.2283    1.2473
    1.6150    3.9500    3.7375    1.0569   17.9000   14.4077    1.2424
    1.7970    4.5700    4.0034    1.1415   21.0000   16.7000    1.2575
    2.0000    5.3400    4.3887    1.2168   24.7000   17.6421    1.4001
                            mean: 0.97 +/- 0.15                 1.1 +/- 0.17

(Nmc=1e4)

MDC set: SG6 (Q100 sine-Gaussians)

The agreement is similar to the Q9 case.

Comparison of measured and predicted hrss values for cWB
    MDC set: SG6
 
                                 hmeas50                       hmeas90 
      freq   hmeas50   hpred50   -------   hmeas90   hpred90   ------- 
     (kHz)   (1e-21)   (1e-21)   hpred50   (1e-21)   (1e-21)   hpred90 
    ------    ------   -------   -------   -------   -------   ------- 
    0.0700    2.6400    2.8536    0.9251   11.7000   11.6465    1.0046
    0.1000    0.9970    1.1229    0.8879    4.3400    4.7279    0.9180
    0.1530    0.6120    0.7099    0.8621    2.7200    2.8516    0.9538
    0.2350    0.5810    0.7073    0.8214    2.5100    2.8944    0.8672
    0.3610    1.0200    1.2051    0.8464    4.3900    4.9669    0.8839
    0.5540    1.0600    1.3040    0.8129    4.5200    5.2950    0.8536
    0.8490    1.5200    1.9325    0.7865    6.9200    7.8943    0.8766
    0.9450    1.8000    2.1857    0.8235    7.5200    8.7266    0.8617
    1.0530    2.0800    2.4494    0.8492    8.6900   10.0953    0.8608
    1.1720    2.2800    2.5906    0.8801    9.7400   10.4998    0.9276
    1.3040    2.4300    2.8717    0.8462   10.5000   11.4668    0.9157
    1.4510    2.7500    3.2218    0.8536   11.7000   13.3915    0.8737
    1.6150    2.9600    3.7270    0.7942   12.2000   14.9998    0.8133
    1.7970    3.5600    4.0620    0.8764   15.2000   15.8400    0.9596
    2.0000    3.7900    4.3928    0.8628   16.9000   17.9179    0.9432
                            mean: 0.85 +/- 0.04                 0.9 +/- 0.05

(Nmc=1e4)

Update: 2008/11/19

I have generalized to code to be able to handle broadband waveforms. The single-pixel code is now called in a for loop over frequency bins, the total likelihood and other statistics are now obtained by summing over multiple frequency bins. This gives good agreement for Gaussians, and does not significantly alter the results for sine-Gaussians. The efficiencies for the WNBs are still in disagreement.

More precisely, for a given waveform type, the X-Pipeline simulation engine is used to create a timeseries for each polarization. These are studied to determine an approximately optimal FFT length to match the bandwidth of the signal. The timeseries are then FFTed, and compared to the spectrum for H1. All frequency bins with squared SNR greater than 1% of the highest squared SNR of any frequency bin are retained. For each remaining frequency bin, the single-pixel coherent likelihood calculations are performed. The total likelihood, rho_effective, etc., are computed as sums over the frequency bins.

Al of the following results were generated using Nmc=1e4 injections for each hrss amplitude.

MDC set: HGA2 (Gaussians)

The agreement is surprisingly good, even for the waveforms with central frequencies well out of the sensitive band (e.g., below the seismic wall).

 
Comparison of measured and predicted hrss values for cWB
    MDC set: hga2
 
                                 hmeas50                       hmeas90 
      freq   hmeas50   hpred50   -------   hmeas90   hpred90   ------- 
     (kHz)   (1e-21)   (1e-21)   hpred50   (1e-21)   (1e-21)   hpred90 
    ------    ------   -------   -------   -------   -------   ------- 
    2.5400    2.3800    2.2195    1.0723   11.6000    8.9068    1.3024
    1.2700    1.6600    1.5635    1.0618    7.7500    6.2014    1.2497
    0.5080    1.1100    1.1223    0.9890    5.0700    4.5413    1.1164
    0.2540    0.9360    0.9096    1.0290    4.3400    3.6243    1.1975
    0.1270    0.9760    1.1016    0.8860    4.5700    4.4190    1.0342
    0.0508    1.8000    2.2345    0.8055    8.4200    8.8370    0.9528
    0.0318    4.5700    5.6559    0.8080   21.2000   22.1456    0.9573
    0.0212   16.7000   18.5986    0.8979   74.2000   72.5519    1.0227
    0.0159   70.5000   44.1530    1.5967  301.0000  174.1281    1.7286
                            mean: 1.02 +/- 0.24                 1.17 +/- 0.24

MDC set: HWNB2 (WNBs)

The agreement is surprisingly similar to the single-pixel case. For a couple fo waveforms we have improved agreement, but for the most part the predicted hrss values are well above the measured ones. The discrepancy is more than a factor of 2 for the hrss90% points. This needs to be investigated.

Comparison of measured and predicted hrss values for cWB
    MDC set: hwnb2
 
                                 hmeas50                       hmeas90 
      freq   hmeas50   hpred50   -------   hmeas90   hpred90   ------- 
     (kHz)   (1e-21)   (1e-21)   hpred50   (1e-21)   (1e-21)   hpred90 
    ------    ------   -------   -------   -------   -------   ------- 
    0.1050    0.8180    1.5192    0.5385    1.9700    6.2121    0.3171
    0.1500    0.6120    0.8564    0.7146    1.4800    3.4607    0.4277
    0.1500    0.7150    0.8781    0.8143    1.5600    3.5640    0.4377
    0.2550    0.5410    0.8069    0.6705    1.2500    3.3029    0.3785
    0.3000    0.6930    1.0517    0.6589    1.6200    4.5300    0.3576
    0.3000    0.8020    1.0463    0.7665    1.6900    4.2308    0.3995
    1.0050    1.9500    2.5799    0.7558    4.6700   10.5674    0.4419
    1.0500    2.0600    2.8762    0.7162    4.6700   11.5929    0.4028
    1.5000    3.1500    4.9261    0.6395    6.9200   19.6194    0.3527
    1.0500    2.5900    2.6512    0.9769    5.4000   10.3866    0.5199
    1.5000    3.5600    4.5320    0.7855    7.3600   18.6752    0.3941
    1.5000    5.9800    5.3046    1.1273   13.9000   21.2886    0.6529
                            mean: 0.76 +/- 0.16                 0.42 +/- 0.09

MDC set: SG1 (Q9 sine-Gaussians)

The agreement is similar to the single-pixel case. The only significant difference is that the predicted hrss for the 2000Hz waveform jumps significantly, presumably because a significant fraction of the hrss falls above the high-frequency cutoff near 2048Hz. The std deviation of the measured:predicted ratios is smaller, indicating better fits overall.

Comparison of measured and predicted hrss values for cWB
    MDC set: SG1
 
                                 hmeas50                       hmeas90 
      freq   hmeas50   hpred50   -------   hmeas90   hpred90   ------- 
     (kHz)   (1e-21)   (1e-21)   hpred50   (1e-21)   (1e-21)   hpred90 
    ------    ------   -------   -------   -------   -------   ------- 
    0.0700    2.4000    2.8312    0.8477   10.5000   11.5063    0.9125
    0.1000    0.9560    1.1591    0.8248    4.2500    4.7453    0.8956
    0.1530    0.5940    0.7194    0.8257    2.6400    2.8823    0.9159
    0.2350    0.5750    0.6669    0.8622    2.4800    2.7218    0.9112
    0.3610    1.0300    1.1689    0.8811    4.4800    4.8494    0.9238
    0.5540    1.1600    1.2919    0.8979    5.0700    5.3439    0.9487
    0.8490    1.7800    1.9375    0.9187    7.9200    7.9144    1.0007
    0.9450    2.0000    2.1241    0.9416    9.2500    8.9392    1.0348
    1.0530    2.3100    2.4367    0.9480   10.2000    9.9394    1.0262
    1.1720    2.5100    2.6094    0.9619   11.5000   10.4159    1.1041
    1.3040    2.8700    2.8772    0.9975   12.7000   11.3511    1.1188
    1.4510    3.2500    3.2124    1.0117   14.4000   12.8654    1.1193
    1.6150    3.4900    3.6342    0.9603   15.7000   14.7494    1.0645
    1.7970    3.9900    4.1135    0.9700   17.9000   16.3071    1.0977
    2.0000    5.0700    6.0835    0.8334   23.0000   24.4636    0.9402
                            mean: 0.91 +/- 0.06                 1 +/- 0.08

MDC set: SG5 (Q3 sine-Gaussians)

The agreement is similar to the single-pixel case. The SG near the violin modes and the 200 Hz SG are handled better; other high-frequencies systematically a bit worse. The std deviations are the same.

Comparison of measured and predicted hrss values for cWB
    MDC set: SG5
 
                                 hmeas50                       hmeas90 
      freq   hmeas50   hpred50   -------   hmeas90   hpred90   ------- 
     (kHz)   (1e-21)   (1e-21)   hpred50   (1e-21)   (1e-21)   hpred90 
    ------    ------   -------   -------   -------   -------   ------- 
    0.0700    1.9100    2.5528    0.7482    8.8700   10.2063    0.8691
    0.1000    0.9560    1.1850    0.8067    4.5700    4.6243    0.9883
    0.1530    0.6450    0.7914    0.8150    2.9600    3.1869    0.9288
    0.2350    0.6320    0.6845    0.9233    2.8700    2.8510    1.0066
    0.3610    0.9080    0.9237    0.9830    4.1600    3.7638    1.1053
    0.5540    1.2300    1.2711    0.9677    5.7400    5.1746    1.1093
    0.8490    1.9100    1.7883    1.0680    8.6900    7.1670    1.2125
    0.9450    2.1500    2.0098    1.0698    9.8400    7.9892    1.2317
    1.0530    2.3800    2.1792    1.0922   11.9000    9.0666    1.3125
    1.1720    2.7200    2.4451    1.1124   12.9000    9.8346    1.3117
    1.3040    3.0800    2.6637    1.1563   14.1000   10.8857    1.2953
    1.4510    3.4600    2.9558    1.1706   16.5000   12.1227    1.3611
    1.6150    3.9500    3.4309    1.1513   17.9000   14.4741    1.2367
    1.7970    4.5700    4.0130    1.1388   21.0000   16.2273    1.2941
    2.0000    5.3400    5.0039    1.0672   24.7000   19.9351    1.2390
                            mean: 1.02 +/- 0.14                 1.17 +/- 0.16

MDC set: SG6 (Q100 sine-Gaussians)

The agreement is similar to the single-pixel case. The 2 lowest frequencies and the violin mode SG are worse; the others are about the same. Note that the broadband code is not expected to handle the Q100 SGs very well because it cannot use FFT lengths longer than 1/8sec, which is too short for the lowest-frequency Q100 waveforms.

Comparison of measured and predicted hrss values for cWB
    MDC set: sg6
 
                                 hmeas50                       hmeas90 
      freq   hmeas50   hpred50   -------   hmeas90   hpred90   ------- 
     (kHz)   (1e-21)   (1e-21)   hpred50   (1e-21)   (1e-21)   hpred90 
    ------    ------   -------   -------   -------   -------   ------- 
    0.0700    2.6400    2.5577    1.0322   11.7000   10.1125    1.1570
    0.1000    0.9970    1.3172    0.7569    4.3400    5.5448    0.7827
    0.1530    0.6120    0.7128    0.8585    2.7200    2.8340    0.9598
    0.2350    0.5810    0.7026    0.8270    2.5100    2.9256    0.8579
    0.3610    1.0200    1.2592    0.8100    4.3900    5.2970    0.8288
    0.5540    1.0600    1.3043    0.8127    4.5200    5.3669    0.8422
    0.8490    1.5200    1.9253    0.7895    6.9200    7.8865    0.8775
    0.9450    1.8000    2.1787    0.8262    7.5200    8.7808    0.8564
    1.0530    2.0800    2.5132    0.8276    8.6900   10.2355    0.8490
    1.1720    2.2800    2.6554    0.8586    9.7400   10.6875    0.9113
    1.3040    2.4300    2.9206    0.8320   10.5000   11.4139    0.9199
    1.4510    2.7500    3.2147    0.8554   11.7000   12.9588    0.9029
    1.6150    2.9600    3.6214    0.8174   12.2000   14.9797    0.8144
    1.7970    3.5600    3.9571    0.8996   15.2000   15.7870    0.9628
    2.0000    3.7900    4.5257    0.8374   16.9000   18.0727    0.9351
                            mean: 0.84 +/- 0.06                 0.9 +/- 0.09

Discussion

The agreement between measured and predicted efficiencies for the linearly polarized waveforms (SGs, Gaussians) is quite good - around the 10% level. There are clearly some systematics in the discrepancies. For example, the predicted values tend to be slightly too high at low frequencies and slightly too high and high frequencies (too high at all frequencies for the Q=100 case).

Sergei has pointed out that the predicted values tend to be relatively higher near the violin modes. He suggests that cWB benefits from the use of LPEF to ameliorate the effects of the lines. The matlab model does not simulate this. The benefit looks smallest for the Q100 case, which makes sense: a Q100 sine-Gaussian at 361 Hz should be relatively less affected by the violin modes at 340-350 Hz.

To-Do Tasks

  1. Explore what's going wrong with the WNBs. Could it be connected with the penalty factor?
  2. Correct the low-frequency thresholds for the ~10% livetime when higher thresholds were used. (THis should have a small effect.)
  3. Re-run for the H1-H2, H1-L1, and H2-L1 networks.
$Id: hrss_comparisons.html,v 1.18 2008/11/19 22:38:47 psutton Exp $