Subject: [DASWG] Questionnaire response for GDS trigger table modification From: John Zweizig Date: Wed, 5 May 2004 08:56:05 -0700 (PDT) This questionnaire is intended to provide a starting point for discussing a proposed change to the LIGO database tables, and to bring forth all of the issues of coordination, etc. Please fill in the answers and circulate it for comment. If you are not sure about any of the answers, please try to find an appropriate person to ask. 1. Who is making this proposal? (Indiate individual(s) and/or working group(s)) John Zweizig with input from the DetChar Glitch subgroup 2. What do you propose to do? (Check one or more) [X] Modify one or more existing database table(s) [ ] Add one or more new database table(s) [ ] Other 3. Briefly describe your proposal. (Examples: "Add two columns to the sngl_burst table"; "Create a table called sngl_pings to store triggers from the 'pings' DSO") Add more columns and better define existing columns of the gds_trigger table to avoid ambiguous use of fields. 4. Explain why the proposed change is necessary. (Examples: "Need to be able to store the time of the peak filter output"; "The 'pings' DSO calculates several interesting event parameters which do not fit into the sngl_burst table") At present there is insufficient data to completely characterize a glitch event and existing fields (columns) are used ambiguously. The new table has better defined fields that can be filled unambiguously from all trigger generators (t-domain, f-domain, t-f, wavelet, etc.) 5. Does the proposal require doing some administrative operation on the DB2 database servers? (Hint: the answer is 'yes' unless the relevant table is never intended to be stored in an LDAS database.) [X] Yes [ ] No 6. If the proposal is to modify an existing LDAS database table, does the necessary DB2 administrative operation preserve existing data in the table? [X] Yes, because the proposal is to add one or more columns [ ] Yes, because of some other reason: _________________ [ ] No, but the table has no data in it yet, so it can safely be deleted and recreated [ ] No, there is existing data which will need special handling [ ] Not applicable 7. Does the proposal require a synchronized change to SQL schema files used by LDAS? (Hint: the answer is 'yes' if the proposal is to modify or add a table in the LDAS database, except for certain administrative operations such as creating or deleting an index) [X] Yes [ ] No 8. Does the proposal require changing LDAS C++ and/or Tcl source code? (Hint: the answer is 'no' if the proposal just modifies or adds a table in the LDAS database, but could conceivably be 'yes' if an architectural change to the general table design is being proposed.) [ ] Yes [X] No 9. Does the proposal require changing LAL? (Certain tables are represented as data structures in LAL) [ ] Yes, and the change must be released synchronously [ ] Yes, but the change does not need to be synchronized [X] No 10. Does the proposal require changing guild? (Guild can display arbitrary table contents, but must be modified if you want to be able to query new tables or select rows from existing tables based on values in newly-added columns) [X] Yes [ ] No 11. What is the anticipated data volume (e.g., number of rows per interferometer per month of LIGO running time) for the new or modified table(s)? Of order 1 row per second. 12. Add comments or additional information (e.g., exact names(s) or columns to be added, exact definition of the proposed new table, etc.) here as appropriate: ------------------------------------------------------------------------------ Existing Fields ---------------------------------------------------------------------------- Name="CREATOR_DB" Type="int_4s" Creator data base ID - Defined during insertion Name="PROCESS_ID" Type="char_u" *** Required field Unique Identifier of the process that created this trigger. Name="FILTER_ID" Type="char_u" Undefined = NULL Filter ID - Not used Name="NAME" Type="string" *** Required field Trigger class name - This name is specified by the monitor that generates the trigger and is often based on the monitor name, e.g. glitchMon. Name="SUBTYPE" Type="string" *** Required field Trigger subtype for monitors that generate more than one type of trigger or operate on multiple channels. This name is often a channel name. Name="IFO" Type="string" *** Required field IFO(s) that the trigger applies to Name="PRIORITY" Type="int_4s" *** Required field An integer priority indicating the importance of the trigger. The TrigBase class assigns the following numeric values, information=0, warning=1, error=2, severe error=3. Name="DISPOSITION" Type="int_4s" *** Required field Bit mask indicating routing for the trigger, production database = 0x01, operator warning = 0x02, test database = 0x04/ Name="BINARYDATA" Type="char_u[1024]" Uninitialized value: "" Additional data stored by the trigger generator. Name="BINARYDATA_LENGTH" Type="int_4s" Uninitialized value: 0 Length of the additional data. A zero value indicates no binary data Name="EVENT_ID" Type="char_u" Supplied by DataBase Unique event ID. -------------------------------------------------------------------------- Existing fields redefined or more fully defined -------------------------------------------------------------------------- Name="START_TIME" Type="int_4s" *** Required field, not zero Name="START_TIME_NS" Type="int_4s" *** Required field GPS seconds and nanoseconds for the start time. Start of a (t,F) tile containing >90% of the signal in f-domain. Sample before significant Signal energy in the t-domain. Name="DURATION" Type="real_4" *** Required field, not zero Duration (tEnd-tStart) in seconds (of the (t,F) tile for f-domain). Name="SIZE" Type="real_4" Unitialized Value: 0 Peak amplitude of the triggered effect (i.e. filtered signal amplitude at time tPeak). May be omitted (set to zero) in cases where this can't be measured (F-domain or wavelets), see significance. Name="SIGNIFICANCE" Type="real_4" Unitialized Value: 0 Number of sigmas represented by the peak amplitude (i.e. SIZE divided by the sigma of the noise the in the appropriate frequency band). Zero indicates that the significance is unknown. Name="FREQUENCY" Type="real_4" Unitialized Value: 0 Low frequency of the (t, F) tile or prefilter response. Name="BANDWIDTH" Type="real_4" Unitialized Value: 0 Width (fmax-fmin) of the (t,F) tile or t-domain filter response. BANDWIDTH=0 indicates that the frequency limits are not known. -------------------------------------------------------------------------- Newly defined fields -------------------------------------------------------------------------- Name="TIME_PEAK" Type="real_4" Null? Offset from the start time to center of the peak amplitude pixel (peak sample time in time-domain monitors). Name="TIME_AVERAGE" Type="real_4" Null? The signal power weighted average time in seconds relative to the start time, e.g., in frequency or wavelet domain monitors: tAvg = Sum(i; (P_i - B_i)*(t_i - tStart)) / Sum(i; P_i - B_i) where i is a pixel(wavelet) index and P_i, B_i, and t_i are the Total Power, expected Background noise power and average time of the ith pixel. Obviously, only pixels with P_i > B_i should be included in the sum. In time domain monitors, individual samples are used instead of pixels. Name="TIME_SIGMA" Type="real_4" Unitialized Value: 0 Gaussian duration estimate of the signal in seconds, i.e. tSig = Sum(i; (P_i - B_i)*(t_i - tAvg)^2) / Sum(i; P_i - B_i) Note that the pixel time granularity is ignored here. This is may be reviewed. especially if some algorithm is created with a single time pixel. Name="FREQ_PEAK" Type="real_4" Unitialized Value: 0 Center frequency of peak amplitude pixel (0.0 in time-domain monitors?). Name="FREQ_AVERAGE" Type="real_4" Unitialized Value: 0 Frequency average of the signal power, i.e. fAvg = Sum(i; (P_i - B_i)*(f_i) / Sum(i; P_i - B_i) where i is a pixel index and P_i, B_i, and f_i are the Power, Background noise power and average frequency of the ith pixel. Name="FREQ_SIGMA" Type="real_4" Unitialized Value: 0 Gaussian bandwidth estimate of the signal, i.e. fSig = Sum(i; (P_i - B_i)*(f_i-fAvg)^2) / Sum(i; P_i - B_i) Note that trigger algorithms that use a single frequency bin (i.e. time domain algorithms) would by construction have fSig=0. Name="NOISE_POWER" Type="real_4" Unitialized Value: 0 Total expected noise power summed over all pixels used in the Gaussian estimate, e.g. Pnoise = Sum(i; B_i) Name="SIGNAL_POWER" Type="real_4" Unitialized Value: 0 Total signal power minus the background power used in the Gaussian estimate, i.e. Psignal = Sum(i; P_i - B_i) Name="PIXEL COUNT" Type="int_4" Unitialized Value: 0 Number of Pixels used in the Gaussian estimate, i.e. Npix = Sum(i; 1) Name="CONFIDENCE" Type = real_4 Unitialized Value: 0 Negative Log of the probability that the measured signal would be caused by expected gaussian noise. A zero confidence indicates that configdence is unknown.