Search


DASWG LAL Doxygen

Docs

How-To's
Technical
Software Docs
Minutes

Download

Browse CVS, Git, or SVN
Software Repositories
OS Security Updates
LIGO software virtual machine
VMware SL6 Install

Participate

SCCB - Software Change Control Board
Edit these pages
Sub-committees
Mailing List
Telecon

Projects

DMT
DQSEGDB
Glue
GraceDB
gstlal
LALSuite
LDAS Tools
LDG Client/Server
LDR
ligoDV
LIGOtools
LVAlert Administration
LVAlert
MatApps
Metaio
NDS Client
PyLAL
LSCSOFT VM

Legacy Projects

geopp
LDAS
LDM
LIGOtools
LSCGIS
Onasys
OSG-LIGO

E11 FAQ

Answers to frequently asked question about E11 online analysis and access to data.

1. How can I access the data?
2. How can I find science-mode segments?
3. How can I get science-mode segments with data quality cuts?
4. How can I get real-time online science-mode segments from the command line?
5. OK, given science-mode, how do I find the data?
6. How can I calibrate the data?
7. Where should output from online analysis go for viewing in the control room?

More about E11 is here: http://blue.ligo-wa.caltech.edu/engrun/E11/index.html

More about online analysis in general is here: https://www.lsc-group.phys.uwm.edu/daswg/projects/online.html

The LSC Data Analysis Software Working Group (DASWG) home page is here: https://www.lsc-group.phys.uwm.edu/daswg/

1. The quickest access beyond the control room is from the local cluster at the site.
To use the cluster at LHO or LLO sign-up for a grid account at this URL:

http://ligo.aset.psu.edu/users/account.shtml

This web page also allows you to obtain access to other LSC clusters.

(back)

2. How to find science-mode segments:

Each run has a web page with links to science-mode segments.  
See the links on this page:

http://blue.ligo-wa.caltech.edu/engrun/E11/index.html

For a science run, replace "engrun" with "scirun" and the run name.

*** CAVEAT *** : Real-time science-mode segments do not contain data
quality cuts!  

(back)

3. How to get science-mode segments with data quality cuts:

Go to web page set up by Keith Riles and Peter Shawhan, and/or use
segwizard from the LIGOtools segments package, to get segments with
various cuts made.  For example, for E11 go to this page:

http://tenaya.physics.lsa.umich.edu/~keithr/E11DQ/

Note that data quality cuts are made after a run ends, and are updated
several times.

(back)

4. Command line real-time online science-mode segments:

LSCsegFind is a command line tool that works like LSCdataFind, but
returns science-mode or locked segments.  

Like LSCdataFind, it has a "--show-types" options which shows which
types of segments are avaiable:
 
$ LSCsegFind --server=ldas-gridmon.ligo-wa.caltech.edu --show-types
                                                                                                                            
 Available types : description
           E11H1 : Conlog Science-mode Segments
           E11H2 : Conlog Science-mode Segments
           H1DMT : DMT Science-mode Segments
        H1DMT_CM : DMT Common-mode Segments
           H2DMT : DMT Science-mode Segments
        H2DMT_CM : DMT Common-mode Segments
            M5H1 : Conlog Science-mode Segments
            M5H2 : Conlog Science-mode Segments
            S3H1 : Conlog Science-mode Segments
            S3H2 : Conlog Science-mode Segments
            S3L1 : Conlog Science-mode Segments

Here is an example return of E11 science-mode segments:

$ LSCsegFind --server=ldas-gridmon.ligo-wa.caltech.edu
--gps-start-time=784897000 --gps-end-time=784899000 --type=E11H1
--output-format="%i %s %e %d"
15 784897898 784898276 378
16 784898283 784898924 641

*** CAVEAT ***: Currently LSCsegFind cannot handle data-quality cuts.
Since LSCsegFind currently just fetches segments from web pages, like
segwizard, eventually it could become a command line version of
segwizard, and obtain segments with such cuts.

Note that online analysis tools are being gathered in a new lscsoft
project called "glue".  This is a work in progress, but some of its
contents are being used for E11. To get glue, checkout this cvs
repository:
CVS/Root = :pserver:USERNAME@gravity.phys.uwm.edu:2402/usr/local/cvs/lscsoft,
CVS/Repository = glue

(back)

5. OK, given science-mode; how do you find the data?

i) Data gets published and moved around by LDR.  During E11
RDS data was sent to CIT.  This happened in near
real-time (though because of a few problems the transfer of
L3 RDS data did fall behind).  Talk to the local sys admin to find out
if data is being transferred to your favorite cluster via LDR.

ii) Use LSCdataFind to find the data.  

*** CAVEAT ***: Even if the data is coming to a site via LDR,
LSCdataFind cannot necessarily find the data until the data has been
published into a location available to the local cluster.  Ask the local
sys admin, if you want to know the availability of data.

LHO, LLO, and CIT have LSCdataFind installed and ready to use at login.

(Note that LSCdataFind is now also distributed with "glue", but only
works where globus pythons libraries have been set up, which is true on
the above clusters.  Most other LSC clusters have it set up
somewhere too.)

Note that LSCdataFind can be run from anywhere it is installed;
the files returned depend on which server is queried.

For example,  to find files at LHO, query the server ldas-gridmon.ligo-wa.caltech.edu

$ LSCdataFind --server=ldas-gridmon.ligo-wa.caltech.edu
--gps-start-time=784897000 --gps-end-time=784899000 --type=RDS_R_L3
--observatory=H
file://localhost/data/node45/frame/E11/L3/LHO/H-RDS_R_L3-784896896-256.gwf
file://localhost/data/node46/frame/E11/L3/LHO/H-RDS_R_L3-784897152-256.gwf
file://localhost/data/node47/frame/E11/L3/LHO/H-RDS_R_L3-784897408-256.gwf
file://localhost/data/node48/frame/E11/L3/LHO/H-RDS_R_L3-784897664-256.gwf
file://localhost/data/node49/frame/E11/L3/LHO/H-RDS_R_L3-784897920-256.gwf
file://localhost/data/node50/frame/E11/L3/LHO/H-RDS_R_L3-784898176-256.gwf
file://localhost/data/node51/frame/E11/L3/LHO/H-RDS_R_L3-784898432-256.gwf
file://localhost/data/node52/frame/E11/L3/LHO/H-RDS_R_L3-784898688-256.gwf
file://localhost/data/node53/frame/E11/L3/LHO/H-RDS_R_L3-784898944-256.gwf

For example,  to find files at CIT, query the server ldas-gridmon.ligo.caltech.edu:

$ LSCdataFind --server=ldas-gridmon.ligo.caltech.edu
--gps-start-time=783897000 --gps-end-time=786899000 --type=RDS_R_L3
--observatory=H
...

If this returns "No URLs found!" then E11 data is not yet available to the cluster at CIT (even though the
data is being transferred there).  The data should be available at CIT soon...

Part of the purpose of E11 is to test the pipelines before S4.  Data should be available in near-real time
at CIT during S4.

(back)

6. How can I calibrate the data?

There was not a well established way to do this for E11.  

The inspiral and burst group did run online during E11 and may be able to provide an answer.

Also, for offline calibration check the Calibration Home Page: http://blue.ligo-wa.caltech.edu/engrun/Calib_Home/

(back)

7. Where should output from online analysis go for viewing in the control room?

Before E11 at LHO a filesystem called /cluster was created for the output of online analysis jobs run on the cluster.

Directories were created under /cluster for the inspiral and burst groups.   These groups and John Zweizig can
provide information on what sort of output can be viewed from the control room.

(back)
$Id$