LSC Data Analysis Software Working Groups

Agenda: 16/17 Aug 2005 at LSC Meeting

Tuesday:

  13:00-15:00 - Infrastructure (DMT, Glue, LDAS, LDR, LIGOTools,
        Onasys) preparations for S5. This session will be a working
        meeting.

  15:30-17:00 - How to find documentation for the LSC Data Grid
        and LSC Data Analysis Software.  Five presentations on
        LSC Software and Computing

                General Intro and LSCSoft CVS Archive HOWTO (Brady):
                explains how to access and use the LSCSoft CVS archive

                LSC DataGrid HOWTO (Koranda): explains how to gain
                access to LSC computing/data resources for large scale
                data analysis

                LAL and LALApps installation HOWTO (Creighton/Brown):
                explains how to install the LAL and LALApps data
                analysis software

                LSC MatApps HOWTO (McNabb): explains how to setup
                environment variables and paths for use with the
                MatApps repository, and how to make use of the
                Makefile scheme

                Doxygen Documentation of LAL (Prix):  the online
                documentation of LAL generated with Doxygen

Wednesday

        09:30-12:00 - Astrophysical Software (DMT, LAL, LALApps,
        MatApps) preparations for S5.  This will be a round table
        planning session to understand interdependencies and make sure
        we know what we need and what we can achieve.

Minutes

Attendance:
	Stuart Anderson, Warren Anderson, Junwei Cao, Ben Johnson, Scott
	Koranda, John McNabb, Peter Shawhan, John Zweizig

Reviewed status of new segments database:
	- table design is done
	- documentation is in expected location
	- must be in place by Oct 1st
	- client and server can be udpated independently
	- server must be udpated before modifications to publishing scripts
	- agreed to schedule software installations in advance so others
	  can keep an eye out for breakage.

DMT's interface to new segments database:
	- work is mostly complete on the specification for the interface
	  between DMT and the database

Deadline:
	- mid september both DMT and the publishing system will be able to
	  talk to the new segments database for real
	- no later than end of august there will be a (manual) test of the
	  DMT/segments interface via exchange of an XML file

LDR:
	- 0.7.0 wants to be released by end of august
	- following will be a push to add as many pieces as possible prior
	  to S5
	- biggest concern was scalability of MySQL, but tests have shown
	  that new setup is able to handle 18 months of full-bore data
	  production (segments, astrophysical searches, frame data, etc.)
	- Patrick:  it would've been nice if there could've been by now an
	  implementation of Guild that talks to the segfind and datafind
	  servers via Glue.
	- John McNabb:  there might be a PSU grad student who could put
	  this together quickly.
	- Patrick:  if it's put together, release it within DASWG first for
	  small-scale testing before a collaboration-wide hit to the
	  servers.
	- Peter had thought he would be able to write it but is quite busy,
	  says it's on his to-do list but has been there for a month.
	- should do a segfind scaling test by mid september
	- Duncan asked about replication of segments database to other
	  clusters
	- Scott says to engage Brian on this

LDAS:
	- there will be one more version before S5 for conversion to
	  Solaris 10, FC4 (maybe), and integrating Tcl/Globus.

Onasys:
	- will check if sqlobject is available for FC3/4 in extras

Tcl/Globus:
	- nothing to report

LDG:
	- working stabily for FC3, FC4 (modulo MySQL?)
	- VDT 1.3.7 includes GT4, to be released hopefully in next couple
	  of weeks.
	- Scott: very impressed with GT4, worth trying to push out the door
	  before S5 if at all possible.
	- Impacts any existing code?  Primary impact will be that
	  LSCdataFind, segFind, etc., need to use it but should work
	  properly.  There might be some of the usual small bugs that are
	  found in any change like this but they haven't been a big deal in
	  the past.

OS:
	- a move to FC4 prior to S5 is desirable
	- by 21st of September, will ask the CompCom to agree to the
	  change to FC4
	- Patrick will e-mail all the upper limit chairs about the stat(2)
	  issue

DASWG mailing list:
	- some people would like a daswg and a daswg-help
	- maybe kill lal-discuss and move to daswg-help

Metadata:
	- Patrick favours rsync for data product transfering
	- What's missing for publishing:  nervous about giving uneducated
	  users the freedom to publish because they can't be trusted to
	  follow the rules (eg. one file per hour)
	- Ben recommends a publishing server that users send publish
	  requests to and that knows about per user limits, etc..

A wishlist for S5:
	- 1 = pre-S5, 2 = mid-S5, 3 = later
2/3	- a metadata service.
2	- a publication service.
2	- LSCdataFind GUI
1	- Keith's GUI (two-way segwizard-like thingy)
1	- get segwizard to use segfind database
1	- segfind replication
?	- replicate level 3 frames between observatories (GEO?)
	- run Onasys at Hanover/AEI for online burst/inspiral search
1	- Onasys web interface safety
1	- online h(t) generation
1	- h(t) publication
1	- released software bundle
1/2	- coincidence under Onasys

Software releases for S5:
	- LAL and LALApps will be released for S5
	- Onasys and Glue will have releases
	- MatApps will not have a release
	- libmetaio needs an update
	- libframe if one comes
	- pylal?

Computer resources:
	- Patrick will engage UL chairs and CompCom for a dialog on needs
	- More thinking will be done about what options are available re.
	  Condor configurations, etc.

$Id: 050816.html,v 1.6 2005/08/18 18:32:00 kipp Exp $