Data Formats sub-committee
Date: Tue 24 Mar 2004 04:34:40 PM CST
From: Patrick Brady
To: Bruce Allen, Stuart Anderson (chair), Albert Lazzarini, John Whelan and John Zweizig.
Subject: Data formats for archival and exchange
The DASWG charge covers long-term planning for software used in data
analysis leading to observational publications of the LSC. As a
community with the long-term goal of gravitational wave astronomy,
the nature of our archival data products is important to allow easy,
correct and continued access to the data.
I would ask the four of you to produce a brief report on the
issue of LSC data formats for archival and exchange. In preparing the
report, I would ask you to address the following:
1) Suggest criteria to distinguish intermediate and
archival/exchange data products.
2) Estimate the level of FTE support required by the
addition of a new archival/exchange data format. For example,
cost for infrastructure modifications to manage the new data
3) Suggest the requirements for documentation and software
tools for reading and writing any adopted archival/exchange
4) Suggest criteria which should be satisfied to elevate an
intermediate product to the level of archival/exchange product.
5) Comment on the suggestion that a Change Control
Board should be (re-)constituted to approve changes to data
format specifications and implementations.
6) Comment on the SFT format in use by the Pulsar Analysis
Group in the context of your responses to the suggestions
7) Comment on the sufficiency of XML-based LIGO lightweight
as a format for the archival/exchange of trigger-based
Other sub-committees have found it useful to circulate a questionaire
to the LSC in order to access the relevant information. As you
develop this report, please keep in mind the long-term view of LSC
data analysis as an enterprise which will continue through the end of
the decade and into the next.
I would like to post this report for the DASWG before 7 April 2004.
This accelerated schedule is intended to both reduce your workload and
to maintain the momentum of the DASWG in these difficult issues.
Whatever the outcome of discussions within the DASWG, I plan to
reconsider all questions relating to software and data formats
starting in early 2005 when I hope we can start strategic planning for
Advanced LIGO data analysis.