Checklist to Begin a Review
Checklist to Begin a ReviewAn analysis project will trigger the establishment of a dedicated review team when the following criteria are met and the burst group formally requests a review.
- A clear statement of the purpose of this project and the targets.
- The search must be a mature end-to-end analysis. This does not necessarily mean that the on-source analysis has been opened or that a the final pipeline tuning is completed, but all parts of the analysis, including the way the result is expected to be presented, should be in place.
- Complete and up-to-date technical documentation of the search including the software used in the search. Software, including pre- and post-processing scripts, must be available in a CVS, SVN, or similar archive available to (and usable by) the reviewers. Data products should also be available. The project page in the burst group webpages should be up-to-date and contain the webpage.
- A formal request to the burst review committee from the burst group for review. The burst group will consider the importance of the analysis to the group's scientific agenda, the exposure that the analysis has within the burst group, and the comfort of the group that the analysis is free of conceptual flaws.
Goals of the Review ProcessThe scientific results, the software used to produce them, the methodology of the search, any technical material documenting the analysis as well as the proposed publication itself must receive attention. In particular, the goal of the review process is to ensure:
- the correctness and meaningfulness of the reported result
- the early detection of errors
- that the results are reproducible
- that the analysis methods are well documented
- that the software is correct and well documented
- that the comments from the Collaborations have been appropriately addressed.
Notes(From an email message by Jolien on June 4, 2007)
An "analysis project" is a subset (not necessarily a proper subset) of a single paper to be published. It is to be a full pipeline taking segments of data through to a specific result. To be explicit, I regard the WaveBurst + CorrPower of the S4 LIGO-Only Untriggered Search to be a single analysis project. The S4 LIGO-GEO Untriggered Search contains two analysis projects: the WaveBurst + CorrPower project and the Coherent WaveBurst project. If the same code is used in a future analysis the review of the analysis project will need to be updated.
The burst review teams will in some ways replace the current software review teams. These teams will be responsible for the entire review and so they will be able to view the software in context of the whole search. The burst group may wish to continue internal software reviewing and this will be helpful to the review teams, but the teams will also need to conduct code reviews.