This comprehensive workshop will provide participants with all the basic knowledge and skills required in order to take advantage of the computational resources provided by the LSC DataGrid.

Through a series of presentations, live demos, and hands-on exercises, attendees will become comfortable working with the LSC DataGrid. Participants will gain first hand experience in job execution, data movement, and security.

Schedule

This document describes the first 2 days. The final ½ day will consist of additional topics described in a separate document.

Audience

Prerequisites

Format

Materials

An archive of the workshop materials, including presentations and labs, is available here.

Setup Requirements IMPORTANT

Alternate Setup. If you already have a GC (General Computing) account on an existing linux machine at LLO, you may use that machine as your LSC DataGrid client. The setup instructions should be followed on that machine as described above. Note that it is still necessary to bring a laptop to the workshop, but only for SSHing to your LSC DataGrid client machine.

Agenda

	Presentations	Lab Exercises
Introduction
45 m	Part 1 - Introduction Welcome & Introductions Review content and format of the workshop Overview of the LSC DataGrid – describe site hardware/software, current status, head node DNS names VDT, GriPhyn/iVDGL Globus, pyGlobus Quick job submission demo??
45 m		Lab 1 - Getting Started Verify network connectivity Setup environment Initialize a proxy certificate Update client-side `.bash_profile`
Security
45 m	Part 2 - Grid Security Introduction to grid security concepts and basic user authentication Discuss PKI, certificates, Certificate Authorities, etc. DOEgrid CA Certificate management Gsi-ssh Proxy certificates The grid-mapfile Software components used by LSC Data Grid Other Related software
60 m		Lab 2 - Security Displaying certificate information Working with proxy certificates “SSHing” without passwords! Update server-side `.bash_profile`
Data Management
45 m	Part 3 - Data Management Discuss data resources on a grid Explain the role of data movement in staging job executions Describe the basic grid technologies involved in moving data around a grid FTP, SCP GridFTP, UberFTP globus-url-copy Software components used by LSC Data Grid Other Related software: RFT
90 m		Lab 3 - Data Management Using SCP to move files around Using globus-url-copy to transfer files between the local machine and a grid server Using globus-url-copy to transfer files between two remote grid servers Using UberFTP to transfer files between the local machine and a grid server Using UberFTP to transfer files between two remote grid servers
45 m	Part 4 - LSC DataGrid Discuss the role of data replication in a grid environment Explain why replication can improve performance, throughput, and scalability of distributed applications Introduce the various approaches for implementing data replication, including Storage Request Broker (SRB), Replica Location Service (RLS) and Lightweight Data Replicator (LDR) LDR (only an intro here – more detail during last ½ day) LSCdataFind
45 m		Lab 4 - LSCdataFind Checking the LSCdataFind Server Showing Observatories and Types Searching for File Names
Break at end of day Thursday
Job Management
45 m	Part 5 - globusrun Explore the unique issues encountered when submitting jobs for execution in a grid environment globusrun & RLS Software components used by LSC Data Grid Other Related software
60 m		Lab 5 - globusrun globus-job-run globus-job-submit globusrun & RSL Staging with globusrun & RSL
30 m	Part 6 - (local) Condor Condor Discuss monitoring, canceling, recovery of failed jobs
60 m		Lab6 - Condor SSH into the Server Displaying Condor Information Submitting Local Condor Jobs Single Job Submission with Requirements Diagnosing & Restarting Non-Running Jobs
30 m	Part 7 - Condor-G Condor-G
60 m		Lab 7 - CondorG Configuring and Starting Condor Displaying Condor Information Single Job Submission Multiple Job Submission Multiple Job Submission using Separate Directories Diagnosing & Releasing Held Jobs Shutting Down Condor
Workflow Management
45 m	Part 8 - DAGman Define the role of meta-schedulers and workflow managers, and why we need jobs to submit other jobs DAGMan
90 m		Lab 8 - DAGMan Running Simple DAGMan Jobs More Complex DAGMan Jobs Recovering Failed DAGMan Jobs
15 m	Pegasus OGRE/GridAnt Software components used by LSC Data Grid Other Related software
Coming Attractions!
30 m	Part 9 - MyProxy MyProxy Credential Repository
45 m		Lab 9 - MyProxy Storing a Credential Retrieving a Credential Destroying a Credential Credential Renewal Condor-G Automatic Credential Renewal
15 m	VOMS, CAS, or EDG mkgridmap for improved management of grid-mapfile & role-based authorization
Break at end of day Friday
On-line Analysis
20 m	Part 10 - Glue and the LSCdataGrid Introduction to Glue Finding segments with LSCsegFind How it all fits together
25 m		Lab 10 - LSCsegFind Installing Glue Checking the LSCsegFind Server Showing Interferometers and Types Searching for Segments
90 m	Part 11 - Onasys Installing Onasys A tour of the example on-line excess power search A tour of the job monitoring web pages Perform an on-line analysis on live data
Automating Complex Work-flow Construction
45 m		Lab 12 - Constructing Pipelines with Glue An introduction to pipeline.py

Grid Workshop for LSC Users