Administering the LSC segment database
Renewing service certificates
The service certificates used by the servers LSCsegFindServer and LDBDServer are valid for one year. The DOE will send a reminder 30 days before the certificate is due to expire and a new certificate should be requested.
The service certificates are stores in the directories
/export/ldbd/etc/grid-security/ldbdserver/ /export/ldbd/etc/grid-security/lscsegfindserver/To renew the certificate, request a new service certificate, but make sure the request does not overwrite the existing certificate or key files. As the user ldbd, run the commands
grid-cert-request -host ldas-cit.ligo.caltech.edu -dir /usr1/ldbd -ca 1c3f2ca8 -service ldbd grid-cert-request -host ldas-cit.ligo.caltech.edu -dir /usr1/ldbd -ca 1c3f2ca8 -service lscsegfindreplacing the value passed to the -host option with the correct host name.
This will create four files called
/usr1/ldbd/ldbdcert_request.pem /usr1/ldbd/ldbdkey.pem /usr1/ldbd/lscsegfindcert_request.pem /usr1/ldbd/lscsegfindkey.pem
Mail the two files named ldbdcert_request.pem and lscsegfindcert_request.pem to Phil Ehrens (or your CA with responsibility for for signing host and service certificates). He will mail you back two files called ldbdcert.pem and lscsegfindcert.pem.
Once you have the certificate file, then move the new certificates into the grid security directories.
mv /usr1/ldbd/ldbdcert.pem /export/ldbd/etc/grid-security/ldbdserver/ /usr1/ldbd/ldbdkey.pem /export/ldbd/etc/grid-security/ldbdserver/ mv /usr1/lscsegfind/lscsegfindcert.pem /export/lscsegfind/etc/grid-security/lscsegfindserver/ /usr1/lscsegfind/lscsegfindkey.pem /export/lscsegfind/etc/grid-security/lscsegfindserver/If there are existing certificate files in these directories, overwrite them with the new ones.
Make sure the permissions on the grid certificates and keys are correct by running
chmod 644 /export/ldbd/etc/grid-security/ldbdserver/ldbdcert.pem chmod 644 /export/ldbd/etc/grid-security/lscsegfindserver/lscsegfindcert.pem chmod 600 /export/ldbd/etc/grid-security/ldbdserver/ldbdkey.pem chmod 600 /export/ldbd/etc/grid-security/lscsegfindserver/lscsegfindkey.pem
Notice that WebSphere uses this ldbd certificate stored in IBM format key database created for each queue.
Also notice that when this key database is created, one specifies its expiration date.
Therefore, WebSphere might stop working when either ldbd certificate or two globus certificates (on which ldbd certificate depends) or key database for a queue expire.
The error logs for queue manager is in
- /var/mqm/qmgrs/QM1/errors at LHO,
- /var/mqm/qmgrs/QM2/errors at LLO,
- /var/mqm/qmgrs/QM3/errors at CIT.
The error message might look, for example, as follows:
------------------------------------------------------------------------ -------
11/13/06 08:38:38
AMQ9652: The remote SSL certificate has expired.
EXPLANATION:
The SSL certificate used by MQ on the remote end of the channel has expired.
The channel is 'QM1_TO_QM3'; in some cases its name cannot be determined and so
is shown as '????'. The channel did not start.
ACTION:
Use your key management tool to provide MQ with a current SSL certificate on
the remote end of the channel. Restart the channel.
----- amqccisx.c : 1013 -------------------------------------------------------
11/13/06 08:38:38
AMQ9999: Channel program ended abnormally.
EXPLANATION:
Channel program 'QM1_TO_QM3' ended abnormally.
ACTION:
Look at previous error messages for channel program 'QM1_TO_QM3' in the error
files to determine the cause of the failure.
The easiest thing to do in this case is to backup and delete the old key database, create the new one for each queue and import there two globus and one ldbd certificates.
- Shutdown all replication and message queues.
- Make sure that all the three certificates are current using with
grid-cert-info -f /export/ldbd/etc/grid-security/ldbdserver/ldbdcert.pem
grid-cert-info -f /usr1/ldbd/ldg/globus/share/certificates/d1b603c3.0
grid-cert-info -f /usr1/ldbd/ldg/globus/share/certificates/1c3f2ca8.0
- Backup and remove key database for each queue.
- For LHO:
tar cvf ~/ssl.tar /var/mqm/qmgrs/QM1/ssl
rm /var/mqm/qmgrs/QM1/ssl/*
- For LLO:
tar cvf ~/ssl.tar /var/mqm/qmgrs/QM2/ssl
rm /var/mqm/qmgrs/QM2/ssl/*
- For CIT:
tar cvf ~/ssl.tar /var/mqm/qmgrs/QM3/ssl
rm /var/mqm/qmgrs/QM3/ssl/*
- For LHO:
- Create a new key database and install globus and ldbd certificates there.
export JAVA_HOME=/opt/mqm/ssl
- For LHO:
openssl pkcs12 -export -out ~/ibmwebspheremqqm1.p12 -name "ibmwebspheremqqm1" -inkey /export/ldbd/etc/grid-security/ldbdserver/ldbdkey.pem -in /export/ldbd/etc/grid-security/ldbdserver/ldbdcert.pem
gsk6cmd -keydb -create -db /var/mqm/qmgrs/QM1/ssl/key.kdb -pw xxxxxxxx -type cms -expire 365 -stash
gsk6cmd -cert -add -db /var/mqm/qmgrs/QM1/ssl/key.kdb -pw xxxxxxxx -label "ESnet Root CA 1" -file ~/ldg/globus/share/certificates/d1b603c3.0 -format ascii
gsk6cmd -cert -add -db /var/mqm/qmgrs/QM1/ssl/key.kdb -pw xxxxxxxx -label "DOEgrids CA 1" -file ~/ldg/globus/share/certificates/1c3f2ca8.0 -format ascii
gsk6cmd -cert -import -file ~/ibmwebspheremqqm1.p12 -pw xxxxxxxx -type pkcs12 -target /var/mqm/qmgrs/QM1/ssl/key.kdb -target_pw xxxxxxxx -target_type cms
- For LLO:
openssl pkcs12 -export -out ~/ibmwebspheremqqm2.p12 -name "ibmwebspheremqqm2" -inkey /export/ldbd/etc/grid-security/ldbdserver/ldbdkey.pem -in /export/ldbd/etc/grid-security/ldbdserver/ldbdcert.pem
gsk6cmd -keydb -create -db /var/mqm/qmgrs/QM2/ssl/key.kdb -pw xxxxxxxx -type cms -expire 365 -stash
gsk6cmd -cert -add -db /var/mqm/qmgrs/QM2/ssl/key.kdb -pw xxxxxxxx -label "ESnet Root CA 1" -file ~/ldg/globus/share/certificates/d1b603c3.0 -format ascii
gsk6cmd -cert -add -db /var/mqm/qmgrs/QM2/ssl/key.kdb -pw xxxxxxxx -label "DOEgrids CA 1" -file ~/ldg/globus/share/certificates/1c3f2ca8.0 -format ascii
gsk6cmd -cert -import -file ~/ibmwebspheremqqm2.p12 -pw xxxxxxxx -type pkcs12 -target /var/mqm/qmgrs/QM2/ssl/key.kdb -target_pw xxxxxxxx -target_type cms
- For CIT:
openssl pkcs12 -export -out ~/ibmwebspheremqqm3.p12 -name "ibmwebspheremqqm3" -inkey /export/ldbd/etc/grid-security/ldbdserver/ldbdkey.pem -in /export/ldbd/etc/grid-security/ldbdserver/ldbdcert.pem
gsk6cmd -keydb -create -db /var/mqm/qmgrs/QM3/ssl/key.kdb -pw xxxxxxxx -type cms -expire 365 -stash
gsk6cmd -cert -add -db /var/mqm/qmgrs/QM3/ssl/key.kdb -pw xxxxxxxx -label "ESnet Root CA 1" -file ~/ldg/globus/share/certificates/d1b603c3.0 -format ascii
gsk6cmd -cert -add -db /var/mqm/qmgrs/QM3/ssl/key.kdb -pw xxxxxxxx -label "DOEgrids CA 1" -file ~/ldg/globus/share/certificates/1c3f2ca8.0 -format ascii
gsk6cmd -cert -import -file ~/ibmwebspheremqqm1.p12 -pw xxxxxxxx -type pkcs12 -target /var/mqm/qmgrs/QM3/ssl/key.kdb -target_pw xxxxxxxx -target_type cms
- For LHO:
Backup and restoration of the database
The database is enabled for online backups and can be backed up using the standard techniques to backup and restore a DB2 database. IBM provides documentation on performing a database backup, restore, and rollforward.