LSCSoft Repository Maintainer HOWTO
Contents
Overview
Pre-built software required for LSC data analysis is distributed from the DASWG software repository packaged for a variety of operating systems. This document explains in a general way how a package is put into the repository for distribution to the LSC. More detailed steps can be found in the Wiki web page LSCSOFT_Notes.
Red Hat based OS's Repository
The Fedora Core 4 [FC4] repository will serve as our reference OS, but this instructions are vaild also for Red Hat based OS's: Fedora Core 7 & 8, RedHat 9, RHEL-SW3, CentOS-5. The tarball used to build the software, the rpms built for the OS, and the srpms are all published into the repository. More information is accessible in the Wiki page LSCSOFT_Notes.
To be functional, each repository directory maintains a XML file, yumgroups.xml, in which all existing and new software must be defined and gathered in groups. This way, YUM can be used to download groups of packages. So, it's important to review this issue first.
I- Maintaining the yumgroups.xml file
"LSCSOFT" is a YUM defined group which is used to yum-download the LSC software bundle from the repository, and
is defined in yumgroups.xml for each release. This file is mostly
self-explanatory, it basically defines a set of groups and associated
rpms. Its estructure is similar to
<comps>
<group>
some identification tags...
<packagelist>
<packagereq type="mandatory">mypackage_1</packagereq>
...
<packagereq type="mandatory">mypackage_n</packagereq>
</packagelist>
</group>
<group>
...
</group>
</comps>
The most important group in this file is "LSCSOFT"; all approved rpms must have an entry of the form
<packagereq type="mandatory">mypackage</packagereq>in the "LSCSOFT" group. If a new package is added to the repository and is approved for release as part of "LSCSOFT", then an entry should be added for each rpm associated with the package.
If you modify the yumgroups.xml file, you should regenerate the header and repository information. For example, if yumgroups.xml is updated in the fedora core 4 repository, then must do (assuming you've access to the repository machine and have the permissions; read ahead for more details)
export LSCSOFT_REPOSITORY=/home/htdocs/uwmlsc/root/daswg/download cd $LSCSOFT_REPOSITORY ./lscsoft-publish-software.py -p fedora -r 4 --arch i386Note the --arch argument which is required; it should be either i386 or x86_64. This argument is ignored when the database is rebuilt, but is still required (this is a bug).
II- Building packates to release into the LSCSOFT repository
These instructions assume you have installed in your build machine all the development packages to build RPMs (gcc compilers, make, autoconf, automake, development
libs, rpm, rpm-build, redhat-rpm-config, etc.) AND a released software package
for which you have a tarball (*.tar.gz), or the *.src.rpm which contains a spec file to build rpms.
su root
mkdir /usr/src/redhat cd /usr/src/redhat mkdir SOURCES SRPMS RPMS SPECS BUILD cd /usr/src/redhat/BUILD mkdir noarch i386 x86_86
mv mypackage-X.YY.tar.gz /usr/src/redhat/SOURCES/ rpmbuild -ta /usr/src/redhat/SOURCES/mypackage-X.YY.tar.gz
rpm -Uhv mypackage-X.YY.src.rpm rpmbuild -ba /usr/src/redhat/SPECS/mypackage.spec
III- Publishing released software
To "publish" mypackage-X.YY.rpm, you must put all your *tar.gz, *rpm, *src.rpm files into the physical repository machine and fulfill some
requirements, in order to others to have access to
your built packages.
export LSCSOFT_REPOSITORY=/home/htdocs/uwmlsc/root/daswg/download mv mypackage-X.YY.tar.gz mypackage-X.YY.rpm mypackage-X.YY.src.rpm $LSCSOFT_REPOSITORY/software/fedora/incoming/
This example publishes your packages to the FC4 production repository:
cd $LSCSOFT_REPOSITORY ./lscsoft-publish-software.py -g $LSCSOFT_REPOSITORY/software/fedora/incoming/ -p fedora -r 4 --arch XXX
To publish packages to the LSCSOFT testing repo add -t at the end of the lscsoft-publish-software.py command.
./lscsoft-publish-software.py -g $LSCSOFT_REPOSITORY/software/fedora/incoming/ -p fedora -r 4 --arch XXX -t
Note that the machine on which lscsoft-publish-software.py is run must have both createrepo and repoview installed since the python script uses these two commands to build the content and webpages of the repository directory you're working on.
This other example publishes your packages to the Centos 5 repository run the following:./lscsoft-publish-software.py -g $LSCSOFT_REPOSITORY/software/centos/incoming/ -p centos -r 5 --arch XXXNote the --arch argument which is required; replace the XXX with either i386 or x86_64.
Now, the software should be available for users using the instructions at Download > Repositories. Go to the link and check this!
IV- Announcement E-mail
Send an Announcement E-mail to DASWG which announcing the release of the software package via LSCSOFT yum group.
Below is sample text for announcing package updates to the lscsoft repository. Make the appropriate changes and send the
email to the Daswg mailing list.
package version x.x and package version x.x... have been published into the LSCSOFT yum repository for fedora-release (x86_64,i386), fedora-release (i386) and redhat-release (i386). Intructions for using the repository are at http://www.lsc-group.phys.uwm.edu/daswg/download/repositories.html If you have the yum repository already configured, then you should just need yum groupupdate "LSCSOFT" This will get these packages and any other updates/dependencies from this repository. For information about these packages see link to package repoview info
Debian Repository
Under construction...Mac OS X
Mac OS X Universal Binaries can be built for distribution using the script create-lscsoft-mac-os-x. You will also need to download portable-libtool into the same directory. To use this script you must have installed the latest XCode tools. You will also need Universal MacPython2.4 and various associated packages. Download and install lscsoft-macpython2.4.dmg.The script will download the supported LSCSoft packages (and some needed non-LSC packages) and build them.
curl -O http://www.lsc-group.phys.uwm.edu/daswg/download/create-lscsoft-mac-os-x curl -O http://www.lsc-group.phys.uwm.edu/daswg/download/portable-libtool chmod +x create-lscsoft-mac-os-x ./create-lscsoft-mac-os-x --fresh
Once the packages are built, you need to package them. Download lscsoft-packages.tar.gz to get the PackageMaker files and associated resources. Untar this in the same directory that you ran create-lscsoft-mac-os-x. This will create a directory Packages. Within this directory open the following .pmproj files, check the version numbers to make sure that they are up-to-date, and build the packages:
curl -O http://www.lsc-group.phys.uwm.edu/daswg/download/software/darwin/lscsoft-packages.tar.gz tar zxvf lscsoft-packages.tar.gz cd Packages open FFTW.pmproj open Frame\ Library.pmproj open GNU\ Scientific\ Library.pmproj open Glue.pmproj open LSCSoft\ User\ Environment.pmproj open MetaIO\ Library.pmproj open Pkg\ Config.pmprojNow open the LSC Software Suite.pmproj and rebuild this package. Make sure that the package gets built in the subdirectory LSC Software Suite.
open LSC\ Software\ Suite.pmprojFinally, make the disk image with the following commands (change the version as appropriate!)
hdiutil create -srcfolder "LSC Software Suite" lscsoft-1.2.dmg hdiutil internet-enable -yes lscsoft-1.2.dmgThat should be it!
Fedora RPM Build Notes (to be moved to Wiki... next session)
This section is to document the process of building the lscsoft rpms on a Fedora Core install and is not intended to be strict steps to follow. The process was documented using glue as an example but should work with most packages.For more RPM info go here!
Building DMT Offline i386 and x86_64 rpms You must do in this order: 1. log into marlin 2. install root rpms 3. install framecpp rpms 4. source root-user-env.sh, framecpp-user-env.sh and lscsoft-user-env.sh 5. rpmbuild dol.spec To test dol: Create a bash script with the contents #! /bin/sh -x echo "Write a frame file" DMTGen -conf DMTGen-write.cfg echo "read it back..." FrDump -infile DMTGen-800000000-16.gwf |grep -v -e "CWVec" -e "time:" -e "chkSum:" |diff - DMTGen-write.out Then, before you run the script setup your environment export ROOT_LOCATION=/opt/lscsoft/root export ROOTSYS=/opt/lscsoft/root source /opt/lscsoft/root/etc/root-user-env.sh source /opt/lscsoft/dol/etc/dol-user-env.sh Building lscsoft-user-env 1. Must have access to daswg cvs. 2. Change directory to lscsoft/lsware/userenv/ 3. Change version number in configure.ac and run automake. Commit the new configure.ac and configure files to cvs. 4. Configure and Make. ./configure; make dist 5. There will now be a lscsoft-user-env.tar.gz. Move This to your build box. 6. Run rpmbuild. rpmbuild -ta lscsoft-user-env.tar.gz 7. Publish tar.gz, rpm and srpm Building lal/lalapps 1. Obtain src.rpms from Jolien 2. Log into jester as root 3. Install lal/lalapps src.rpms 4. Source user env script source /opt/lscsoft/lscsoft-user-env.sh 5. Build lal spec in /usr/src/redhat/SPECS rpmbuild -ba lal.spec 6. Install all three lal rpms (lal, lal-devel, lal-debuginfo) 7. Source lal user env script source /opt/lscsoft/lal/etc/lal-user-env.sh 8. Build lalapps spec in /usr/src/redhat/SPECS rpmbuild -ba lalapps.spec 9. Install lalapps rpms (lalapps, lalapps-debuginfo) 10. Test source /opt/lscsoft/lalapps/etc/lalapps-user-env.sh lal-version lalapps_version Building Libframe 1. Checkout lscsoft cvs 2. Change dirs to lsware 3. cvs update -r newest_tagged_release libframe 4. cd libframe 5. ./00boot.sh 6. ./configure --prefix=/opt/lscsoft/libframe --disable-python 7. make dist 8. scp libframe-version.tar.gz user@marlin:/usr/src/redhat/SOURCES/ 9. log into marlin 10. rpmbuild -ta /usr/src/redhat/SOURCES/libframe-version.tar.gz