UWMLSC > Beowulf Systems > Nemo
  

PXE booting a new copy of a slave, nfs, or nfsdata bios image.

  • Step 1:
    • Log into nemo.phys.uwm.edu as root
    • cd to /tftpboot/pxelinux.cfg
    • Run ./pxe_symlink.sh [slave node low] [slave node high] [image you want to pxeboot]
      • example ./pxe_symlink.sh 1 780 default.nemo_node_motherboard_bios_only_02-26-07-update
      • this will flash a the most current bios image on slave nodes 1 through 780
      • use ./nfs_pxe_symlink.sh for nfs boxes
      • use ./nfsdata_pxe_symlink.sh for nfsdata boxes
    • These scripts leave a hex version of the target machines IP address in this directory. You can ls to confirm that they have been placed there. As long as that file is there the machine it points to will boot from the network.
  • Step 2:
    • Reboot the machines you outlined in step one
    • This can be done with:
      • /root/root_tools/pdsh_all.sh -s [low slave node] [high slave node] -n [low nfs node] [high nfs node] -d [low nfsdata node] [hight nfsdata node] "/sbin/reboot"
        • eg: /root/root_tools/pdsh_all.sh -s 1 780 "/sbin/reboot" will reboot slave nodes 1 through 780
        • Running /root/root_tools/pdsh_all.sh with oout arguments will dump a help screen to standard out.
      • Physically pressing the buttons on the machines
      • /usr/bin/ipmitool -I lanplus -U ADMIN -P ADMIN -H b[machine name] power cycle
        • eg: /usr/bin/ipmitool -I lanplus -U ADMIN -P ADMIN -H bs0001 power cycle reboots s0001 from the bmc.
  • Step 3:
    • While the machines are rebooting connect a monitor to one of them. You will see the process of PXEbooting and flashing of the bios. When everything is finished the machine will be left at the a: prompt.
    • Delete the links in nemo:/tftpboot/pxelinux/ that were created in step one.
    • Once all machines are idle you can press the reset switch their face plates. This is the only way to reboot them unless we find a way to delete the PXE link and reboot the machines from the dos boot disk.


Design for automated BIOS update and configuration procedure

  1. Requirements
  2. Settings to put on nodes
  3. Our solution
  4. How it was done
  5. Components

Our Requirements

    A "hands off" way to update the following:

    • The BIOS firmware
    • CMOS settings in BIOS
    • BMC firmware

Desired BIOS Settings

  • Here is a text file that contains the bios settings we want set on the Nemo slave nodes.

Our solution

    One bootable FreeDOS 2.88 MB floppy image that

    • creates a ramdisk, at "C:"
    • automatically flashes the BIOS
    • automatically updates CMOS settings to predefined values
    • copys the BMC update utility onto C: and unzips it
    • automatically updates the BMC firmware
    • reboots the system

How it was done

    The following steps were done automate the entire process

    • installed a floppy drive in the test node, updated the BIOS and then configured the CMOS settings appropriately, then booted from the floppy image referenced below, ran smcmos.exe -r set.img which reads the CMOS settings from the motherboard and writes them to a file on the floppy called set.img
    • mounted the 2.88 MB FreeDOS floppy image, as root mount -o loop FDSTD.288.img /CRAP
    • mounted each of the other images and copied the relevant parts from each of the autoexec.bat and config.sys
    • mounted the floppy used earlier to record the CMOS settings, and copied the set.img file from the floppy to our new 2.88MB floppy image
    • on the 2.88MB image edited the new config.sys to load the FreeDOS extended memory manager FDXMS.SYS, and the TDSK.EXE ramdisk TSR and set up a 4MB ramdisk.
    • also on that image, edited the autoexec.bat to: define the RAMdrive as "C:", run utilities not yet understood by us but declared necessary by SuperMicro, run the flashing script flash.bat, run smcmos.exe -w set.img, copy the zipped utility for updating the BMC firmware onto "C:", unzip that utility, run that utility.

Components

    BIOS updater

    • BIOS ROM image provided by SuperMicro, this is the newest version that includes fixes for the BMC "forgetting" its MAC address and for preserving the boot order after BIOS flashing and auto CMOS setting.

    CMOS settings updater

    • CMOS utility DOS until for reading-from-CMOS/writing-to-file and reading-from-file/writing-to-cmos CMOS settings.
    • Floppy image of a FreeDOS image containing the SMCMOS.EXE utility, this is needed for getting the desired CMOS settings to work on the node after updating the BIOS. For this to work, you must update the BIOS, make sure the BIOS settings are how you want them, boot from this image and run smcmos.exe -r set.img which will read the CMOS settings, write them to the file called set.img. You then power down the machine, take this floppy to another machine, mount the floppy and copy set.img file to a place where you can later put it on the boot image you will use to auto-set the CMOS settings.

    BMC firmware flash updater

    • BMC firmware updater DOS floppy image provided by SuperMicro, this floppy image does NOT include the FreeDOS TURBODISK ramdisk TSR

    memdisk kernel

    • memdisk kernel image distributed in syslinux rpm

    TurboDisk RAM disk

    • TurboDisk 242b zip file. TurboDisk is a DOS TSR that provides a ramdisk. This is used as additional space for all the utils associated with the BMC updater. It is distributed as part of the FreeDOS distribution. The preceding link seems to come and go, but a mirror can be found here).

    2.88 MB FreeDOS image

    • 2.88 MB bootable FreeDOS floppy image. I put all other parts on this image

Check this page for dead links, sloppy HTML, or a bad style sheet; or strip it for printing.
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.