The installation process continues...

The shelving units in our computer room, with about 3/4 of the machines installed. There are 13 on each of the bottom two shelves, then 10 on the third shelf (leaving room for the networking switches) and 12 machines on the top shelf. Total weight of all the machines is about 1500 lbs. Notice the "safety-belt" straps that have been added to the ends to prevent any machines from toppling off the end of the shelving. The square bar angling down and to the right in the top right of the photo is actually a bit of hollow plastic drainpipe, that carries an extension cord, two monitor cables, two keyboard cables, and two mouse cables from the equipment rack on the left to the console table on the right.

During the installation process -- here about 36 of the machines are on the shelves. The networking switch is located right in the middle of the third shelf.

The back of the shelving unit -- the bottom 24 machines are operational, the top 24 are just in the process of being installed. Notice the plastic rain-gutters (white on the inside) being used as cable trays to support the weight of the CAT-5 cabling. Just behind these are 20-amp power strips (two per shelf) which provide a convenient access to power from anywhere on the shelving. Unfortunately our machines won't boot without keyboards attached, so the keyboards are stored on the top of each machine, and are left plugged in. [If anyone knows where to buy "keyboard eliminators" or "keyboard emulators" for IBM PS/2 style keyboards, please email Bruce Allen.]

Close-up of the back of node 19. The key hanging from a twist-tie is the key for opening the case. Notice the unused RJ-45 and thin ethernet connectors just above the loops of the power cord. These are the original (supplied with the machine) 10BaseT ethernet ports. The ethernet board which is actually being used is hidden behind the loops of the power cord, and is a 100BaseT card. The systems are cooled internally by two variable-speed fans. One, located at the bottom front of the case, pulls in air through the lower part of the grill. The other fan, located internally in the power supply, exhausts through the grill above the orange tag on the power cord.

Close-up view of the switches. The top switch is ready for the installation of the second twenty-four machines - the bottom switch is already connected. Sitting on top of the switches is our "temperature monitoring system"- a Radio Shack indoor/outdoor thermometer with a min/max memory. The switch has green "connected" lights that provide instant visual feedback about which machines are up and networked and which are not. Here (during the installation process) the only three nodes that are operating are the Master (N001) and slaves 23 and 24 (N023 and N024).

Bala seated at the terminal in the machine room. We normally use the Beowulf over the net, but it's convenient to have a terminal in the room for installation and when something goes wrong. The panel on the right is an (out-of-service) power distribution panel for some of the Physics labs. 100 amps at 5 volts, anyone?

Wensheng Hua working on the power wiring. Initially the electrician put the outlets for our circuits on the ground, which was a real mistake. We have now had the wiring moved 10 feet up the wall, so that we can walk all the way around the computer shelving with no obstructions on the floor, since the wiring is now overhead.

The power panel breaker box for the Beowulf system. The large black box hanging off the bottom right hand corner of the panel box is a surge protector that protects all the circuits within the box. As shown there are 3 breakers on the left (60 amps each) for the three-phase power coming into the box. On the right are 6 20-amp breakers for the computer power-strips. Eventually we will add 2 additional 20-amp circuits so that we will have a total of eight, one for each of the eight power strips. In our experiments, we were able to put 10 computers on a single 20-amp circuit without triggering the circuit breaker, so powering only 6 machines per 20-amp circuit should be plenty of margin. [Actual measurements of the current consumption show a total of about 60 amps for all 48 machines, so about 1.2 amps or 145 watts/machine. Thus eight 20-amp circuits give us about a 2.5 safety margin.] Heat dissipation in the room may be a problem in the future -- there are about 7200 Watts of heat (about a 2-ton air conditioning load) to dissipate. Fortunately the HVAC people were able to crank up the supply of chilled air to the room to ~ 500 CFM, and this keeps the temperature in the mid 70s. We'll probably add cheap temperature-logging to the master at some point and log the temperature one per minute at a few different locations in the room).

Wensheng Hua at back of the equipment rack.

R. Balasubramanian taking a break in front of the machine rack.

Bala seated at the terminal in the machine room. Notice the switch to the right which allows the monitor/keyboard/mouse to be connected to two different computers. Position # 1 is permanently connected to the master (node 1) machine, and Position # 2 is connected to a set of three long cables which can be plugged into any machine on the rack. It turned out that this particular switch was a piece of junk (one could not properly boot a machine connected to it: the AlphaBIOS keyboard check fails) and has been replaced with a Belkin one that does allow proper booting.

This shows the cabling for the "Position # 2" option coming down the back of the rack. The cables are long enough to plug into any machine.

The cabling for the "Position # 2" option.