I don't have any Linux cluster servers to look after but do manage 2 Linux server farms. They are similar to clusters in that they are both groups of RHEL servers that all have to have an identical config. The larger of these farms is 12 RHEL 5.3 servers, and to roll out changes to them all, I wanted to be able to make the change once, and after it was verified, make the same change on the other 11 servers. I started off by writing this script on server1:

#!/bin/sh

echo -n "Enter command to run (on one line): "
read STRING

for SERVER in server2 server3 server4 server5 server6 etc; do

echo -e "\033[1;31m$SERVER says:\033[m"
ssh $SERVER "$STRING"

done

echo -n "Do you want to run the command locally? (y/n) :"
read ANSWER

case "$ANSWER" in

y|Y) echo -e "\033[1;31mlocalhost says:\033[m"; $STRING ;;
N|n) exit 1 ;;
esac

For this to work I had to create ssh keys on each server using:

ssh-keygen -t dsa

And then install the newly generated key (~/ssh/id_dsa.pub) into the authorized_keys file on server1. This works for running simple commands one at a time.

For other tasks its sometimes necessary to manage multiple ssh sessions at once, for example to monitor resources using htop or tailing log files. On Linux you can use ClusterSSH (cssh) but this turned out to be a royal pain in the butt to get working on CentOS or RHEL. It worked OK in an Ubuntu VM but was a bit clunky and I felt there would be a better way of managing multiple PuTTy windows, since I am using a Windows 7 laptop for my day-to-day stuff.

There are quite a few goodies for this on the Links page on the Putty website and this is where I found Putty Command Sender. Quite simply you type your command into the command sender, and much like clusterssh, it sends it to all the putty windows you have open.

puttys
Putty Command Sender

It's not so great for editing files in Vi on 12 servers at once but it is possible - you can send cursor movements as well as lines of code or single commands. The only thing you have to watch out for in PuttyCS is that all Putty windows have updated before you start typing your next command, otherwise the last window to update may miss the first few characters of the next command.

Finally to launch my sessions in groups I'm using Putty Session Manager. Other alternatives are available but I found this one to be the best. It's lightweight nature fits in with Putty nicely.