AJO - The Asynchronous Job Operator Portal for Grid Engine (2013-02-09)
AJO
simplifies the task of job submission and file-staging over a secure
connection from a submission portal (or just your local computer) to your
compute farm. It was an internal project at RDLAB and recently published
under the open source GPL3 license. The scripts are downloadable as a tarball
or accessible via SVN from
http://svn-rdlab.lsi.upc.edu/subversion/ajo/public. The username and
password is both public_ajo.
All what it requires is an ssh client installation as well as Ruby. AJO is
a set of Ruby scripts including a configuration file (config.rb),
which must be adapted to your environment. You have to set the hostname of
the Grid Engine submission host, the path to your remote Grid Engine
installation ($SGE_ROOT). For the
encryption you need to set cipher salt and keys. The next thing to
configure in the configuration file is the local directories and files
with data you need on your remote Univa Grid Engine cluster as well as the
output directories/files you need back on your local host. Those
files/directories are going to be copied transparently to the cluster
during job launching. Finally there is a section where you can insert the
GE job script contents (the job scripts are generated on the fly). Each of
them is going to be started as an own Grid Engine job.
After you configured your template you can launch it with:
./ajo -c config.rb -s Job submitted correctly. The job identifier is da558c9bf39fd052806e69d6afbc36a7e0718a53604eaff47bf6efd081fe40a239aa6572...
The status of your jobs you can track remotely with a secure token generated during submission.
./ajo -q da558c9bf39fd052806e69d6afbc36a7e0718a53604eaff47bf6efd081fe40a239aa... Your job has finished running on Sat Feb 9 09:36:58 2013. You can now do './ajo --retrieve ID' to download the output files and folders.
This token based system makes it an ideal candidate for using within a web based job submission portal.
Finally you want to get the output back to your local machine.
./ajo --retrieve da558c9bf39fd052806e69d6afbc36a7e0718a53604eaff47bf.... Downloaded the output to /tmp/tmp.ZpiQWNfD..
More detailed information about how it works you can find on their web page at RDLAB.