AJO - The Asynchronous Job Operator Portal for Grid Engine (2013-02-09)

AJO simplifies the task of job submission and file-staging over a secure connection from a submission portal (or just your local computer) to your compute farm. It was an internal project at RDLAB and recently published under the open source GPL3 license. The scripts are downloadable as a tarball or accessible via SVN from http://svn-rdlab.lsi.upc.edu/subversion/ajo/public. The username and password is both public_ajo.

All what it requires is an ssh client installation as well as Ruby. AJO is a set of Ruby scripts including a configuration file (config.rb), which must be adapted to your environment. You have to set the hostname of the Grid Engine submission host, the path to your remote Grid Engine installation ($SGE_ROOT). For the encryption you need to set cipher salt and keys. The next thing to configure in the configuration file is the local directories and files with data you need on your remote Univa Grid Engine cluster as well as the output directories/files you need back on your local host. Those files/directories are going to be copied transparently to the cluster during job launching. Finally there is a section where you can insert the GE job script contents (the job scripts are generated on the fly). Each of them is going to be started as an own Grid Engine job.

After you configured your template you can launch it with:

      ./ajo -c config.rb -s
      Job submitted correctly. The job identifier is
      da558c9bf39fd052806e69d6afbc36a7e0718a53604eaff47bf6efd081fe40a239aa6572...
 

The status of your jobs you can track remotely with a secure token generated during submission.

      ./ajo -q
      da558c9bf39fd052806e69d6afbc36a7e0718a53604eaff47bf6efd081fe40a239aa...
      Your job has finished running on Sat Feb 9 09:36:58 2013. You can now do
      './ajo --retrieve ID' to download the output files and folders.
 

This token based system makes it an ideal candidate for using within a web based job submission portal.

Finally you want to get the output back to your local machine.

      ./ajo --retrieve da558c9bf39fd052806e69d6afbc36a7e0718a53604eaff47bf....
      Downloaded the output to /tmp/tmp.ZpiQWNfD..
 

More detailed information about how it works you can find on their web page at RDLAB.