Univa Grid Engine 8.4.1 Released (2016-07-19)

Univa Grid Engine 8.4.1 was released by Univa’s engineering team last week. This is the first update for the major 8.4 release which brings in main fixes as well as new capabilities.

The major feature introduced in 8.4 is native Docker support. Native Docker support of Univa Grid Engine means that Docker enabled hosts are automatically discovered (without any configuration needed by the admin) and the installed images are forwarded to the global scheduler of Univa Grid Engine. The user just needs to select a image name and then its job script or binary is executed on an selected execution host within the automatically created Docker container. Since the Docker containers are not anymore child processes of UGE and therefore not directly supervised by Univa Grid Engine (for accounting and resource limitation for example) a new process is injected in the container which supervises the job within the container as Univa Grid Engine does it for jobs outside of a container. Working directories and job spooling directories are automatically mounted inside the container.

Univa Grid Engine 8.4.1 adds now capabilities to launch containers with all the parameters you know from Docker run with a new qsub parameter called -xd (to select ports or devices for example).

Univa Grid Engine is a proven and a highly scalable (thousands of compute nodes) solution not just for traditional compute jobs, now also for Docker containers, which comes with enterprise features like accounting, reporting, access control, fair usage of compute resources through a set of policies which controls job priorities, simplification of administration and job submission through job classes, APIs, and quotas on container usage.

Update of the Univa Grid Engine Vagrant integration (2016-06-15)

I just updated the Vagrant integration for Univa Grid Engine so that it allows you to create an Univa Grid Engine 8.4 test cluster with three hosts just by calling vagrant up.

Just place the new Univa Grid Engine 8.4 Demo Packages ge-8.4.0-demo-bin-lx-amd64.tar.gz and ge-8.4.0-demo-common.tar.gz in the directory. You can download them from Univa

The referenced CentOS box was required to be updated to version 7.2, so if it is not yet cached locally by Vagrant a bigger download will follow. Also Docker is now going to be installed (and a config change needs to be done) on the hosts so that you easily can try the Docker integration by requesting images for your jobs. Also a simple script which downloads, compiles, and installs runc is in the directory.

For any kind of problems please create an issue at the github repository directly.

After the intallation process you can run your first Docker job in Univa Grid Engine:

• vagrant up --> this might take a while
• vagrant ssh --> you are now on the master node
• qhost --> just showing if the cluster is up and running
• docker pull golang:latest --> we need an image for our container
• cd /nfs
• warning: mounting non-fitting /etc in Docker could become dirty
• qsub -b y -l docker,docker_images="*golang:latest*" -wd /nfs -xdv /nfs:/nfs,/etc:/etc -S /bin/sh /bin/sleep 123
• docker ps --> it is launched locally because our image is only there / watch the created container
• qdel --> remove the job and along with it the container
• docker ps --> see that the container is removed
• launch now an interactive job
• qrsh -l docker,docker_images="*golang:latest*" -xdv /nfs:/nfs,/etc:/etc
• you are now in the container
• ps -ef --> see PID namespace isolation within the container

Univa Tech Days and Upcoming Univa Grid Engine Webinar (2016-06-14)

For more information about Univa Grid Engine 8.4 please also consider the upcoming webinar. Since seats are limited, please register early!

Also Univa's Tech Days are back! For more information please check out following link.

Univa Grid Engine 8.4 Release (2016-06-14)

Univa Grid Engine 8.4 is out!

You’ve probably already come across the news that my dear colleagues of the Univa Grid Engine development team released the 5th major Univa Grid Engine version in the year five after releasing the first Univa Grid Engine version 8.0.

Again, too many improvements to cover them all here in detail. If you are interested in all changes please consult the release-notes published here at univa.com.

Just a few highlights for the moment I really think they should be considered by all Grid Engine users as I think they will open many more use cases for running applications as simple as possible for the user and admins.

First of all, of course, the new native Docker support. Docker is everywhere, everyone likes to experiment with it, or use it already since a while in production. This is also true for Grid Engine users. While the previous versions of Univa Grid Engine contained some integration scripts the new 8.4 release comes with built-in out-of-the-box Docker support which couldn’t made it simpler for both, the users and the administrators.

Built-in Docker Support - How it works in a nutshell

To summarize:

• the execution daemon automatically recognizes the docker daemon and marks the host as docker host

• all images which are locally available are reported to the central Univa Grid Engine scheduler and treated as resources

• when the user selects a docker image as a required resource the job is routed to an appropriate host. Then the execution daemon automatically creates an Docker container for the job and launches the given application in the container. Within the container a new component (called co-shepherd) is launched which supervises the job and adds additional flexibility to the container due to its support for prolog, epilog and other known Univa Grid Engine features within and outside of the container

That means without further configuration the users can simply run their scripts and applications by simply doing

qsub -l docker,docker_images=java:latest ./my_application.sh

Of course this can be simplified further by creating job classes for it which allows the administrator to create quotas on specific containers and configure fine grained access control by putting job classes in the queue configuration. All the flexibility you know from Univa Grid Engine jobs are applied to Docker containers at scale.

Other Features and Functionalities

There are plenty of other features and improvements. One of my favorite is certainly the new -tcon parameter for qsub. Univa Grid Engine is certainly pretty good (and scalable) with handling job arrays, i.e. jobs which are launched thousands of times but processing each time different data sets. Those job arrays can be limited with the -tc parameter in order to prevent that too many tasks of the same jobs are started in parallel. With the new -tcon parameter (task concurrency) you can now instruct the scheduler that either all or none of the tasks should be started. By using this parameter you can now co-schedule a bunch of tasks which are working together. All what needs to be done is reading out the SGE_TASK_ID environment variable in the job script and then launch the specific application. Of course they have to share all the same properties (like limits).

More functionalities were implemented around diagnosing the system, profiling and monitoring are constantly improved in each version. Meanwhile a new man page (sge_diagnostics) summarizes all the new capabilities in a single place.

For the case you are now curious about the new release and want to try it out. Please go ahead and download the new 48-core limited trial version directly from Univa.

For setting up a 3 VM test cluster you can use the Vagrant integration, but please note it is still CentOS 6.7 based - I’m going to update it soon (just change the box version in the Vagrant file). Also the VERSION string should be set to the version you have in the installation.sh script. For questions regarding that please create issues directly on github.com.

Free Univa Webinars in April (2016-4-4)

Univa is hosting some free webinars about Docker and Navops in April.

• [webinar] Docker 101 Date: Thursday, April 21, 2016
  Time: 10am PDT / 1pm EDT Register Here

• [webinar] Docker 101 Date: Tuesday, April 26, 2016
  Time: 10am UK Time / 11am CEST / 5pm China Time / 6pm Japan Time Register Here

• [Online Meetup] Going Cloud-Native with Navops Launch and Docker Date: Thursday, April 28, 2016
  Time: 10am PDT / 1pm EDT Register Here