Job Accounting and Cluster Usage in Univa Grid Engine (2012-07-17)

This article introduces the different methods how cluster usage can be accessed in Univa Grid Engine

Job accounting and cluster usage data can be accessed in Univa Grid Engine at different layers. The most basic layer is accessing raw text files generated by Grid Engine at runtime. After a default installation the qmaster process writes out an accounting file which can be found at $SGE_ROOT/default/common/accounting. It contains information about past jobs, which already finished and the (runtime) system. The layout of the file is described in the man page (man 5 accounting). This file is accessed directly by the qacct command line tool, which also requires a running qmaster where qacct connects to in order to get some lists. The second text file (contained in the same directory) is the reporting file. In order to let qmaster write the file it must be activated (qconf -mconf and then setting reporting from false to true in the reporting_params row). The structure is described in the man page reporting.

Univa Grid Engine also includes a package called ARCo which, when installed starts a process called dbwriter. The job of dbwrites is to parse both files and writes the content into a SQL database (preferably PostgreSQL). After doing this the actual data is usually deleted from the reporting file. The SQL interface can be seen as the most flexible interface to derive reports from the cluster because all data is stored there in a structured form for long time. It can be processed by an huge amount of tools which offers SQL access.

The commercial version of Univa Grid Engine comes with an additional tool: UniSight. UniSight allows not just a single view over different Grid Engine (and other resource manager) installations, it also loads the data into a Data Warehouse in order to get fast reports about jobs metrics and cluster usages even when you have huge amount of jobs. It provides a web interface where you can configure, store and run different reports including graphics and different ways to export the data (like PDFs, Excel formal, or CSVs).

The following picture illustrates the different ways of how to get information about your cluster usage and job accounting in Univa Grid Engine.