Univa Grid Engine 8.1 Features (Part 4) - New Spooling Method: PostgreSQL Spooling (2012-06-01)

Grid Engine supported 3 different spooling methods in the past: classic spooling (where the data is written directly into files on the filesystem), berkeleydb spooling (where the data is written into a BDB file) and berkeleydb RPC spooling (which allowed RPC access to a non-local BDB data over the network). Since BDB RPC spooling was removed in UGE 8.0.0 because sleepycat (BDB) deprecated the RPC support itself there was a need to add a new safe spooling method with network access support. Local BDB spooling has the disadvantage that it only supports filesystems with POSIX file semantics (NFS4) but not NFS3, when it is opened in an non-private mode (which is needed since different processes may access the DB the same time!). Hence when having a fail-over configuration with a Grid Engine shadow daemon you couldn't use NFS3, which is still often found in compute clusters.

Now, with the advent of Univa Grid Engine 8.1 you have a third option again: PostgreSQL spooling. It combines the advantages of a fast and reliable DB driven spooling with supporting a fail-over configuration even on filesystems like NFS3. It is very simple to setup, too.

Btw.: PostgreSQL can be already configured in the DB writer, which collects your job accounting information and UniSight (which builds up data warehouses based on the DBs and visualizes your cluster usage) is also shipped with PostgreSQL.