DESCRIPTION
Complex reflects the format of the Univa Grid Engine complex configura-
tion. The definition of complex attributes provides all pertinent
information concerning the resource attributes a user may request for a
Univa Grid Engine job via the qsub(1) -l option and for the interpreta-
tion of these parameters within the Univa Grid Engine system.
The Univa Grid Engine complex object defines all entries which are used
for configuring the global, the host, and queue object. The system has
a set of pre defined entries, which are assigned to a host or queue per
default. In a addition can the user define new entries and assign them
to one or multiple objects. Each load value has to have its correspond-
ing complex entry object, which defines the type and the relational
operator for it.
defining resource attributes
The complex configuration should not be accessed directly. In order to
add or modify complex entries, the qconf(1) options -Mc and -mc should
be used instead. While the -Mc option takes a complex configuration
file as an argument and overrides the current configuration, the -mc
option bring up an editor filled in with the current complex configura-
tion.
The provided list contains all definitions of resource attributes in
the system. Adding a new entry means to provide: name, shortcut, type,
relop, requestable, consumable, default, and urgency. The fields are
described below. Changing one is easily done by updating the field to
change and removing an entry by deleting its definition. An attribute
can only be removed, when it is not referenced in a host or queue
object anymore. Also does the system have a set of default resource
attributes which are always attached to a host or queue. They cannot be
deleted nor can the type of such an attribute be changed.
working with resource attributes
Before a user can request a resource attribute it has to be attached to
the global, host, or cqueue object. The resource attribute exists only
for the objects, it got attached to ( if it is attached to the global
object(qconf -me global), it exits system wide, host object: only on
that host (qconf -me NAME): cqueue object: only on that cqueue (qconf
-mq NAME)).
When the user attached a resource attribute to an object, one also has
to assign a value to it; the resource limit. Another way to get a
resource attribute value is done by configuring a load sensor for that
attribute.
Default queue resource attributes
In its default form it contains a selection of parameters in the queue
configuration as defined in queue_conf(5). The queue configuration
parameters being requestable for a job by the user in principal are:
s_stack
h_stack
s_core
h_core
s_rss
h_rss
Default host resource attributes
The standard set of host related attributes consists of two categories.
he first category is built by several queue configuration attributes
which are particularly suitable to be managed on a host basis. These
attributes are:
slots
s_vmem
h_vmem
s_fsize
h_fsize
(please refer to queue_conf(5) for details).
Note: Defining these attributes in the host complex is no contradiction
to having them also in the queue configuration. It allows maintaining
the corresponding resources on a host level and at the same time on a
queue level. Total virtual free memory (h_vmem) can be managed for a
host, for example, and a subset of the total amount can be associated
with a queue on that host.
The second attribute category in the standard host complex are the
default load values Every sge_execd(8) periodically reports load to
sge_qmaster(8). The reported load values are either the standard Univa
Grid Engine load values such as the CPU load average (see uptime(1)) or
load values defined by the Univa Grid Engine administration (see the
load_sensor parameter in the cluster configuration sge_conf(5) and the
Univa Grid Engine Installation and Administration Guide for details).
The characteristics definition for the standard load values is part of
the default host complex, while administrator defined load values
require extension of the host complex. Please refer to the file
<sge_root>/doc/load_parameters.asc for detailed information on the
standard set of load values.
Overriding attributes
One attribute can be assigned to the global object, host object, and
queue object at the same time. On the host level it might get its value
from the user defined resource limit and a load sensor. In case that
the attribute is a consumable, we have in addition to the resource
limit and its load report on host level also the internal usage, which
the system keeps track of. The merge is done as follows:
In general an attribute can be overridden on a lower level
- global by hosts and queues
- hosts by queues and load values or resource limits on the same
level.
one is kept.
Note, Univa Grid Engine allows backslashes (\) be used to escape new-
line (\newline) characters. The backslash and the newline are replaced
with a space (" ") character before any interpretation.
FORMAT
The principal format of a complex configuration is that of a tabulated
list. Each line starting with a '#' character is a comment line. Each
line despite comment lines define one element of the complex. A element
definition line consists of the following 8 column entries per line (in
the order of appearance):
name
The name of the complex element to be used to request this attribute
for a job in the qsub(1) -l option. A complex attribute name (see com-
plex_name in sge_types(1)) may appear only once across all complexes,
i.e. the complex attribute definition is unique.
shortcut
A shortcut for name which may also be used to request this attribute
for a job in the qsub(1) -l option. An attribute shortcut may appear
only once across all complexes, so as to avoid the possibility of
ambiguous complex attribute references.
type
This setting determines how the corresponding values are to be treated
Univa Grid Engine internally in case of comparisons or in case of load
scaling for the load complex entries:
o With INT only raw integers are allowed.
o With DOUBLE floating point numbers in double precision (decimal and
scientific notation) can be specified.
o With TIME time specifiers are allowed. Refer to queue_conf(5) for a
format description.
o With MEMORY memory size specifiers are allowed. Refer to
queue_conf(5) for a format description.
o With BOOL the strings TRUE and FALSE are allowed. When used in a
load formula (refer to sched_conf(5) ) TRUE and FALSE get mapped
into '1' and '0'.
o With STRING all strings are allowed and is used for wildcard regular
boolean expression matching. Please see sge_types(1) manpage for
expression definition.
Examples:
-l arch="*x*|sol*" :
results in NEITHER "arch=lx-x86" NOR "arch=sol-amd64"
-l arch="lx2[4|6]-amd64" :
results in "arch=lx24-amd64" OR "arch=lx26-amd64"
o CSTRING is like STRING except comparisons are case insensitive.
o RESTRING is like STRING and it will be deprecated in the future.
o HOST is like CSTRING but the expression must match a valid hostname.
relop
The relation operator. The relation operator is used when the value
requested by the user for this parameter is compared against the corre-
sponding value configured for the considered queues. If the result of
the comparison is false, the job cannot run in this queue. Possible
relation operators are "==", "<", ">", "<=", ">=" and "EXCL". The only
valid operator for string type attributes is "==".
The "EXCL" relation operator implements exclusive scheduling and is
only valid for consumable boolean type attributes. Exclusive means the
result of the comparison is only true if a job requests to be exclusive
and no other exclusive or non-exclusive jobs uses the complex. If the
job does not request to be exclusive and no other exclusive job uses
the complex the comparison is also true.
requestable
The entry can be used in a qsub(1) resource request if this field is
set to 'y' or 'yes'. If set to 'n' or 'no' this entry cannot be used
by a user in order to request a queue or a class of queues. If the
entry is set to 'forced' or 'f' the attribute has to be requested by a
job or it is rejected.
To enable resource request enforcement the existence of the resource
has to be defined. This can be done on a cluster global, per host and
per queue basis. The definition of resource availability is performed
with the complex_values entry in host_conf(5) and queue_conf(5).
consumable
The consumable parameter can be set to either 'yes' ('y' abbreviated),
'no' ('n') or 'JOB' ('j'). It can be set to 'yes' and 'JOB' only for
numeric attributes (INT, DOUBLE, MEMORY, TIME - see type above). If set
to 'yes' or 'JOB' the consumption of the corresponding resource can be
managed by Univa Grid Engine internal bookkeeping. In this case Univa
Grid Engine accounts for the consumption of this resource for all run-
ning jobs and ensures that jobs are only dispatched if the Univa Grid
Engine internal bookkeeping indicates enough available consumable
resources. Consumables are an efficient means to manage limited
resources such a available memory, free space on a file system, network
bandwidth or floating software licenses.
A consumable defined by 'y' is a per slot consumables which means the
To enable consumable resource management the basic availability of a
resource has to be defined. This can be done on a cluster global, per
host and per queue basis while these categories may supersede each
other in the given order (i.e. a host can restrict availability of a
cluster resource and a queue can restrict host and cluster resources).
The definition of resource availability is performed with the com-
plex_values entry in host_conf(5) and queue_conf(5). The complex_val-
ues definition of the "global" host specifies cluster global consumable
settings. To each consumable complex attribute in a complex_values list
a value is assigned which denotes the maximum available amount for that
resource. The internal bookkeeping will subtract from this total the
assumed resource consumption by all running jobs as expressed through
the jobs' resource requests.
Note: Jobs can be forced to request a resource and thus to specify
their assumed consumption via the 'force' value of the requestable
parameter (see above).
Note also: A default resource consumption value can be pre-defined by
the administrator for consumable attributes not explicitly requested by
the job (see the default parameter below). This is meaningful only if
requesting the attribute is not enforced as explained above.
See the Univa Grid Engine Installation and Administration Guide for
examples on the usage of the consumable resources facility.
default
Meaningful only for consumable complex attributes (see consumable
parameter above). Univa Grid Engine assumes the resource amount denoted
in the default parameter implicitly to be consumed by jobs being dis-
patched to a host or queue managing the consumable attribute. Jobs
explicitly requesting the attribute via the -l option to qsub(1) over-
ride this default value.
urgency
The urgency value allows influencing job priorities on a per resource
base. The urgency value effects the addend for each resource when
determining the resource request related urgency contribution. For
numeric type resource requests the addend is the product of the urgency
value, the jobs assumed slot allocation and the per slot request as
specified via -l option to qsub(1). For string type requests the
resources urgency value is directly used as addend. Urgency values are
of type real. See under sge_priority(5) for an overview on job priori-
ties.
SEE ALSO
sge_intro(1), sge_types(1), qconf(1), qsub(1), uptime(1), host_conf(5),
queue_conf(5), sge_execd(8), sge_qmaster(8)
Univa Grid Engine Installation and Administration Guide.
COPYRIGHT
See sge_intro(1) for a full statement of rights and permissions.
Man(1) output converted with
man2html