You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Within a task, it is possible to access input and intermediate parameters, but it is not currently possible to access the actual values of attributes in the runtime, hints, or meta* sections.
For example, let's say that I request 25 GB of memory using memory: 25G. When my task is running, I know that I have at least 25 GB, but I don't know how much memory I actually have. It may be useful to know the exact resources provisioned, for example so I can specify a -t parameter to bwa or an -Xmx parameter to Java. I realize that it is possible to get some of this information within the command block using standard linux commands, but there may be differences in the outputs of these commands between platforms.
I propose that we add a reserved variable name wdl to the environment of every task that can be used by expressions in declarations and in the command section (but not in expressions in the runtime section) to access attributes of the runtime environment, as well as task metadata.
command <<<
bwa mem -t ~{wdl.runtime.cpu - 1} ...
>>>
output {
# If this task is running on DNAnexus, populate the# instance_type output with the actual instance type# that was selected to run this task, otherwise nullString? instance_type = wdl.hints.dx.instance_type
}
Details:
wdl.
runtime.
String? container: the image used to run the task; always has a protocol (e.g. docker://myrepo/myimg:tag); null if no container was used
Int cpu: actual number of cpus provisioned by the engine
Int memory: actual number of bytes of memory provisioned by the engine
Map[String, Float] disks: mapping of mount point to disk space in bytes provisioned by the engine
Int gpu: the number of GPUs provisioned; 0 if the engine is not able to determine if a GPU is available; 1 if the engine knows that at least 1 GPU is available, but is not able to determine exactly how many GPU are available
Int retries: the number of times the task has been retried; starts as 0 and is incremented by 1 each time the task is retried
Int? returnCode: the actual return code from the command section; the value is null except in the output section
hints: a copy of the attributes from the hints section
The engine may omit hints provided by the user and/or add extra hints at runtime; thus, attributes of hints are optional; an attempt to access one of these attributes returns null if it is not defined (rather than throwing an error)
meta: a copy of the attributes from the meta section
parameter_meta: a copy of the attributes from the parameter_meta section
Semantics:
The wdl variable is an object
The wdl variable is always present in the environment
The second-level attributes of wdl (i.e. runtime, hints, meta, paramater_meta) are always defined and are objects
The attributes of wdl.runtime are always defined. If the engine cannot determine the actual value of one of the runtime attributes, it must populate the attribute with the requested value (or the default value, if no override was specified by the user).
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Within a task, it is possible to access input and intermediate parameters, but it is not currently possible to access the actual values of attributes in the
runtime
,hints
, ormeta*
sections.For example, let's say that I request 25 GB of memory using
memory: 25G
. When my task is running, I know that I have at least 25 GB, but I don't know how much memory I actually have. It may be useful to know the exact resources provisioned, for example so I can specify a-t
parameter to bwa or an-Xmx
parameter to Java. I realize that it is possible to get some of this information within the command block using standard linux commands, but there may be differences in the outputs of these commands between platforms.I propose that we add a reserved variable name
wdl
to the environment of every task that can be used by expressions in declarations and in the command section (but not in expressions in theruntime
section) to access attributes of the runtime environment, as well as task metadata.Details:
wdl.
runtime.
String? container
: the image used to run the task; always has a protocol (e.g.docker://myrepo/myimg:tag
); null if no container was usedInt cpu
: actual number of cpus provisioned by the engineInt memory
: actual number of bytes of memory provisioned by the engineMap[String, Float] disks
: mapping of mount point to disk space in bytes provisioned by the engineInt gpu
: the number of GPUs provisioned; 0 if the engine is not able to determine if a GPU is available; 1 if the engine knows that at least 1 GPU is available, but is not able to determine exactly how many GPU are availableInt retries
: the number of times the task has been retried; starts as 0 and is incremented by 1 each time the task is retriedInt? returnCode
: the actual return code from the command section; the value is null except in theoutput
sectionhints
: a copy of the attributes from thehints
sectionhints
are optional; an attempt to access one of these attributes returns null if it is not defined (rather than throwing an error)meta
: a copy of the attributes from themeta
sectionparameter_meta
: a copy of the attributes from theparameter_meta
sectionSemantics:
wdl
variable is an objectwdl
variable is always present in the environmentwdl
(i.e.runtime
,hints
,meta
,paramater_meta
) are always defined and are objectswdl.runtime
are always defined. If the engine cannot determine the actual value of one of theruntime
attributes, it must populate the attribute with the requested value (or the default value, if no override was specified by the user).Beta Was this translation helpful? Give feedback.
All reactions