Workflow syntax for OpenTestFactory orchestrator¶
Note
This workflow syntax closely matches GitHub Actions workflow syntax. If you are new to GitHub actions and want to learn more, see “Workflow syntax for GitHub Actions.”
A workflow is a configurable automated process made up of one or more jobs. You must create a YAML file to define your workflow configuration.
About YAML syntax for workflows¶
Workflow files must use YAML syntax and may have a .yml
or .yaml
file extension. If you are new to YAML and want to learn more, see “Learn YAML in five minutes.”
You may store workflow files anywhere in your repository.
Usage limits¶
There may be sizing limits on OpenTestFactory orchestrator workflows. The following are the minimum that must be supported.
- Jobs - There can be at least 1024 jobs per workflow (including generated jobs).
- Steps - There can be at least 1024 steps per job (including generated steps).
- Job matrix - A job matrix may generate at least 256 jobs per workflow run.
An implementation may provide lower minimums if it provides a way to extend its limits so that they reach or exceed the above minimums.
Mandatory and optional sections¶
A workflow must have at least a metadata
section and a jobs
section. It may have additional variables
, resources
, hooks
, and defaults
sections. Those additional sections may be empty or omitted if not used.
metadata:
name: mona
variables:
...
resources:
...
hooks:
...
defaults:
...
jobs:
...
metadata.name
¶
Required The name of the workflow. This name can be used by tools to provide a human-friendly reference.
metadata.namespace
¶
The name of the namespace the workflow is part of. If unspecified, the workflow will be part of the default
namespace.
variables
¶
A map of environment variables that are available to all jobs and steps in the workflow. You can also set environment variables that are only available to a job or step. For more information, see jobs.<job_id>.variables
and jobs.<job_id>.steps.variables
.
When more than one environment variable is defined with the same name, the orchestrator uses the most specific environment variable. For example, an environment variable defined in a step will override job and workflow variables with the same name, while the step executes. A variable defined for a job will override a workflow variable with the same name, while the job executes.
Variables specified in the workflow file are by default subject to the execution environment shell’s expansions and substitution rules. Use the verbatim: true
option or the target shell’s escape conventions if you need the literal content. For more information, see “Context and expression syntax for OpenTestFactory workflows.”
Examples¶
This first example defines a simple SERVER
variable, which is subject to the execution environment shell’s expansions and substitution rules.
variables:
SERVER: production
echo "$SERVER" # -> production
This second example defines three variables that are subject to the execution environment shell’s expansions and substitution rules, and a fourth one, NOT_INTERPRETED
, which is not.
INTERPRETED_1: The current directory is `pwd`
INTERPRETED_2:
value: The current directory is `pwd`
verbatim: false
INTERPRETED_3:
value: The current directory is `pwd`
NOT_INTERPRETED:
value: Use `pwd` to get the current directory
verbatim: true
echo "$INTERPRETED_1" # -> The current directory is /home/user
echo "$INTERPRETED_2" # -> The current directory is /home/user
echo "$INTERPRETED_3" # -> The current directory is /home/user
echo "$NOT_INTERPRETED" # -> Use `pwd` to get the current directory
resources
¶
A map of resources that are available to all jobs and steps in the workflow. You can define
three kinds of resources: testmanagers
, repositories
, and files
.
testmanagers
and repositories
resources are a way to reuse resources in your
workflow. files
resources allow providing local or external files to your workflow.
resources.repositories
¶
An array of source code repository definitions. Each repository definition is a map and must have the name
, type
, repository
, and endpoint
entries.
Example of a repository resource¶
This example defines a myrepo
repository:
resources:
repositories:
- name: myrepo
type: bitbucket
repository: example/my-example-repo
endpoint: https://bitbucket.org
resources.repositories.name
¶
Required The name of the repository, which is used to refer to the definition. It must be unique among the repository definitions.
resources.repositories.type
¶
Required The type of the repository. git
, github
, gitlab
, and bitbucket
must be supported.
resources.repositories.repository
¶
Required The reference of the repository.
resources.repositories.endpoint
¶
Required The endpoint to use to access the source code manager.
resources.files
¶
An array of externally-provided files.
Example of a file resource¶
This example defines one file. This file will have to be joined when launching the workflow:
resources:
files:
- dataset
The following function will put a copy of the provided file in the current execution environment so that the following steps can use it:
- uses: actions/put-file@v1
with:
file: dataset
path: dataset.xml
hooks
¶
An array of hook definitions that apply to all jobs and steps in the workflow, including generated ones.
Each hook definition is a map. It specifies the events that trigger it, and the actions it performs when triggered. It may also contain an if
conditional that further limit its scope.
Hooks specified in a workflow complement the hooks defined by installed provider plugins. Their before
actions are performed before the ones defined by the provider plugins and their after
actions are performed after the ones defined by the provider plugins.
before_steps from workflow-defined hook 1
before_steps from workflow-defined hook 2
before_steps from provider-defined hook
steps from function
after_steps from provider-defined hook
after_steps from workflow-defined hooks 2
after_steps from workflow-defined hooks 1
hooks.name
¶
Required The name of the hook, as displayed by the orchestrator.
Example¶
hooks:
- name: my-hook
hooks.events
¶
Required A non-empty array of events that trigger the hook. Events are described by their categoryPrefix
, category
, and categoryVersion
labels.
You can use the _
placeholder to match any category
or categoryPrefix
if you have not specified the other one. You can specify a given categoryVersion
, but this must not be the only specifier.
Examples¶
This hook will be triggered by my-plugin/foo
, my-plugin/bar
, and my-other-plugin/baz
:
hooks:
- name: my-fishy-hook
events:
- categoryPrefix: my-plugin
- categoryPrefix: my-other-plugin
category: _
All providers will trigger this hook whenever they are used (either explicitly in the workflow, or in generated jobs or steps):
hooks:
- name: my-global-hook
events:
- category: _
hooks.before
¶
Required if hooks.after
not specified The steps to include before performing the triggering task.
Example¶
hooks:
- name: foo
events:
- category: _
before:
- run: echo foo!
hooks.after
¶
Required if hooks.before
not specified The steps to include after performing the triggering task.
Example¶
hooks:
- name: my before/after hook
events:
- categoryPrefix: my-plugin
category: my-task
before:
- run: echo before my-plugin/my-task
after:
- run: echo after my-plugin/my-task
hooks.if
¶
You can use the if
conditional to prevent a hook from being triggered unless a condition is met. You can use any supported context and expression to create a conditional.
When you use expressions in an if
conditional, you may omit the expression syntax (${{ }}
) because the orchestrator automatically evaluates the if
conditional as an expression. For more information, see “Context and expression syntax for OpenTestFactory workflows.”
defaults
¶
A map of default settings that will apply to all jobs in the workflow. You can also set default settings that are only available to a job. For more information, see jobs.<job_id>.defaults
.
When more than one default setting is defined with the same name, the orchestrator uses the most specific default setting. For example, a default setting defined in a job will override a default setting that has the same name defined in a workflow.
defaults.run
¶
You can provide default shell
and working-directory
options for all run
steps in a workflow. You can also set default settings for run
that are only available to a job. For more information, see jobs.<job_id>.defaults.run
. You cannot use contexts or expressions in this keyword.
When more than one default setting is defined with the same name, the orchestrator uses the most specific default setting. For example, a default setting defined in a job will override a default setting that has the same name defined in a workflow.
Example¶
defaults:
run:
shell: bash
working-directory: scripts
jobs
¶
A workflow run is made up of one or more jobs. Jobs run in parallel by default. To run jobs sequentially, you can define dependencies on other jobs using the jobs.<job_id>.needs
keyword.
Each job runs in an execution environment specified by runs-on
.
You can run an unlimited number of jobs as long as you are within the workflow usage limits. For more information, see “Usage limits.”
jobs.<job_id>
¶
Each job must have an ID to associate with the job. The key job_id
is a string and its value is a map of the job’s configuration data. You must replace <job_id>
with a string that is unique to the jobs object. The <job_id>
must start with a letter or _
and contain only alphanumeric characters, -
, or_
.
Example¶
jobs:
my_first_job:
name: My first job
my_second_job:
name: My second job
jobs.<job_id>.name
¶
The name of the job displayed by the orchestrator.
jobs.<job_id>.needs
¶
Identifies any jobs that must complete successfully before this job will run. It can be a string or an array of strings. If a job fails, all jobs that need it are skipped unless the jobs use a conditional statement that causes the job to continue.
Example¶
jobs:
job1:
job2:
needs: job1
job3:
needs: [job1, job2]
In this example, job1
must complete successfully before job2
begins, and job3
waits for both job1
and job2
to complete.
The jobs in this example run sequentially:
job1
job2
job3
jobs.<job_id>.runs-on
¶
Required The environment to run the job on.
Examples¶
runs-on: robotframework
runs-on: [linux, robotframework]
Each execution environment has one or more tags. So, you can prescribe where the job should be run by specifying a single tag or an array of tags: an environment will be selected for the execution only if it has (at least, it can have some other additional ones) these specified tags.
For more information, see Execution environments.
jobs.<jobs_id>.outputs
¶
A map of outputs for a job. Job outputs are available to all downstream jobs that depend on this job. For more information on defining job dependencies, see jobs.<job_id>.needs
.
Job outputs are strings, and job outputs containing expressions are evaluated at the end of each job.
To use job outputs in a dependent job, you can use the needs
context. For more information, see “Context and expression syntax for OpenTestFactory workflows.”
Example¶
jobs:
job1:
runs-on: ubuntu-latest
# Map a step output to a job output
outputs:
output1: ${{ steps.step1.outputs.test }}
output2: ${{ steps.step2.outputs.test }}
steps:
- id: step1
run: echo "::set-output name=test::hello"
- id: step2
run: echo "::set-output name=test::world"
job2:
runs-on: ubuntu-latest
needs: job1
steps:
- run: echo ${{needs.job1.outputs.output1}} ${{needs.job1.outputs.output2}}
jobs.<job_id>.variables
¶
A map of environment variables that are available for all steps in the job. You can also set environment variables for the entire workflow or an individual step. For more information, see variables
and jobs.<job_id>.steps.variables
.
When more than one environment variable is defined with the same name, the orchestrator uses the most specific environment variable. For example, an environment variable defined in a step will override job and workflow variables with the same name, while the step executes. A variable defined for a job will override a workflow variable with the same name, while the job executes.
Variables specified in the workflow file are by default subject to the execution environment shell’s expansions and substitution rules. Use the verbatim: true
option or the target shell’s escape conventions if you need the literal content. For more information, see “Context and expression syntax for OpenTestFactory workflows.”
Example¶
jobs:
job1:
variables:
FIRST_NAME: Mona
LAST_NAME:
value: O'Hare
verbatim: true
jobs.<job_id>.defaults
¶
Use jobs.<job_id>.defaults
to create a map of default settings that will apply to all steps in the job. You can also set default settings for the entire workflow. For more information, see defaults
.
When more than one default setting is defined with the same name, the orchestrator uses the most specific default setting. For example, a default setting defined in a job will override a default setting that has the same name defined in a workflow.
jobs.<job_id>.defaults.run
¶
Provide default shell
and working-directory
to all run steps in the job. Context and expression are not allowed in this section.
You can provide default shell
and working-directory
options for all run
steps in a job. You can also set default settings for run
for the entire workflow. For more information, see defaults.run
. You cannot use contexts or expressions in this keyword.
When more than one default setting is defined with the same name, the orchestrator uses the most specific default setting. For example, a default setting defined in a job will override a default setting that has the same name defined in a workflow.
Example¶
jobs:
job1:
runs-on: ubuntu-latest
defaults:
run:
shell: bash
working-directory: scripts
jobs.<job_id>.if
¶
You can use the if
conditional to prevent a job from running unless a condition is met. You can use any supported context and expression to create a conditional.
When you use expressions in an if
conditional, you may omit the expression syntax (${{ }}
) because the orchestrator automatically evaluates the if
conditional as an expression. For more information, see “Context and expression syntax for OpenTestFactory workflows.”
jobs.<job_id>.generator
¶
Selects a generator to run. A generator creates a series of jobs that replace the current job.
We strongly recommend that you include the version of the generator you are using. If you do not specify a version, it could break your workflow or cause unexpected behavior when the generator owner publishes an update.
- Using the specific major generator version allows you to receive critical fixes and security patches while still maintaining compatibility. It also assures that your workflow should still work.
Generators may require inputs that you must set using the with
keyword. Review the generator’s README file to determine the inputs required.
Example using versioned generators¶
jobs:
my_job:
# Reference the major version of a release
generator: foobar/hello_world@v1
my_second_job:
# Reference a minor version of a release
generator: foobar/hello_world@v1.2
jobs.<job_id>.with
¶
A map of the input parameters defined by the generator plugin. Each input parameter is a key/value pair.
Example¶
Defines the three input parameters (first_name
, middle_name
, and last_name
) defined by the hello_world
generator.
jobs:
generator: foobar/hello_world@v1
with:
first_name: Mona
middle_name: The
last_name: Octocat
jobs.<job_id>.steps
¶
A job contains a sequence of tasks called steps
. Steps can run commands or functions. Not all steps run functions, but all functions run as a step. Each step runs in its own process in the execution environment and has access to the workspace and file system. Because steps run in their own process, changes to environment variables are not preserved between steps. The orchestrator provides built-in steps to set up and complete a job.
You can run an unlimited number of steps as long as you are within the job usage limits. For more information, see “Usage limits.”
Example¶
metadata:
name: Greeting from Mona
jobs:
my-job:
name: My Job
runs-on: ubuntu-latest
steps:
- name: Print a greeting
variables:
MY_VAR: Hi there! My name is
FIRST_NAME: Mona
MIDDLE_NAME: The
LAST_NAME: Octocat
run: |
echo $MY_VAR $FIRST_NAME $MIDDLE_NAME $LAST_NAME.
jobs.<job_id>.steps[*].id
¶
A unique identifier for the step. You can use the id
to reference the step in contexts. For more information, see “Context and expression syntax for OpenTestFactory workflows.”
jobs.<job_id>.steps[*].if
¶
You can use the if
conditional to prevent a step from running unless a condition is met. You can use any supported context and expression to create a conditional.
When you use expressions in an if
conditional, you may omit the expression syntax (${{ }}
) because the orchestrator automatically evaluates the if
conditional as an expression. For more information, see “Context and expression syntax for OpenTestFactory workflows.”
Example using contexts¶
This step only runs when the runner OS is Windows.
steps:
- name: My first step
if: ${{ runner.os == 'windows' }}
run: echo This step is running on windows.
Example using status check functions¶
The “My backup step” step only runs when the previous step of a job fails. For more information, see “Context and expression syntax for OpenTestFactory workflows.”
steps:
- name: My first step
uses: monacorp/function-name@v1
- name: My backup step
if: ${{ failure() }}
uses: actions/heroku@v2
jobs.<job_id>.steps[*].name
¶
A name for your step to display on the orchestrator.
jobs.<job_id>.steps[*].uses
¶
Selects a function to run as part of a step in your job. A function is a reusable unit of code.
We strongly recommend that you include the version of the function you are using. If you do not specify a version, it could break your workflow or cause unexpected behavior when the provider owner publishes an update.
- Using the specific major function version allows you to receive critical fixes and security patches while still maintaining compatibility. It also assures that your workflow should still work.
Functions may require inputs that you must set using the with
keyword. Review the provider’s README file to determine the inputs required.
Providers are plugins that add functions. For more details, see “Providers” chapter.
Example using versioned functions¶
steps:
# Reference the major version of a release
- uses: actions/setup-node@v1
# Reference a minor version of a release
- uses: actions/setup-node@v1.2
jobs.<job_id>.steps[*].run
¶
Runs command-line programs using the execution environment’s shell. If you do not provide a name
, the step name will default to the text specified in the run
command.
Commands run using non-login shells by default. You can choose a different shell and
customize the shell used to run commands. For more information, see “jobs.<job_id>.steps[*].shell
.”
Each run
keyword represents a new process and shell in the execution environment. When you provide multi-line commands, each line runs in the same shell. For example:
- A single-line command:
- name: Install Dependencies
run: npm install
- A multi-line command:
- name: Clean install dependencies and build
run: |
npm ci
npm run build
Using the working-directory
keyword, you can specify the working directory of where to run the command.
- name: Clean temp directory
run: rm -rf *
working-directory: ./temp
jobs.<job_id>.steps[*].shell
¶
You can override the default shell settings in the execution environment using
the shell
keyword. You can use built-in shell
keywords, or you can define a
custom set of shell options. The shell command that is run internally executes
a temporary file that contains the commands specified in the run
keyword.
Supported platform | shell parameter |
Description | Command run internally |
---|---|---|---|
All | bash |
The default shell on non-Windows platforms. When specifying a bash shell on Windows, the bash shell included with Git for Windows is used. | bash --noprofile --norc -eo pipefail {0} |
All | python |
Executes the python command. | python {0} |
All | pwsh |
The PowerShell Core. The orchestrator appends the extension .ps1 to your script name. |
pwsh -command ". '{0}'" |
Windows | cmd |
The default shell on Windows. The orchestrator appends the extension .cmd to your script name and substitutes for {0} . |
%ComSpec% /D /E:ON /V:OFF /S /C "CALL "{0}"" |
Windows | powershell |
The PowerShell Desktop. The orchestrator appends the extension .ps1 to your script name. |
powershell -command ". '{0}'" |
For pwsh
and python
, the command used must be installed in the execution environment.
For powershell
, the execution policy must allow script execution on the execution
environment. See “about_Execution_Policies” for more information.
Example running a script using bash¶
steps:
- name: Display the path
run: echo $PATH
shell: bash
Example running a script using Windows cmd
¶
steps:
- name: Display the path
run: echo %PATH%
shell: cmd
Example running a python script¶
steps:
- name: Display the path
run: |
import os
print(os.environ['PATH'])
shell: python
Custom shell¶
You can set the shell
value to a template string using
command [options...] {0} [more_options...]
The orchestrator interprets the first whitespace-delimited word of the string as the command and inserts the file name for the temporary script at {0}
.
For example:
steps:
- name: Display the environment variables and their values
run: |
print %ENV
shell: perl {0}
The command used, perl
in this example, must be installed in the execution environment.
Exit codes and error action preference¶
For built-in shell keywords, we provide the following defaults that are executed by the execution environments. You should use these guidelines when running shell scripts.
-
bash
:- Fail-fast behavior using
set -e o pipefail
: Default for bash and built-in shell. It is also the default when you do not provide an option on non-Windows platforms. - You can opt out of fail-fast and take full control by providing a template string to the shell options. For example,
bash {0}
. - sh-like shells exit with the exit code of the last command executed in a script, which is also the default behavior for functions. The execution environment will report the status of the step as fail/succeed based on this exit code.
- Fail-fast behavior using
-
powershell
/pwsh
:- Fail-fast behavior when possible. For
pwsh
andpowershell
built-in shell, we will prepend$ErrorActionPreference = 'stop'
to script contents.
- Fail-fast behavior when possible. For
-
cmd
- There does not seem to be a way to fully opt into fail-fast behavior other than writing your script to check each error code and respond accordingly. Because we can’t provide that behavior by default, you need to write this behavior into your script.
cmd.exe
will exit with the error level of the last program it executed, and it will return the error code to the execution environment.
jobs.<job_id>.steps[*].with
¶
A map of the input parameters defined by the function. Each input parameter is a key/value pair. Input parameters are set as environment variables. The variable is prefixed with INPUT_
and converted to upper case.
Example¶
Defines the three input parameters (first_name
, middle_name
, and last_name
) defined by the hello_world
function. These input variables will be accessible to the hello-world
function as INPUT_FIRST_NAME
, INPUT_MIDDLE_NAME
, and INPUT_LAST_NAME
environment variables.
jobs:
my_first_job:
steps:
- name: My first step
uses: actions/hello_world@master
with:
first_name: Mona
middle_name: The
last_name: Octocat
jobs.<job_id>.steps[*].variables
¶
Sets environment variables for steps to use in the execution environment. You can also set environment variables for the entire workflow or a job. For more information, see variables
and jobs.<job_id>.variables
.
When more than one environment variable is defined with the same name, the orchestrator uses the most specific environment variable. For example, an environment variable defined in a step will override job and workflow variables with the same name, while the step executes. A variable defined for a job will override a workflow variable with the same name, while the job executes.
Variables specified in the workflow file are by default subject to the execution environment shell’s expansions and substitution rules. Use the verbatim: true
option or the target shell’s escape conventions if you need the literal content. For more information, see “Context and expression syntax for OpenTestFactory workflows.”
Example¶
steps:
- name: My first function
variables:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
FIRST_NAME: Mona
LAST_NAME:
value: O'ctocat
verbatim: true
jobs.<job_id>.steps[*].continue-on-error
¶
Prevents a job from failing when a step fails. Set to true
to allow a job to pass when this step fails.
jobs.<job_id>.steps[*].timeout-minutes
¶
The maximum number of minutes to run the step before killing the process.
jobs.<job_id>.timeout-minutes
¶
The maximum number of minutes to let a job run before the orchestrator automatically cancels it. Default: 360
jobs.<job_id>.continue-on-error
¶
Prevents a workflow run from failing when a job fails. Set to true
to allow a workflow run to pass when this job fails.