Skip to content

Understanding OpenTestFactory Orchestrator

Learn about the core concepts and various components of the OpenTestFactory orchestrator, and see an example that shows you how to add testing automation to your repository.

Overview

The OpenTestFactory orchestrator helps you automate testing tasks within your software development life cycle. Workflows are trigger-driven, meaning that you can run a series of commands after a specified trigger has occurred.

For example, every time someone creates a pull request for a repository, you can automatically run a command that executes a software testing script.

The components of the OpenTestFactory Orchestrator

Below is a list of the multiple orchestrator components that work together to run jobs. You can see how these components interact with each other.

Components Components

Workflows

A workflow is a configurable automated process that will run one or more jobs. Workflows are defined by a YAML file checked into your repository.

Workflows are defined in the .opentf/workflows directory in a repository, and a repository can have multiple workflows, each of which can perform a different set of tasks. For example, you can have one workflow to test pull requests, another workflow to test your application every time a release is created, and still another workflow that runs specific tests every time someone adds a label.

For more information about workflows, see “Using workflows.”

Triggers

A trigger is a specific activity that launches a workflow. For example, activity can originate from your CI toolchain when someone pushes a commit to a repository or when a pull request is created. You can also launch a workflow when an external event occurs.

Jobs

A job can either be a generator job or a set of steps that are executed in the same execution environment.

You can configure a job’s dependencies with other jobs; by default, jobs have no dependencies and run in parallel with each other. When a job takes a dependency on another job, it will wait for the dependent job to complete before it can run. For example, you may have multiple build jobs for different architectures that have no dependencies, and a packaging job that is dependent on those jobs. The build jobs will run in parallel, and when they have all completed successfully, the packaging job will run.

For more information about jobs, see “Using jobs.”

Generators

Generators jobs are standalone commands that produce jobs. You can create your own generators, or use and customize generators created by the OpenTestFactory orchestrator community. To use a generator in a workflow, you must include it as a job.

Steps

A step is an individual task that can run commands in a job. Each step is either a shell script that will be executed or a function that will be run. Steps are executed in order and are dependent on each other. Since each step is executed in the same execution environment, you can share data from one step to another. For example, you can have a step that builds your application followed by a step that tests the application that was built.

Functions

Functions are standalone commands that are combined into steps to create a job. Functions are the smallest portable building block of a workflow. You can create your own functions, or use and customize functions created by the OpenTestFactory community. To use a function in a workflow you must include it as a step.

For more information about functions, see “Creating plugins.”

Execution environments

An execution environment is a server that the orchestrator can talk to. You can use an execution environment that is specific to the system you want to test, or you can use a collection of generic execution environments that each hosts a specific testing framework you have code for. An execution environment listens for available jobs, runs one job at a time, and reports the progress, logs, and results back to the OpenTestFactory orchestrator.

Create an example workflow

The orchestrator uses YAML syntax to define the workflow. Each workflow is stored as a separate YAML file in your code repository, in a directory named .opentf/workflows.

  1. In your repository, create the .opentf/workflows directory to store your workflow files.

  2. In the .opentf/workflows directory, create a new file called learn-opentf-orchestrator.yaml and add the following code.

    .opentf/workflows/learn-opentf-orchestrator.yaml
     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    metadata:
      name: learn-opentf-orchestrator
    jobs:
      my-first-job:
        runs-on: [linux, robotframework]
        steps:
        - uses: actions/checkout@v2
          with:
            repository: https://github.com/robotframework/RobotDemo.git
        - uses: robotframework/robot@v1
          with:
            datasource: RobotDemo/keyboard_driven.robot
        - run: ls -l
    
  3. Commit these changes and push them to your repository.

Your new OpenTestFactory orchestrator workflow file is now installed in your repository and will run each time someone triggers the orchestrator on your repository.

Understanding the workflow file

To help you understand how YAML syntax is used to create a workflow file, this section explains each line of the introduction’s example:

Line Content Explanation
(1) metadata: Groups together the metadata elements.
(2) name: learn-opentf-orchestrator The name of the workflow.
(3) jobs: Groups together all the jobs that run in the learn-opentf-orchestrator workflow file.
(4) my-first-job: Defines the name of the my-first-hob job stored within the jobs section.
(5) runs-on: [linux, robotframework] Configures the job to run on an execution environment that has the linux and robotframework tags (it may have other tags too). If there is no such execution environment available, the orchestrator waits until one become available.
The workflow fails if none is found (after a configurable delay).
(6) steps: Groups together all the steps that run in the my-first-job job. Each item nested under this section is a separate function or shell command.
(7) uses: actions/checkout@v2 The uses keyword tells the job to retrieve v2 of the function named actions/checkout@v2. This is a function that checks out your repository and downloads it to the execution environment, allowing you to run actions against your code (such as testing tools). You must use the checkout function any time your workflow will run against the repository’s code.
(8) with: The with keyword provides parameters to the function.
(9) repository: https://github.com/robotframework/RobotDemo.git The actions/checkout@v2 function has a repository parameter that specifies the repository to check out.
(10) uses: robotframework/robot@v1 This function runs the specified Robot Framework datasource.
(11) with: The with keyword provides parameters to the function.
(12) datasource: RobotDemo/keyboard_driven.robot The roboframework/robot@v1 function has a datasource parameter that specifies the datasource to use.
(13) run: ls -l The run keyword tells the job to execute a script on the execution environment. In this case, you are using ls to show the content of the current directory.

Visualizing the workflow file

In this diagram, you can see the workflow file you just created and how the OpenTestFactory orchestrator components are organized in a hierarchy. Each step executes a single function or shell command. Steps 1 and 2 use prebuilt community functions. Step 3 runs shell commands directly on the execution environment. To find more prebuilt functions for your workflows, see “Finding and customizing plugins.”

flowchart TB
  subgraph job[Job: my-first-job]
    subgraph one[Step 1]
      step_1["Checkout repository<br/>-uses: actions/checkout@v2"]
    end
    subgraph two[Step 2]
      step_2["Run robotframework datasource<br/>-uses: robotframework/robot@v1"]
    end
    subgraph three[Step 3]
      step_3["Run shell command<br/>-run: ls -l"]
    end
  end

Next steps

To continue learning about the OpenTestFactory orchestrator, see “Finding and customizing plugins.”