Skip to content

Quickstart for OpenTestFactory Orchestrator

Add an OpenTestFactory Orchestrator workflow to an existing repository in 5 minutes or less.

Introduction

You only need an existing OpenTestFactory Orchestrator service to create and run a workflow. In this guide, you will add a workflow that run tests using the robotframework/robot provider. The workflow uses Robot Framework to run a set of tests in an existing execution environment.

Installation

  1. Prepare a Robot Framework execution environment

    This is a docker image that has Robot Framework installed, so that you can run Robot Framework tests easily.

    docker run -d \
      --name robotenv \
      --env PASSWORD_ACCESS=true \
      --env USER_NAME=otf \
      --env USER_PASSWORD=secret \
      opentestfactory/robot-framework-runner:latest
    
    docker run -d ^
      --name robotenv ^
      --env PASSWORD_ACCESS=true ^
      --env USER_NAME=otf ^
      --env USER_PASSWORD=secret ^
      opentestfactory/robot-framework-runner:latest
    
    docker run -d `
      --name robotenv `
      --env PASSWORD_ACCESS=true `
      --env USER_NAME=otf `
      --env USER_PASSWORD=secret `
      opentestfactory/robot-framework-runner:latest
    
  2. Prepare an OpenTestFactory Orchestrator that is linked to this execution environment

    docker run -d \
      --name orchestrator \
      --link=robotenv \
      -p 7774:7774 \
      -p 7775:7775 \
      -e SSH_CHANNEL_HOST=robotenv \
      -e SSH_CHANNEL_USER=otf \
      -e SSH_CHANNEL_PASSWORD=secret \
      -e SSH_CHANNEL_TAGS=ssh,linux,robotframework \
      -e SSH_CHANNEL_PORT=22 \
      opentestfactory/allinone:latest
    
    docker run -d ^
      --name orchestrator ^
      --link=robotenv ^
      -p 7774:7774 ^
      -p 7775:7775 ^
      -e SSH_CHANNEL_HOST=robotenv ^
      -e SSH_CHANNEL_USER=otf ^
      -e SSH_CHANNEL_PASSWORD=secret ^
      -e SSH_CHANNEL_TAGS=ssh,linux,robotframework ^
      -e SSH_CHANNEL_PORT=22 ^
      opentestfactory/allinone:latest
    
    docker run -d `
      --name orchestrator `
      --link=robotenv `
      -p 7774:7774 `
      -p 7775:7775 `
      -e SSH_CHANNEL_HOST=robotenv `
      -e SSH_CHANNEL_USER=otf `
      -e SSH_CHANNEL_PASSWORD=secret `
      -e SSH_CHANNEL_TAGS=ssh,linux,robotframework `
      -e SSH_CHANNEL_PORT=22 `
      opentestfactory/allinone:latest
    
  3. Get the generated token

    docker logs orchestrator 2>&1 \
        | grep --after-context=10 "Creating temporary JWT token"
    
    :: CMD does not display context around a pattern, so the easiest way is
    :: to use the `more` command to display the log page per page.  The
    :: token is typically in the first few pages.
    docker logs orchestrator 2>&1 | more
    
    docker logs orchestrator 2>&1 `
        | Select-String -Pattern 'Creating temporary JWT token' -Context 1,10
    

    Your token is displayed, surrounded by quotes:

    What a token looks like in logs

    In this example, the token starts with eyJ and ends with RPU. It will be different for you.

  4. Assign this token value to an environment variable

    export TOKEN=eyJ...
    
    set TOKEN=eyJ...
    
    $Env:TOKEN = "eyJ..."
    

Create your first workflow

You need test scripts to run. In a real-world case, you would use your own application source code repository, and your tests source code would be in another source code repository.

Here, you will fork an existing public repository that contains a set of Robot Framework scripts, https://github.com/robotframework/RobotDemo. (If you have a GitHub account, you can simply go to this repository and click on the ‘Fork’ button on the upper left of the page. If you use another git hosting platform, please refer to its documentation. Be sure to make your forked directory public and give it the ‘RobotDemo’ name.)

In the following steps, replace https://your-scm-manager/RobotDemo.git by the appropriate URL for your forked repository.

  1. Clone your repository in a local directory.

    git clone https://your-scm-manager/RobotDemo.git
    
  2. In this repository, create a .opentf/workflows directory.

    Your application source code repository is the best place to put your OpenTestFactory workflows on, so that they may change as your application grows.

    The .opentf/workflows name is not mandatory but grouping things is a good practice.

    cd RobotDemo
    mkdir -p .opentf/workflows
    
    cd RobotDemo
    mkdir .opentf\workflows
    
    cd RobotDemo
    mkdir .opentf/workflows
    
  3. Copy the following YAML content in the .opentf/workflows/workflow.yaml file (be sure to update the repository URL).

    .opentf/workflows/workflow.yaml
    metadata:
      name: Robot Framework Example
    variables:
      GREETINGS: hello world
    jobs:
      # Set the job key.  The key is displayed as the job name when
      # a job name is not provided
      keyword-driven:
        runs-on: robotframework
        steps:
        - run: echo $GREETINGS
        - uses: actions/checkout@v2
          with:
            repository: https://your-scm-manager/RobotDemo.git
        - uses: robotframework/robot@v1
          with:
            datasource: RobotDemo/keyword_driven.robot
    

    In a realistic case you would then commit your changes and then push them to your git repository.

  4. To run your workflow, run the following command.

    curl -X POST \
      --data-binary @.opentf/workflows/workflow.yaml \
      -H "Authorization: Bearer $TOKEN" \
      -H "Content-type: application/x-yaml" \
      http://localhost:7774/workflows
    
    curl -X POST ^
      --data-binary @.opentf/workflows/workflow.yaml ^
      -H "Authorization: Bearer %TOKEN%" ^
      -H "Content-type: application/x-yaml" ^
      http://localhost:7774/workflows
    
    curl.exe -X POST `
      --data-binary @.opentf/workflows/workflow.yaml `
      -H "Authorization: Bearer $Env:TOKEN" `
      -H "Content-type: application/x-yaml" `
      http://localhost:7774/workflows
    

    It will give you something like the following:

    {
        "apiVersion": "v1",
        "code": 201,
        "details": {
            "workflow_id": "..."
        },
        "kind": "Status",
        "message": "Workflow Robot Framework Example accepted (workflow_id=...).",
        "metadata": {},
        "reason": "Created",
        "status": "Success"
    }
    

    Again, in a realistic case, this command would be run by your CI toolchain, each time your git repository changes.

The orchestrator will then execute your workflow.

View your workflow results

  1. Check the logs of the orchestrator. This is a place where you observe your workflow executions.

    curl \
      -H "Authorization: Bearer $TOKEN" \
      http://localhost:7775/workflows
    
    curl ^
      -H "Authorization: Bearer %TOKEN%" ^
      http://localhost:7775/workflows
    
    curl.exe `
      -H "Authorization: Bearer $Env:TOKEN" `
      http://localhost:7775/workflows
    

    This lists all running and recently running workflows:

    {
        "apiVersion": "v1",
        "code": 200,
        "details": {
            "items": [
                "b2f27d1b-b947-4a7f-8627-5a3bdd73103e"
            ]
        },
        "kind": "Status",
        "message": "Running and recent workflows",
        "metadata": {},
        "reason": "OK",
        "status": "Success"
    }
    
  2. You can then query the orchestrator to get the details of a specific workflow execution.

    In the following command, replace ... by the relevant workflow ID (b2f...03e in the example above, but your workflow ID will differ).

    curl \
      -H "Authorization: Bearer $TOKEN" \
      http://localhost:7775/workflows/.../status
    
    curl ^
      -H "Authorization: Bearer %TOKEN%" ^
      http://localhost:7775/workflows/.../status
    
    curl.exe `
      -H "Authorization: Bearer $Env:TOKEN" `
      http://localhost:7775/workflows/.../status
    

    This will return a (quite indigest) JSON document, detailing the workflow execution.

This usage of the curl command is handy for a quick test, but in a more realistic case you would use a tool such as opentf-ctl, which would provide a much nicer experience.

> opentf-ctl run workflow .opentf/workflows/workflow.yaml --wait --token $TOKEN
Workflow c6f5f5ab-b115-416b-9165-08a055cb44df is running.
Workflow Robot Framework Example
(running in namespace 'default')
[2022-08-22T14:14:47] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] Requesting execution environment providing ['ssh', 'robotframework'] in namespace 'default' for job 'keyword-driven'
[2022-08-22T14:14:47] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] Running command: echo $GREETINGS
[2022-08-22T14:14:47] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] hello world
[2022-08-22T14:14:47] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] Running action actionscheckoutv2
[2022-08-22T14:14:48] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] Cloning into 'RobotDemo'...
[2022-08-22T14:14:48] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] Running action robotframeworkrobotv1
[2022-08-22T14:14:56] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] ==============================================================================
[2022-08-22T14:14:56] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] Keyword Driven :: Example test cases using the keyword-driven testing appro...
[2022-08-22T14:14:56] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] ==============================================================================
[2022-08-22T14:14:56] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] Push button                                                           | PASS |
[2022-08-22T14:14:56] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] ------------------------------------------------------------------------------
[2022-08-22T14:14:56] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] Push multiple buttons                                                 | PASS |
[2022-08-22T14:14:56] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] ------------------------------------------------------------------------------
[2022-08-22T14:14:56] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] Simple calculation                                                    | PASS |
[2022-08-22T14:14:56] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] ------------------------------------------------------------------------------
[2022-08-22T14:14:56] [job 183d1eb0-8fbf-44bb-9ccc-c8e4bd00eec8] Longer calculation                                                    | PASS |
...
Workflow completed successfully.

Please refer to “Running commands” for more on opentf-ctl.

More workflow templates

The OpenTestFactory provides preconfigured workflow templates that you can customize to create your own testing workflows.

You can browse the full list of workflow templates in the opentestfactory/templates repository.

Next steps

The Robot Framework workflow you just added can be integrated in your CI/CD toolchain to run any time code is pushed to your repository to help you spot errors and inconsistencies in your code. But this is only the beginning of what you can do with the OpenTestFactory orchestrator. Your repository can contain multiple workflows that trigger different jobs based on different events. The orchestrator can help you automate nearly every aspect of your application test processes. Ready to get started? Here are some helpful resources for taking your next steps with the OpenTestFactory orchestrator: