Creating a Provider Plugin

This guide shows you the minimal steps required to build a provider plugin.

Introduction

In this guide, you will learn about the basic components needed to create and use a packaged provider plugin. To focus this guide on the components needed to package the plugin, the functionality of the plugin’s code is minimal. The plugin provides a function that prints “Hello World” in the logs or “Hello {who-to-greet}” if you provide a custom name.

This guide uses the OpenTestFactory Orchestrator Toolkit module to speed up development. For more information, see the opentestfactory/python-toolkit repository.

Once you complete this project, you should understand how to build your own provider plugin and test it in a workflow.

To ensure your plugins are compatible with all OpenTestFactory Orchestrator deployments (Linux, Windows, …), the packaged code you write should be pure and not rely on other non-portable binaries.

Warning

When creating workflows and plugins, you should always consider whether your code might execute untrusted input from possible attackers. Certain contexts should be treated as untrusted input, as an attacker could insert their own malicious content. For more information, see “Understanding the risk of script injections.”

Prerequisites

You may find it helpful to have a basic understanding of the OpenTestFactory Orchestrator environment variables:

• Using environment variables

Before you begin, you’ll need to create a repository.

1. Create a new repository on GitHub/GitLab/BitBucket/.... You can choose any repository name or use “hello-world-provider-plugin” like this example.
2. Clone your repository to your computer.
3. From your terminal, change directories into your new repository.

cd hello-world-provider-plugin

The plugin toolkit package

The opentf-toolkit is a Python package that allow you to quickly build Python plugins with more consistency.

The opentf-toolkit core package provides an interface to the workflow commands, input and output variables, exit statuses, and debug messages.

The toolkit offers more than the core package. For more information, see the opentestfactory/python-toolkit repository.

At your terminal, install the opentf-toolkit package.

pip install --upgrade opentf-toolkit

Creating a Plugin Descriptor

Create a new plugin.yaml file in the hello-world-provider-plugin directory with the following example code. For more information, see “Descriptor syntax.”

This file describes the function your plugin implements.

plugin.yaml

# plugin.yaml
apiVersion: opentestfactory.org/v1alpha1
kind: ProviderPlugin
metadata:
  name: greet
  description: Greet someone
  action: example/greet@v1
cmd: python -m main
events:
- categoryPrefix: example
  category: greet
  categoryVersion: v1
inputs:
  who-to-greets:
    description: Who to greet
    required: false
    default: World
outputs:
  random-id:
    description: Random number
    value: ${{ steps.random-number-generator.outputs.random-id }}

Tip

If your plugin implements more than one function, there should be one YAML document per function in your descriptor. (YAML documents are separated by ---.)

The name filed must match the plugin’s name (in this case, greet, as specified in the make_plugin() call below).

The action field is a unique identifier for your plugin’s function. It is a string that follows the format prefix/name@version.

The events field is a list of events that your plugin listens for. There must be at least one item in this list, but there may be more, if your function is known under more than one name.

Here, the categoryPrefix, category, and categoryVersion entries reflect your function’s name, example/greet@v1.

The inputs field is a dictionary of input variables that your function accepts.

Each input variable is a dictionary with the following fields, description and required.

If a given input is not required (as is the case here), you can provide a default value.

Here, your function will have one optional input, who-to-greets. If this input is not provided when calling your function, it will default to "World".

The outputs field is a dictionary of output variables that your function produces.

Here, your function will produce one output, random-id. This output will be a random number.

Creating a Web Service

The `opentf-toolkit` is a Python package that allow you to quickly
build Python plugins with more consistency.

The opentf-toolkit module streamlines the process if you want to write your plugin in Python. For more information on doing things in a less assisted way, see (TODO) “Writing plugins the hard way.”

In your new hello-world-provider-plugin directory, create a new file called main.py, with the following code.

main.py

from opentf.toolkit import make_plugin, run_plugin

from .implementation import handler

plugin = make_plugin(
    name='greet',
    description='A helloworld provider.',
    provider=handler,
)

if __name__ == '__main__':
    run_plugin(plugin)

Writing the Plugin Code

Provider plugin functions must return a possibly empty list of steps. Each step has a definition. For more information about the steps syntax, see “Workflow syntax for OpenTestFactory Orchestrator.”

The following Python script example uses the who-to-greet input variable to print “Hello {who-to-greet}” in the logs and maps the random generated number to the random-id output variable.

As the input variable has been described in the descriptor, you do not have to worry whether is has been specified by your users or not. The opentf-toolkit will take care of it for you.

Your plugin can work on windows and linux or macos execution environments. So, as the basic shell commands you need have slightly different syntax for each, you will need to write a conditional statement to handle the different execution environments.

At first glance, the following Python script example seems to use the who-to-greet input variable to pring “Hello {who-to-greet}.” in the log file.

implementation (BAD).py

# implementation (BAD).py

def handler(inputs):
    return [
        {
            'if': "runner.os == 'windows'",
            'run': 'echo Hello ' + inputs['who-to-greets'] + '.',
        },
        {
            'if': "runner.os != 'windows'",
            'run': 'echo "Hello ' + inputs['who-to-greets'] + '."',
        },
        {
            'if': "runner.os != 'windows'",
            'id': 'random-number-generator',
            'run': 'echo "::set-output name=random-id::$(echo $RANDOM)"',
        },
        {
            'if': "runner.os == 'windows'",
            'id': 'random-number-generator',
            'run': 'echo ::set-output name=random-id::%RANDOM%',
        },
    ]

Alas, while this may work great most of the time, it relies on user input (the ones who will use your plugin) without sanitizing it.

That’s something you should never do on production code.

As you just want to display the user’s input, if provided, you can use Python’s shlex.quote() or subprocess.list2cmdline() functions, or you can use the orchestrator’s verbatim variables.

implementation.py (GOOD)

# implementation.py (GOOD)

from shlex import quote
from subprocess import list2cmdline

def handler(inputs):
    return [
        {
            'if': "runner.os == 'windows'",
            'run': list2cmdline(['echo', 'Hello ' + inputs['who-to-greets'] + '.']),
        },
        {
            'if': "runner.os != 'windows'",
            'run': 'echo ' + quote('Hello ' + inputs['who-to-greets'] + '.'),
        },
        {
            'if': "runner.os != 'windows'",
            'id': 'random-number-generator',
            'run': 'echo "::set-output name=random-id::$(echo $RANDOM)"',
        },
        {
            'if': "runner.os == 'windows'",
            'id': 'random-number-generator',
            'run': 'echo ::set-output name=random-id::%RANDOM%',
        },
    ]

Tip

Writing the plugin code in a separate file is not mandatory, but it is a good practice to keep your code organized.

Creating a README

To let people know how to use your plugin, you can create a README file. A README is most helpful when you plan to share your plugin publicly, but is also a great way to remind you or your team how to use the plugin’s functions.

In your hello-world-provider-plugin directory, create a README.md file that specifies the following information:

• A detailed description of what the plugin’s function does.
• Required input and output arguments.
• Optional input and output arguments.
• Environment variables the function uses.
• An example of how to use your function in a workflow.

README.md

# Hello world function

This function prints "Hello World" or "Hello" + the name of a person to greet to
the log.

## Inputs

### `who-to-greets`

The name of the person to greet.  Default `"World"`.

## Outputs

### `random-id`

A random number.

## Example usage

- id: hello
  uses: example/greet@v1
  with:
    who-to-greets: 'Mona the Octocat'
- run: echo "the random number is ${{ steps.hello.outputs.random-id }}"

Commit your Changes

It is always a good idea to frequently save your changes. From your terminal, commit your plugin.yaml, implementation.py, main.py, and README.md files.

git add plugin.yaml implementation.py main.py README.md
git commit -m "My first plugin is hopefully ready"

Testing out your Plugin in a Workflow

Now you are ready to test your function out in a workflow. Plugins made using the opentf-toolkit need a configuration file that define the context in which the plugin will run. The default name for this configuration file is conf/greet.yaml (replace greet with the name of your plugin if you have used another name in your make_plugin() call above).

Note

If needed, you can override the default configuration file name by setting the --config command-line option.

conf/greet.yaml

apiVersion: opentestfactory.org/v1alpha1
kind: ProviderConfig
current-context: allinone
contexts:
- context:
    port: 7785
    host: 0.0.0.0
    ssl_context: disabled
    trusted_authorities:
    - /etc/opentf/*
    enable_insecure_login: true
    eventbus:
        endpoint: http://127.0.0.1:38368
        token: reuse
  name: allinone

You may have to adjust the highlighted lines to match your environment. Your plugin must be able to reach your orchestrator’s event bus, and your orchestrator’s event bus must be able to reach your plugin.

You are then ready to start your plugin.

python main.py --context allinone

If everything went well, you should see the following message:

[2024-05-29 16:17:10,417] INFO in greet: Serving on http://127.0.0.1:7785

Now, create a new directory called .opentf/workflows in your hello-world-provider-plugin directory. In this directory, create a new file called demo.yaml with the following code.

.opentf/workflows/demo.yaml

metadata:
  name: my first provider
jobs:
  hello_world_job:
    runs-on: linux
    name: A job to say hello
    steps:
    - id: hello
      uses: example/greet@v1
      with:
        who-to-greets: "Mona the Octocat"
    # Use the output from the hello step
    - run: echo "the random number is ${{ steps.hello.outputs.random-id }}"

To run your workflow, execute the following command:

opentf-ctl run workflow .opentf/workflows/demo.yaml -w

You should get something like this:

Workflow 95e5bff4-c73f-4a17-a2a8-448e5fd2ba34 is running.
Workflow my first provider
(running in namespace 'default')
[2024-05-29T16:49:34] [job 1ae04af5-a834-446e-98fc-6b356629dd4f] Requesting execution environment providing linux in namespace 'default' for job 'A job to say hello'
[2024-05-29T16:49:34] [job 1ae04af5-a834-446e-98fc-6b356629dd4f] Running function examplegreetv1
[2024-05-29T16:49:37] [job 1ae04af5-a834-446e-98fc-6b356629dd4f] Hello Mona the Octocat.
[2024-05-29T16:49:37] [job 1ae04af5-a834-446e-98fc-6b356629dd4f] Running command: echo "the rando...
[2024-05-29T16:49:37] [job 1ae04af5-a834-446e-98fc-6b356629dd4f] the random number is 30417
[2024-05-29T16:49:37] [job 1ae04af5-a834-446e-98fc-6b356629dd4f] Releasing execution environment for job 'A job to say hello'
Workflow completed successfully.

Commit, Tag, Push

From your terminal, commit your plugin.yaml, implementation.py, main.py, and README.md files.

It is best practice to also add a version tag for releases of your plugin. For more information on versioning your plugin, see “About plugins.”

git add plugin.yaml implementation.py main.py README.md
git commit -m "My first plugin is ready"
git tag -a -m "My first plugin release" v1
git push --follow-tags

Next Steps