Creating a Generator Plugin¶

In this guide you will learn how to build a generator plugin.

Introduction¶

In this guide, you’ll learn about the basic components needed to create and use a packaged generator plugin. To focus this guide on the components needed to package the plugin, the functionality of the plugin’s code is minimal. The plugin prints “Hello {who-to-greet}”, using the provided custom name.

This guide uses the OpenTestFactory Orchestrator Toolkit module to speed up development. For more information, see the opentestfactory/python-toolkit repository.

Once you complete this project, you should understand how to build your own generator plugin and test it in a workflow.

To ensure your plugins are compatible with all OpenTestFactory Orchestrator deployments (Linux, Windows, …), the packaged code you write should be pure and not rely on other non-portable binaries.

Warning

When creating workflows and plugins, you should always consider whether your code might execute untrusted input from possible attackers. Certain contexts should be treated as untrusted input, as an attacker could insert their own malicious content. For more information, see “Understanding the risk of script injections.”

Prerequisites¶

You may find it helpful to have a basic understanding of OpenTestFactory Orchestrator environment variables:

Using environment variables

Before you begin, you will need to create a repository.

Create a new repository on GitHub/GitLab/BitBucket/.... You can choose any repository name or use “hello-world-generator-plugin” like this example.
Clone your repository to your computer.
From your terminal, change directories into your new repository.

cd hello-world-generator-plugin

The plugin toolkit package

The opentf-toolkit is a Python package that allow you to quickly build Python plugins with more consistency.

The opentf-toolkit core package provides an interface to the workflow commands, input and output variables, exit statuses, and debug messages.

The toolkit offers more than the core package. For more information, see the opentestfactory/python-toolkit repository.

At your terminal, install the opentf-toolkit package.

pip install --upgrade opentf-toolkit

Creating a Plugin Descriptor¶

Create a new plugin.yaml file in the hello-world-generator-plugin directory you created above. For more information, see “Descriptor syntax.”

plugin.yaml

# plugin.yaml
apiVersion: opentestfactory.org/v1alpha1
kind: GeneratorPlugin
metadata:
  name: hello
  description: Greet someone
  action: example/hello@v1
cmd: python -m main
events:
- categoryPrefix: example
  category: hello
  categoryVersion: v1
inputs:
  who-to-greet:
    description: Who to greet
    required: true

The name filed must match the plugin’s name (in this case, hello, as specified in the make_plugin() call below).

The action field is a unique identifier for your plugin’s generator. It is a string that follows the format prefix/name@version.

The events field is a list of events that your plugin listens for. There must be at least one item in this list, but there may be more, if your generator is known under more than one name.

Here, the categoryPrefix, category, and categoryVersion entries reflect your generator’s name, example/hello@v1.

The inputs field is a dictionary of input variables that your generator accepts.

Each input variable is a dictionary with the following fields, description and required.

If a given input is not required (as is the case here), you can provide a default value.

Here, your generator will have one required input, who-to-greet.

Creating a Web Service¶

Generator plugins are simple web services. They subscribe to specific events on startup, and publish events in response.

The opentf-toolkit module streamlines the process if you want to write your plugin in Python. For more information on doing things in a less assisted way, see (TODO) “Writing plugins the hard way.”

In your new hello-world-generator-plugin directory, create a new main.py file.

main.py

# main.py
from opentf.toolkit import make_plugin, run_plugin

from .implementation import generate

plugin = make_plugin(
    name='hello',
    description='A helloworld generator.',
    generator=generate
)

if __name__ == '__main__':
    run_plugin(plugin)

Writing the Plugin Code¶

Generator plugins must return a possibly empty collection of jobs. Each job has a name and a definition. For more information about the jobs syntax, see “Workflow syntax for OpenTestFactory Orchestrator.”

Job names and step IDs are local to the returned collection of jobs. They do not conflict with names and IDs used in the referring workflow.

At first glance, the following Python script example seems to use the who-to-greet input variable to print “Hello {who-to-greet}” in the log file.

implementation (BAD).py

# implementation (BAD).py

def generate(inputs):
  jobs = {
    'job1': {
      'runs-on': 'linux',
      'steps': [
        {
          'run': 'echo "Hello ' + inputs['who-to-greet'] + '".',
        },
      ]
    }
  }
  return jobs

Alas, while this may work great most of the time, it relies on user input (the ones who will use your plugin) without sanitizing it.

That’s something you should never do on production code.

As you just want to display the user’s input, if provided, you can use Python’s shlex.quote() function, or you can use the orchestrator’s verbatim variables.

implementation.py (GOOD)

# implementation.py (GOOD)

from shlex import quote


def generate(inputs):
  jobs = {
    'job1': {
      'runs-on': 'linux',
      'steps': [
        {
          'run': 'echo ' + quote('Hello ' + inputs['who-to-greet'] + '.'),
        },
      ]
    }
  }
  return jobs

Creating a README¶

To let people know how to use your plugin, you can create a README file. A README is most helpful when you plan to share your plugin publicly, but is also a great way to remind you or your team how to use the plugin.

In your hello-world-generator-plugin directory, create a README.md file that specifies the following information:

A detailed description of what the plugin does.
Required input arguments.
Optional input arguments.
Environment variables the plugin uses.
An example of how to use your plugin in a workflow.

README.md

# Hello world generator plugin

This function prints "Hello World" or "Hello" + the name of a person to greet to
the log.

## Inputs

### `who-to-greet`

The name of the person to greet.

## Example usage

my_job:
  generator: example/hello@v1
  with:
    who-to-greet: 'Mona the Octocat'

Commit your Changes¶

From your terminal, commit your plugin.yaml, implementation.py, main.py, and README.md files.

It is best practice to also add a version tag for releases of your plugin. For more information on versioning your plugin, see “About plugins.”

git add plugin.yaml implementation.py main.py README.md
git commit -m "My first plugin is ready"

Testing out your Plugin in a Workflow¶

Now you are ready to test your function out in a workflow. Plugins made using the opentf-toolkit need a configuration file that define the context in which the plugin will run. The default name for this configuration file is conf/hello.yaml (replace hello with the name of your plugin if you have used another name in your make_plugin() call above).

Note

If needed, you can override the default configuration file name by setting the --config command-line option.

conf/hello.yaml

apiVersion: opentestfactory.org/v1beta2
kind: ServiceConfig
current-context: allinone
contexts:
- context:
    port: 17785
    host: 0.0.0.0
    ssl_context: disabled
    trusted_authorities:
    - /etc/opentf/*
    enable_insecure_login: true
    eventbus:
        endpoint: http://127.0.0.1:38368
        token: reuse
  name: allinone

You may have to adjust the highlighted lines to match your environment. Your plugin must be able to reach your orchestrator’s event bus, and your orchestrator’s event bus must be able to reach your plugin.

You are then ready to start your plugin.

python main.py --context allinone

If everything went well, you should see the following message:

[2024-05-29 16:17:10,417] INFO in hello: Serving on http://127.0.0.1:17785

Now, create a new directory called .opentf/workflows in your hello-world-provider-plugin directory. In this directory, create a new file called demo.yaml with the following code.

.opentf/workflows/demo.yaml

metadata:
  name: my first generator
jobs:
  hello_world_job:
    runs-on: linux
    name: A job to say hello
    generator: example/hello@v1
    with:
      who-to-greet: "Mona the Octocat"

To run your workflow, execute the following command:

opentf-ctl run workflow .opentf/workflows/demo.yaml -w

Workflow 2272a80c-89e1-4f7f-ad7e-b61a7763dd63 is running.
Workflow my first generator
(running in namespace 'default')
[2024-07-30T12:15:00] [job 1af63c91-fa8e-407a-971c-8b2d272da689] Running generator job A job to say hello
[2024-07-30T12:15:01] [job fcbbd157-8258-42a7-a80f-162d650a63a9] Requesting execution environment providing ['linux'] in namespace 'default' for job 'job1'
[2024-07-30T12:15:01] [job fcbbd157-8258-42a7-a80f-162d650a63a9] Hello Mona the Octocat.
[2024-07-30T12:15:01] [job fcbbd157-8258-42a7-a80f-162d650a63a9] Releasing execution environment for job 'job1'
Workflow completed successfully.

Commit, Tag, Push¶