S3 publisher plugin¶

This publisher plugin uploads reports and attachments to any S3-compatible bucket.

It requires you to define S3 credentials and a target bucket. It is enabled by default but will do nothing if those elements are not defined properly.

It exposes no user-facing endpoints.

There are two service-specific configuration file options.

Environment variables¶

Logging¶

You can set the S3PUBLISHER_DEBUG_LEVEL (all upper-cased) or DEBUG_LEVEL environment variables to DEBUG to add additional information in the console for the launched service. It defaults to INFO. (Please note that setting DEBUG_LEVEL to DEBUG will produce tons of logs.)

The possible values are NOTSET, DEBUG, INFO, WARNING, ERROR, and FATAL. Those values are from the most verbose, NOTSET, which shows all logs, to the least verbose, FATAL, which only shows fatal errors.

If S3PUBLISHER_DEBUG_LEVEL is not defined then the value of DEBUG_LEVEL is used (or INFO if DEBUG_LEVEL is not defined either).

Access logs are only shown at NOTSET and DEBUG levels.

Configuration options¶

You can also set the two configuration options using environment variables:

S3PUBLISHER_BUCKET sets the bucket name.
S3PUBLISHER_S3CREDENTIALS sets the path to the S3 credentials file.

If those environment variables are defined, they override the values in the configuration file.

Configuration file¶

This plugin has a configuration file (s3publisher.yaml by default) that describes the common service configuration options.

If no configuration file is found it will default to the following values:

apiVersion: opentestfactory.org/v1beta2
kind: ServiceConfig
current-context: default
contexts:
- name: default
  context:
    port: 443
    host: 127.0.0.1
    ssl_context: adhoc
    eventbus:
      endpoint: https://127.0.0.1:38368
      token: invalid-token
    bucket: example
    s3credentials: s3credentials.json

The configuration included in the ‘allinone’ image is described in “Common settings.” The listening port is 7787 and the bind address is 127.0.0.1 as the service exposes no user-facing endpoints.

There are two service-specific configuration options besides the common ones that must be defined for the plugin to work properly: bucket and s3credentials.

Buckets¶

The bucket entry in the context specifies a target bucket name. Attachments will be stored in this bucket, under a {workflow_id}/{filename} key.

S3 credentials¶

The specified S3 credentials file must contain 4 entries:

region_name: a string
endpoint_url: a string
aws_access_key_id: a string
aws_secret_access_key: a string

It may contain additional entries, which are not used.

If the specified credential file is missing at startup time, attachments will be ignored by this publication plugin until a proper set of credentials is provided.

If the credential file changes, there is no need to restart the plugin: it will detect the changes and use the new set of credentials.

Example S3 credential file

{
  "region_name": "fr-par",
  "endpoint_url": "https://s3.example.com",
  "aws_access_key_id": "my_access_key_id",
  "aws_secret_access_key": "my_secret_access_key"
}

Subscriptions¶

The S3 publisher plugin subscribes to the following events:

`kind`	`apiVersion`
`ExecutionResult`	opentestfactory.org/v1alpha1
`WorkflowResult`	opentestfactory.org/v1alpha1

The S3 publisher plugin exposes an /inbox endpoint that is used by the event bus to post relevant events.

Launch command¶

If you want to manually start the S3 publisher plugin, use the following command:

python -m opentf.plugins.s3publisher.main [--context context] [--config configfile]

Additional command-line options are available and described in “Command-line options.”