Skip to content

Quality gate service

A service that is used to check the quality gate status of workflows. Exposes one user-facing endpoint that can be queried to get quality gate info on workflows.

A quality gate is a set of rules a workflow execution must satisfy.

Configuration

This module has a configuration file (qualitygate.yaml by default) that describes the host, port, ssl_context, trusted_authorities, and logfile to use. It can also enable insecure logins.

If no configuration file is found it will default to the following values:

apiVersion: opentestfactory.org/v1beta2
kind: ServiceConfig
current-context: default
contexts:
- context:
    port: 443
    host: 127.0.0.1
    ssl_context: adhoc
    services:
      observer:
        endpoint: https://127.0.0.1
        token: reuse
  name: default

ssl_context is either adhoc, a list of two items (certificate file path and private key file path), or disabled (not recommended, will switch to plain HTTP).

A context can also contain a trusted_authorities, which is a list of public key files, used for token validation.

A context can also allow for insecure (token-less) logins, if enable_insecure_login is set to true (by default, insecure logins are disabled).

Insecure logins, if enabled, are only allowed from a given address (127.0.0.1 by default). This can be overridden by specifying insecure_bind_address.

Usage

python3 -m opentf.qualitygate.qualitygate [--context context] [--config configfile]

Endpoint

This module exposes one endpoint:

  • /workflows/{workflow_id}/qualitygate[?mode=strict|passing] (GET)

If mode is not specified, strict is assumed.

Whenever calling this endpoint, a signed token must be specified via the Authorization header.

This header will be of form:

Authorization: Bearer xxxxxxxx

It must be signed with one of the trusted authorities specified in the current context.

/workflows/{workflow_id}/qualitygate

This endpoint returns a status manifest (a JSON document) with one of the following standard HTTP response codes: OK, Invalid, Unauthorized, or NotFound.

OK means the service knows the specified workflow. The details.status contains the quality gate status for the workflow. It can be one of the following:

  • SUCCESS: the workflow has succeeded and passes the quality gate.
  • NOTEST: the workflow has succeeded, but it produced no test result.
  • FAILURE: the workflow has failed, or failed the quality gate.
  • RUNNING: the workflow execution is still ongoing.

Invalid means the specified mode is not one of the expected values.

NotFound means the workflow is not known. It may be because the workflow is not yet known to the quality gate service, or that the workflow is no longer known to the quality gate service. Quality gate results for a given workflow are kept for a limited period after workflow completion (typically 60 minutes, but this can be configured).

Unauthorized means the provided token is invalid or not recognized by the quality gate service.

Modes

The quality gate service allows for multiple ‘modes’. Two modes are always available: strict and passing. If no mode is specified, strict is used.

  • strict means a workflow will pass the quality gate if and only if it has completed successfully and all test cases were OK. (It implies that a workflow with no test case will pass the quality gate if it has completed successfully).

  • passing means a workflow will pass the quality gate if it has completed successfully, event if some test cases were not OK.

Workflow status examples

This is a status manifest for a workflow with at least one failed test, using the passing mode. As long as the workflow has completed successfully, it will pass the quality gate:

curl \
  http://127.0.0.1/workflows/5dcc66dd-5d0d-4dcf-9b18-f39eafc0f279/qualitygate?mode=passing

{
    "kind": "Status",
    "apiVersion": "v1",
    "metadata": {},
    "status": "Success",
    "message": "",
    "details": {
        "status": "SUCCESS"
    },
    "code": 200,
}

This is a status manifest for same workflow, using the default (strict) mode. As it has at least one failed test, it fails that quality gate:

curl \
  http://127.0.0.1/workflows/5dcc66dd-5d0d-4dcf-9b18-f39eafc0f279/qualitygate

{
    "kind": "Status",
    "apiVersion": "v1",
    "metadata": {},
    "status": "Success",
    "message": "",
    "details": {
        "status": "FAILURE"
    },
    "code": 200,
}

This last example is when requesting the status of an unknown workflow:

curl \
  http://127.0.0.1/workflows/00000000-0000-0000-0000-000000000000/qualitygate

{
    "kind": "Status",
    "apiVersion": "v1",
    "metadata": {},
    "status": "Failure",
    "reason": "NotFound",
    "message": "Workflow 00000000-0000-0000-0000-000000000000 not found.",
    "code": 404
}