Quality gate service¶

This service is used to check the quality gate status of workflows.

A quality gate is a set of rules a workflow execution must satisfy. It exposes two user-facing endpoints that can be used by clients.

It has two service-specific configuration options besides the common ones.

Environment variables¶

Logging¶

You can set the QUALITYGATE_DEBUG_LEVEL (all upper-cased) or DEBUG_LEVEL environment variables to DEBUG to add additional information in the console for the launched service. It defaults to INFO. (Please note that setting DEBUG_LEVEL to DEBUG will produce tons of logs.)

The possible values are NOTSET, DEBUG, INFO, WARNING, ERROR, and FATAL. Those values are from the most verbose, NOTSET, which shows all logs, to the least verbose, FATAL, which only shows fatal errors.

If QUALITYGATE_DEBUG_LEVEL is not defined then the value of DEBUG_LEVEL is used (or INFO if DEBUG_LEVEL is not defined either).

Access logs are only shown at NOTSET and DEBUG levels.

Configuration Options¶

You can also set three service-specific configuration options using environment variables:

QUALITYGATE_WORKERS_MAX_WAIT_SECONDS sets the maximum duration to wait for workflow workers to complete their jobs (defaults to 30).
QUALITYGATE_DEFINITIONS_PATH sets the path to the directory containing quality gate definition files.
QUALITYGATE_DEFAULT_SCOPE sets the default scope applied to all quality gates (defaults to 'true').

Quality Gate Definitions¶

The QUALITYGATE_DEFINITIONS (all upper-cased) environment variable, if defined, must refer to an existing file that contains quality gate definitions.

(Two default quality gate modes are provided and are always available: strict and passing.)

For more information about quality gate definitions, see “Quality gate syntax.”

Tip

If the content of this referred file changes, the quality gates definitions used by the quality gate service will change accordingly. You do not need to restart the quality gate plugin.

Configuration File¶

This service has a configuration file (qualitygate.yaml by default) that describes the host, port, ssl_context, and trusted_authorities to use. It can also enable insecure logins.

If no configuration file is found it will default to the following values:

apiVersion: opentestfactory.org/v1beta2
kind: ServiceConfig
current-context: default
contexts:
- context:
    port: 443
    host: 127.0.0.1
    ssl_context: adhoc
    services:
      observer:
        endpoint: https://127.0.0.1
        token: reuse
    workers_max_wait_seconds: 30
    definitions_path: /app/qualitygates
    default_scope: 'true'
  name: default

The configuration included in the ‘allinone’ image is described in “Common settings.” The listening port is 12312 and the bind address is 0.0.0.0 as the service exposes user-facing endpoints.

It has three service-specific configuration options besides the common ones.

Workers Max Wait Seconds¶

This limit must be an integer. If the entry is missing, the default value will be assumed. It must be defined in the currently-used context.

workers_max_wait_seconds sets the maximum duration to wait for workflow workers to complete their jobs. If not specified, defaults to 30 seconds.

When a workflow completes and the quality gate service is queried, it checks if all workers have finished their jobs. While there are active workers on the workflow, the service considers it as running. If the workers do not complete within the specified time, the service will stop waiting and generate quality gate based on the available data.

Definitions Path¶

This definitions path must be a string. If the entry is missing, the default value will be assumed. It must be defined in the currently-used context.

definitions_path sets the path to the directory containing quality gate definition files. If not specified, defaults to /app/qualitygates.

See “Default Quality Gates and Definition Files Handling” below for more information.

Default Scope¶

This default scope value must be a string. If the entry is missing, the default value will be assumed. It must be defined in the currently-user context.

default_scope sets the default scope applied to all quality gates. If not specified, defaults to 'true'.

This default scope is cumulative with rules scopes. In other terms, if the default_scope is set to test.technology==robotframework, all quality gate rules will be calculated for Robot Framework tests only.

Default Quality Gates and Definition Files Handling¶

The Quality gate service provides two default quality gates, strict and passing. They apply to all test results and their respective thresholds are 100% and 0%. These definitions can be overridden using quality gate definition files.

An example of a quality gate definition file that overrides default quality gates scope and threshold is:

qualitygates:
- name: strict
  rules:
  - name: User-defined strict rule
    rule:
      scope: contains(test.runs-on, 'foo')
      threshold: 90%
- name: passing
  rules:
  - name: User-defined passing rule
    rule:
      scope: contains(test.runs-on, 'foo')
      threshold: 10%

The service looks for the quality gate definitions in two locations:

The file defined by the QUALITYGATE_DEFINITIONS environment variable.
The directory defined by the definitions_path parameter (/app/qualitygates by default).

Since there may be more than one definition file, the default quality gate definitions are prioritized. Definitions from the file specified by the QUALITYGATE_DEFINITIONS variable have the highest priority.

The definitions_path directory can contain multiple definition files. You can assign numerical prefixes to set their priority. For example, if the definitions_path directory contains the files 01_def.yaml and 02_def.yaml, the second file will take precedence over the first.

This prioritization logic also applies to all other quality gates specified in definition files. In other words, all same-named quality gates across different files are prioritized, and only those with the highest priority are applied.

Example¶

Assume that the QUALITYGATE_DEFINITIONS environment variable is set to the path of the env_qg.yaml definition file, and the definitions_path directory contains the files 01_def.yaml and 02_def.yaml:

env_qg.yaml

qualitygates:
- name: strict
  rules:
    ...
- name: my.qualitygate
  rules:
    ...

01_def.yaml

qualitygates:
- name: strict
  rules:
    ...
- name: passing
  rules:
    ...
- name: my.qualitygate
  rules:
    ...

02_def.yaml

qualitygates:
- name: passing
  rules:
    ...
- name: custom.qualitygate
  rules:
    ...

First, the quality gates from the 01_def.yaml definition file are read. These are then overridden by the quality gate definitions from the 02_def.yaml, and finally, the env_qg.yaml quality gates take precedence, overriding existing quality gates. As a result, when applying one of these quality gates to a completed workflow:

for the strict quality gate, the definition from the env_qg.yaml will be applied,
for the passing quality gate, the definition from the 02_def.yaml will be applied,
for the my.qualitygate quality gate, the definition from the env_qg.yaml will be applied,
for the custom.qualitygate, the definition from the 02_def.yaml will be applied.

Subscriptions¶

The quality gate service subscribes to no event.

Launch command¶

If you want to manually start the quality gate service, use the following command:

python -m opentf.qualitygate.qualitygate [--context context] [--config configfile]

Additional command-line options are available and described in “Command-line options.”