Defining quality gate¶
A quality gate is a set of rules a workflow execution must satisfy. You must create a YAML file to define your quality gate or gates.
Quality gate files must use YAML syntax and may have a
Quality gate structure¶
A quality gate has a mandatory
qualitygates section, which must contain at least
one quality gate.
qualitygates: - name: daily.quality.gate rules: - name: IHM tests rule: scope: test.technology == 'junit' threshold: 55% failure-status: [failure, blocked] - name: API tests ... - name: nightly.quality.gate rules: ...
Required The name of the quality gate. This name is used as the
value when querying the quality gate service endpoints.
Required The quality gate rules list. This section must contain at least one rule that will be used to evaluate a workflow execution.
The name of a quality gate rule. Even if optional, it is recommended to name the rule to have more readable quality gate results.
Required A quality gate rule definition, that contains rule scope, threshold and failure statuses. Rule scope and threshold are mandatory.
Required A quality gate rule scope. Only the tests matching the scope will be evaluated.
The rule scope uses workflow context syntax
and is based on the
test context. It contains information about the executed test
||The name of a job containing the executed test case.|
||The executed test case technology.|
||The name of an action that executed the test case.|
||The tags describing the test case execution environment.|
Example usage of the quality gate rule scope¶
This example quality gate applies three rules for a workflow with
several jobs containing JUnit and Cypress tests.
The first rule uses the
scope: true statement, which matches all tests in a workflow,
the second targets only the tests that were executed on Linux inside the job named
and the third refers to JUnit tests that do not use
qualitygates: - name: example.quality.gate rules: - name: All tests rule: scope: true threshold: 90% - name: API tests rule: scope: (contains(test.runs-on, 'linux')) && (test.job == 'API') threshold: 100% - name: Webapp tests rule: scope: (test.technology == 'junit') || (test.uses != 'junit/junit@v1') threshold: 75%
Required A minimum threshold of success (percentage). The quality gate will be considered as passed if the average success rate of the tests matching the rule scope is greater than or equal to the threshold.
The statuses of the test for which it is considered as failed. If
is not set, it will default to
[error, failure, blocked]. The possible values