Postman¶

This plugin provides functions that handle Postman tests. It has been validated with Postman 8.12.1 and should work with any recent version of Postman.

It can be used directly in a workflow, or indirectly via generators (such as those providing access to test case managers).

A working newman installation with its HTML reporter (newman-reporter-html) must be available in the targeted execution environments.

The functions have a postman category prefix.

Functions¶

postman/postman@v1¶

Run a Postman collection.

Warning

If the function is used more than once in a job, it is up to the caller to ensure no previous test execution results remain before executing a new collection.

Inputs¶

The function has the following inputs:

collection (required)

The Postman collection to run.
folder (optional)

Specify a single folder to run from a collection. (--folder option)
environment (optional)

Specify a Postman environment as a JSON [file]. (-e, --environment option)
iteration-data (optional)

Specify a data file to use either JSON or CSV. (-d, --iteration-data option)
globals (optional)

Specify a Postman globals file as JSON [file]. (-g, --globals option)
iteration-count (optional)

Define the number of iterations to run. (-n, --iteration-count option)
delay-request (optional)

Specify a delay (in ms) between requests [number]. (--delay-request option)
timeout-request (optional)

Specify a request timeout (in ms) for a request. (--timeout-request option)
bail (optional)

Stop the runner when an invalid folder was specified using the --folder option or an error was encountered in general. (--bail option)
extra-options (optional)

Any other option.

Reports¶

The function generates the following reports:

newman-run-report.xml

Surefire report (XML).

It has the application/vnd.opentestfactory.postman-surefire+xml content type.
newman-run-report.html

Report formatted in HTML.

Example¶

- uses: postman/postman@v1
  with:
    collection: path/to/collection (JSON_file|URL)

- uses: postman/postman@v1
  with:
    collection: path/to/collection (JSON_file|URL)
    folder: folderName
    environment: path/to/Postman_environment (JSON_file|URL)
    iteration-data: path/to/Data_file (JSON|CSV)
    globals: path/to/Postman_globals_file (JSON)
    iteration-count: value1 (number)
    delay-request: value2 (number in ms)
    timeout-request: value3 (number in ms)
    bail: ('folder'|'failure'|true)

    extra-options: any other option

postman/execute@v1¶

An execute function for use by generators.

Test Reference format¶

The test reference format used by postman/execute@v1 is as follows:

{project}/{path_to_collection}[#{folder}[#{request}]]

With:

{project} (required): name of the project on the source code repository.
{path_to_collection} (required): path and name of the Postman collection file, from the root of the project (with the .postman_collection.json extension).

No support of renamed collection files

The name of the file defines the name of the collection, i.e. if the file is called <name>.postman_collection.json, then the collection name is considered to be <name>. This name <name> is the one used to analyze the execution report and determine the test result. If the collection file has been renamed, this mechanism will no longer work.
{folder} (optional): name of a specific folder containing requests in the Postman collection file (Newman’s --folder option).
{request} (optional): name of a specific request to parse for which you want to obtain a particular result.

Note

The {request} precision in the test reference has no impact on the execution, but only on test result determination.

So, even when defining the specific level of the request, all requests defined in the folder will be executed (this means, for example, that if several test references point toward the same folder but toward different requests, then all the requests of the folder will be executed several times).

The test reports include the traces of all executed requests. But, only the Error / Failed / Success status corresponding to the specific request is used to determine the test case result.

Warning

Unicity of the specified folder

If the root directory of the project contains several folders with the name {folder}, Newman will only execute the first found one. So, it is strongly advised to use unique names for the deepest directories.

Inputs¶

The function has the following inputs:

test (required)

The test reference.

Reports¶

The function generates the following reports:

newman-run-report.xml

Surefire report (XML).

It has the application/vnd.opentestfactory.postman-surefire+xml content type.
newman-run-report.html

Report formatted in HTML.

Customization¶

The POSTMAN_EXTRA_OPTIONS environment variable can be used to pass additional parameters to the newman run command.

If defined it will be appended to the end of the command line.

The following parameters are used in the provider-generated newman run command:

newman run "{collection_path}" \
  -g _opentf_global_params.json -e _opentf_environment_params.json \
  --reporters junit,html \
  --reporter-junit-export newman-run-report.xml \
  --reporter-html-export newman-run-report.html $POSTMAN_EXTRA_OPTIONS

You must avoid passing, via the POSTMAN_EXTRA_OPTIONS variable, the command line parameters that conflict with the parameters already used, or the parameters that impact the generation or alter the path of the reports expected by the orchestrator (view “Reports” section).

Extended HTML reports¶

Postman HTML reports can be extended using newman-reporter-htmlextra package. It should be installed globally on the execution environment:

npm install -g newman-reporter-htmlextra

newman-reporter-htmlextra package provides HTML reports containing execution summary, requests details, and detailed information on failed and skipped test cases.

To obtain extended HTML reports, pass the following values to the POSTMAN_EXTRA_OPTIONS environment variable:

--reporters junit,htmlextra --reporter-junit-export newman-run-report.xml --reporter-htmlextra-export newman-run-report.html

It will override the provider-generated newman run command reporters parameters. Two reports will be generated at the end of the run: Surefire XML report and extended HTML report.

For example, it is possible to set the POSTMAN_EXTRA_OPTIONS variable value using a provider hook that will run before each execute command:

hooks:
- name: newman-reporter-htmlextra (windows)
  events:
  - categoryPrefix: postman
    category: execute
  if: runner.os == 'windows'
  before:
  - run: echo set POSTMAN_EXTRA_OPTIONS=--reporters junit,htmlextra --reporter-junit-export newman-run-report.xml --reporter-htmlextra-export newman-run-report.html >>%OPENTF_VARIABLES%
- name: newman-reporter-htmlextra (linux)
  events:
  - categoryPrefix: postman
    category: execute
  if: runner.os == 'linux'
  before:
  - run: echo 'POSTMAN_EXTRA_OPTIONS="--reporters junit,htmlextra --reporter-junit-export newman-run-report.xml --reporter-htmlextra-export newman-run-report.html"' >>$OPENTF_VARIABLES

Example¶

- uses: postman/execute@v1
  with:
    test: path/to/collection#folderName#requestName

postman/params@v1¶

A params function for use by generators.

params function has mandatory data and format inputs.

Inputs¶

The function has the following inputs:

data (required)

The data to use for the automated test.
format (required)

The format to use for the automated test data.

Example¶

- uses: postman/params@v1
  with:
    data:
      global:
        key1: value1
        key2: value2
      test:
        key1: value3
        key3: value4
    format: format

format must so far be SQUASHTM_FORMAT (tm.squashtest.org/params@v1).

data can have two keys:

global for defining global parameters.
test for defining test parameters.

Using with inception¶

Please refer to “Inception” for more information on what inception is.

Preload the inception environment with at least the test execution report data.

Example¶

my_workflow.yaml

metadata:
  name: Postman Inception
resources:
  files:
  - xml_report
  - html_report
jobs:
  my_job:
    runs-on: inception
    steps:
    - uses: actions/prepare-inception@v1
      with:
        newman-run-report.xml: ${{ resources.files.xml_report }}
        newman-run-report.html: ${{ resources.files.html_report }}
    - uses: postman/execute@v1
      with:
        test: OpenWeather/postman_collection.json

You can use the following command to run it:

BashCMDPowerShell

opentf-ctl \
    run workflow my_workflow.yaml \
    -f xml_report=report.xml \
    -f html_report=report.html

opentf-ctl ^
    run workflow my_workflow.yaml ^
    -f xml_report=report.xml ^
    -f html_report=report.html

opentf-ctl `
    run workflow my_workflow.yaml `
    -f xml_report=report.xml `
    -f html_report=report.html

Configuration¶

Hooks can be defined for the provided functions. This can be done in workflow definitions or at the orchestrator level so that they apply to all your workflows.

Configuration at the orchestrator level is done by setting the POSTMAN_PROVIDER_HOOKS environment variable.

Please refer to “Common Provider Settings” for more information.