Events¶

Generalities¶

All events are JSON documents. The content of those JSON documents is defined by schemas.

They all have an apiVersion, metadata, and a kind part. They may have additional parts.

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "mykind",
  "metadata": {
    "name": "myname",
    // ...
    }
  },
  // ...
}

Timestamps are in ISO 8601 format (YYYY-MM-DDT[HH[:MM[:SS[.mmm[uuu]]]]][+HH:MM])

Matrix of produced and consumed core events¶

In the following matrix, C means the service consumes the message, P means the service (may) produces the message. And all services may produce ExecutionError messages.

Service	Workflow	GeneratorCommand	GeneratorResult	ProviderCommand	ProviderResult	ExecutionCommand	ExecutionResult	WorkflowCompleted	ExecutionError	WorkflowCanceled	Notification
Receptionist	P
Killswitch										P
Observer	C	C	C	C	C	C	C	C	C	C	C
Arranger	C	P	C	P	C	P	C	P	C / P	C / P	P
Generators		C	P								P
Providers				C	P						P
Channels						C	P				P
Publishers							C	C		C	P

Event formats¶

`Workflow`¶

A Workflow event is produced by the Receptionist core service whenever it receives a valid request.

The format of the Workflow event is the order received, plus a metadata.workflow_id entry and a metadata.creationTimestamp entry. It otherwise respects the Workflow schema.

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "Workflow",
  "metadata": {
    "Workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383"
  },
  // ...
}

`GeneratorCommand`¶

For each generator job, a GeneratorCommand event is produced.

The format of the GeneratorCommand event uses the metadata of the Workflow event, the job definition containing the generator block, as well as the evaluated parameters specified in the job definition:

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "GeneratorCommand",
  "metadata": {
    "workflow_id": "...",
    "job_id": "...",
    "name": "...",
    "creationTimestamp": "...",
    // ...
  },
  "generator": "my/generator@v1",
  "with": {
    "test_plan": "foo",
    "test_suite": "bar",
    "test_iteration": "baz"
  }
}

`GeneratorResult`¶

For each GeneratorCommand event considered by a generator service, a GeneratorResult event is produced.

The format of the GeneratorResult event takes the metadata of the associated GeneratorCommand event, and contains a jobs part which may contain one or more job definitions, which will either have a steps part or a generator part:

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "GeneratorResult",
  "metadata": {
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383"
    // ...
  },
  // ...,
  "jobs": {
    "job-123": {
      "name": "Persistence tests",
      "needs": "ui-tests",
      "strategy": {
        "parallel": 4
      },
      "steps": [
        // ...
      ]
    }
  }
}

`ProviderCommand`¶

For each non-elementary step (directly specified in the command or obtained after expansion), a ProviderCommand event is produced.

The format of the ProviderCommand event includes the metadata of the associated Workflow event, as well as the context elements (variables, resources, job parameters excluding the other steps):

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "ProviderCommand",
  "metadata": {
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383"
    // ...
  },
  "step": {
    // ...
  }
  // ...,
}

`ProviderResult`¶

For each ProviderCommand event considered by a provider plugin, a ProviderResult event is produced.

The format of the ProviderResult event uses the metadata of the associated ProviderCommand event and contains a steps part which may contain zero or more steps, elementary or not:

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "ProviderResult",
  "metadata": {
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383"
    // ...
  },
  // ...,
  "steps": [
    // ...
  ]
}

`ExecutionCommand`¶

For each elementary step defined directly or via a ProviderResult event, an ExecutionCommand event is produced.

The format of the ExecutionCommand event uses the metadata of the associated ProviderResult event, and contains a scripts block which is the set of elementary instructions that must be executed in the execution environment. If this script block produces deliverables that are intended to be published, the event will also contain a reports block, which is the set of elementary instructions that must be executed in the execution environment to retrieve the produced reports. If the elementary instructions of the scripts block are executed asynchronously, the ExecutionCommand event will also contain a status block, which is the set of elementary instructions which will make it possible to determine whether the execution ended, successfully or not:

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "ExecutionCommand",
  "metadata": {
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383"
    // ...
  },
  "runs-on": ["windows", "robotframework"],
  "scripts": ["ls -l"]
  // ...
}

`ExecutionResult`¶

For each ExecutionCommand event that contains a reports block, and which is successfully completed, an ExecutionResult event is generated.

The format of the ExecutionResult event uses the metadata of the associated ExecutionCommand event. It also contains a list of document references and a list of output values, as well as a numeric status.

The numeric status is 0 if no error occurred while running the corresponding command script on the execution environment. It is the script returned error code otherwise. If no continue-on-error: true is set for the corresponding elementary step, the job it is part of will fail.

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "ExecutionResult",
  "metadata": {
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383"
    // ...
  },
  // ...
  "attachments": {
    "name_1": "url_1",
    // ...
  },
  "outputs": {
    "key_1": "value_1",
    // ...
  },
  "status": 0
}

`ExecutionError`¶

At any time between the creation of a Workflow event and until the production of a WorkflowCompleted event, one or more ExecutionError events can be produced.

The format of the ExecutionError event uses the metadata of the associated Workflow event. It may include a details block if such data is available.

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "ExecutionError",
  "metadata": {
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383"
    // ...
  }
}

Given the asynchronous nature of some processing associated with commands, some events associated with the command may still occur after an execution error.

`WorkflowCompleted`¶

When all the ExecutionCommand events associated with a workflow have led to the production of an ExecutionResult event and no ExecutionError event associated with the workflow has been produced, a WorkflowCompleted event is produced.

The format of the WorkflowCompleted event uses the metadata of the associated Workflow event:

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "WorkflowCompleted",
  "metadata": {
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383"
    // ...
  }
}

`WorkflowCanceled`¶

A WorkflowCanceled event is produced to inform the interested services of the unsuccessful end of a workflow.

There are two causes for those events. After the generation of an ExecutionError a WorkflowCanceled event is produced with a “failed” status. And if the Killswitch service is called, a WorkflowCanceled event is produced with a “cancelled” status.

The format of the WorkflowCanceled event reuses the metadata of the associated Workflow event, and can include a details entry of the associated ExecutionError event:

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "WorkflowCanceled",
  "metadata": {
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383"
    // ...
  },
  "details": {
    "reason": "...",
    "status": "failed" // or "cancelled"
  }
}

Given the asynchronous nature of some processing associated with commands, some events associated with the cancelled workflow may still occur after its cancellation.

`Notification`¶

Services can produce notifications, that must only carry an informational value.

{
    "apiVersion": "opentestfactory.org/v1alpha1",
    "kind": "Notification",
    "metadata": {
        "name": "Report processor",
        "workflow_id": "...",
        // ...
    },
    "spec": {
        // ...
    }
}

Some notifications are standardized. Services may produce them, but they then should respect their semantics, as other services consume them and have some associated expectations.

Standardized entries are in the spec section. They contain no dots (.).

Non-standardized entries must include a dot (.) but may not start with opentestfactory. or org.opentestfactory.

Here are some examples of valid and invalid non-standardized entry names:

myentry                        invalid (does not contain a dot)
.myentry                       valid
com.example.notif              valid
org.opentestfactory.notif      invalid (starts with a reserved string)

The values associated with those non-standardized entries can be any valid JSON values.

Events¶

Generalities¶

Matrix of produced and consumed core events¶

Event formats¶

Workflow¶

GeneratorCommand¶

GeneratorResult¶

ProviderCommand¶

ProviderResult¶

ExecutionCommand¶

ExecutionResult¶

ExecutionError¶

WorkflowCompleted¶

WorkflowCanceled¶

Notification¶