Events¶

Generalities¶

All events are JSON documents. The content of those JSON documents is defined by schemas.

They all have an apiVersion, metadata, and a kind part. They may have additional parts.

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "mykind",
  "metadata": {
    "name": "myname",
    // ...
    }
  },
  // ...
}

Timestamps are in ISO 8601 format (YYYY-MM-DDT[HH[:MM[:SS[.mmm[uuu]]]]][+HH:MM])

Matrix of produced and consumed core events¶

In the following matrix, C means the service consumes the message, P means the service (may) produces the message. All services involved in workflow execution may produce ExecutionError messages. All services may also produce Notification messages.

Service	Workflow	WorkflowCancellation	WorkflowCompleted	WorkflowCanceled	WorkflowResult	ExecutionCommand	ExecutionResult	ExecutionError	GeneratorCommand	GeneratorResult	ProviderCommand	ProviderResult	Notification
Receptionist	P
Killswitch		P
Observer	C	C	C	C	C	C	C	C	C	C	C	C	C
Arranger	C	C	P	P		P	C	C / P	P	C	P	C	P
Channels						C	P	P					P
Generators								P	C	P			P
Providers								P			C	P	P
Publishers			C	C	C		C						P

Event formats¶

`Workflow`¶

A Workflow event is produced by the Receptionist core service whenever it receives a valid request.

The format of the Workflow event is the order received, plus a metadata.workflow_id entry and a metadata.creationTimestamp entry. It also always contains a metadata.namespace entry. It otherwise respects the Workflow schema.

{
  "apiVersion": "opentestfactory.org/v1",
  "kind": "Workflow",
  "metadata": {
    "Workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383",
    "namespace": "..."
  },
  // ...
}

`WorkflowCancellation`¶

A WorkflowCancellation event is produced by the Killswitch core service whenever it receives a valid request.

It respects the WorkflowCancellation schema.

It may include a details entry. This details entry may contain a reason and a source entries (both strings, if present), as well as any number of entries that contain a . in their names. Those extra entries may be any JSON object.

{
  "apiVersion": "opentestfactory.org/v1",
  "kind": "WorkflowCancellation",
  "metadata": {
    "name": "...",
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383"
    // ...
  }
}

`WorkflowCompleted`¶

When all the ExecutionCommand events associated with a workflow have led to the production of an ExecutionResult event and no ExecutionError event associated with the workflow has been produced, a WorkflowCompleted event is produced.

The format of the WorkflowCompleted event uses the metadata of the associated Workflow event.

It respects the WorkflowCompleted schema.

{
  "apiVersion": "opentestfactory.org/v1",
  "kind": "WorkflowCompleted",
  "metadata": {
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383"
    // ...
  }
}

`WorkflowCanceled`¶

A WorkflowCanceled event is produced to inform the interested services of the unsuccessful end of a workflow.

After the generation of an ExecutionError a WorkflowCanceled event is produced with a “failed” status.

After the reception of a WorkflowCancellation event, a WorkflowCanceled event is produced with a “cancelled” status.

The format of the WorkflowCanceled event reuses the metadata of the associated Workflow event, and can include a details entry of the associated ExecutionError event.

It respects the WorkflowCanceled schema.

{
  "apiVersion": "opentestfactory.org/v1",
  "kind": "WorkflowCanceled",
  "metadata": {
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383"
    // ...
  },
  "details": {
    "reason": "...",
    "status": "failed" // or "cancelled"
  }
}

Given the asynchronous nature of some processing associated with commands, some events associated with the canceled workflow may still occur after its cancellation.

`WorkflowResult`¶

A WorkflowResult event is produced to inform the interested services of the availability of a global workflow result.

It respects the WorkflowResult schema.

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "WorkflowResult",
  "metadata": {
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383",
    "attachments": {
      // ...
    }
    // ...
  },
  // ...
  "attachments": {
    "name_1": "url_1",
    // ...
  }
}

`ExecutionCommand`¶

For each elementary step defined directly or via a ProviderResult event, an ExecutionCommand event is produced.

The format of the ExecutionCommand contains a scripts block which is the set of elementary instructions that must be executed in the execution environment.

For a given job, the ExecutionCommand events are produced in order. The sequence is as follows:

An execution command with a step_sequence_id of CHANNEL_REQUEST is published
At least one channel handler makes an offer (if none do, the channel request is published repeatedly, and if no offer is made in received the job fails)
One offer is selected (all remaining ExecutionCommand pertaining to this job will have a metadata.channel_id entry)
An execution command with a step_sequence_id of 0 is published
Once the selected channel handler publishes a response (ExecutionResult), and if a following runnable step exists, the step sequence ID is incremented and the process continues
If no following step exists, an execution command with a step_sequence_id of CHANNEL_RELEASE is published

It respects the ExecutionCommand schema.

{
  "apiVersion": "opentestfactory.org/v1",
  "kind": "ExecutionCommand",
  "metadata": {
    "workflow_id": "...",
    "name": "...",
    "job_id": "...",
    "job_origin": "...",
    "step_sequence_id": ...,
    "creationTimestamp": "2021-09-10T13:48:49.582383"
    // ...
  },
  "runs-on": ["windows", "robotframework"],
  "scripts": ["ls -l"]
  // ...
}

`ExecutionResult`¶

For each ExecutionCommand published, an ExecutionResult event is generated.

The format of the ExecutionResult event uses the metadata of the associated ExecutionCommand event. It may also contain a list of document references, a list of output values, console logs, as well as a numeric status.

The numeric status is 0 if no error occurred while running the corresponding command script on the execution environment. It is the script’s returned error code otherwise. If no continue-on-error: true is set for the corresponding elementary step, the job it is part of will fail.

It respects the ExecutionResult schema.

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "ExecutionResult",
  "metadata": {
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383",
    "attachments": {
      // ...
    }
    // ...
  },
  // ...
  "attachments": {
    "name_1": "url_1",
    // ...
  },
  "outputs": {
    "key_1": "value_1",
    // ...
  },
  "logs": [
    // ...
  ],
  "status": 0
}

`ExecutionError`¶

At any time between the creation of a Workflow event and the production of a WorkflowCompleted event, one or more ExecutionError events can be produced.

The format of the ExecutionError event uses the metadata of the associated Workflow event. It may include a details block if such data is available.

It respects the ExecutionError schema.

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "ExecutionError",
  "metadata": {
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383"
    // ...
  },
  "details": {}
}

Given the asynchronous nature of some processing associated with commands, some events associated with the command may still occur after an execution error.

`GeneratorCommand`¶

For each generator job, a GeneratorCommand event is produced.

The format of the GeneratorCommand event uses the metadata of the Workflow event, the job definition containing the generator block, as well as the evaluated parameters specified in the job definition:

The metadata.creationTimestamp entry is overwritten to correspond to the event’s production timestamp.

It respects the GeneratorCommand schema.

{
  "apiVersion": "opentestfactory.org/v1alpha1",
  "kind": "GeneratorCommand",
  "metadata": {
    "workflow_id": "...",
    "job_id": "...",
    "name": "...",
    "namespace": "...",
    "creationTimestamp": "...",
    // ...
  },
  "generator": "my/generator@v1",
  "with": {
    "test_plan": "foo",
    "test_suite": "bar",
    "test_iteration": "baz"
  }
}

`GeneratorResult`¶

For each GeneratorCommand event considered by a generator service, a GeneratorResult event is produced.

The format of the GeneratorResult event takes the metadata of the associated GeneratorCommand event, and contains a jobs part which may contain one or more job definitions, which will either have a steps part or a generator part:

It respects the GeneratorResult schema.

{
  "apiVersion": "opentestfactory.org/v1",
  "kind": "GeneratorResult",
  "metadata": {
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383"
    // ...
  },
  // ...,
  "jobs": {
    "job-123": {
      "name": "Persistence tests",
      "needs": "ui-tests",
      "strategy": {
        "parallel": 4
      },
      "steps": [
        // ...
      ]
    }
  }
}

`ProviderCommand`¶

For each non-elementary step (directly specified in the command or obtained after expansion), a ProviderCommand event is produced.

The format of the ProviderCommand event includes the metadata of the associated Workflow event, as well as the context elements (variables, resources, job parameters excluding the other steps):

It respects the ProviderCommand schema.

{
  "apiVersion": "opentestfactory.org/v1",
  "kind": "ProviderCommand",
  "metadata": {
    "workflow_id": "...",
    "name": "...",
    "namespace": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383",
    "job_id": "...",
    "job_origin": [],
    "step_id": "...",
    "step_origin": []
    // ...
  },
  "runs-on": "...",
  "contexts": {
    // ...
  },
  "step": {
    "uses": "..."
    // ...
  }
  // ...,
}

`ProviderResult`¶

For each ProviderCommand event considered by a provider plugin, a ProviderResult event is produced.

The format of the ProviderResult event uses the metadata of the associated ProviderCommand event and contains a steps part which may contain zero or more steps, elementary or not:

It respects the ProviderResult schema.

{
  "apiVersion": "opentestfactory.org/v1",
  "kind": "ProviderResult",
  "metadata": {
    "workflow_id": "...",
    "creationTimestamp": "2021-09-10T13:48:49.582383"
    // ...
  },
  // ...,
  "steps": [
    // ...
  ]
}

`Notification`¶

Services can produce notifications, that must only carry an informational value.

{
    "apiVersion": "opentestfactory.org/v1alpha1",
    "kind": "Notification",
    "metadata": {
        "name": "Report processor",
        "workflow_id": "...",
        // ...
    },
    "spec": {
        // ...
    }
}

Some notifications are standardized. Services may produce them, but they then should respect their semantics, as other services consume them and have some associated expectations.

Standardized entries are in the spec section. They contain no dots (.).

Non-standardized entries must include a dot (.) but may not start with opentestfactory. or org.opentestfactory.

Here are some examples of valid and invalid non-standardized entry names:

myentry                        invalid (does not contain a dot)
.myentry                       valid
com.example.notif              valid
org.opentestfactory.notif      invalid (starts with a reserved string)

The values associated with those non-standardized entries must be valid JSON values.

Events¶

Generalities¶

Matrix of produced and consumed core events¶

Event formats¶

Workflow¶

WorkflowCancellation¶

WorkflowCompleted¶

WorkflowCanceled¶

WorkflowResult¶

ExecutionCommand¶

ExecutionResult¶

ExecutionError¶

GeneratorCommand¶

GeneratorResult¶

ProviderCommand¶

ProviderResult¶

Notification¶