Events¶
Generalities¶
All events are JSON documents. The content of those JSON documents is defined by schemas.
They all have an apiVersion
, metadata
, and a kind
part. They may have
additional parts.
{
"apiVersion": "opentestfactory.org/v1alpha1",
"kind": "mykind",
"metadata": {
"name": "myname",
// ...
}
},
// ...
}
Timestamps are in ISO 8601 format (YYYY-MM-DDT[HH[:MM[:SS[.mmm[uuu]]]]][+HH:MM]
)
Matrix of produced and consumed core events¶
In the following matrix, C
means the service consumes the message, P
means
the service (may) produces the message. And all services may produce ExecutionError
messages.
Service | Workflow | GeneratorCommand | GeneratorResult | ProviderCommand | ProviderResult | ExecutionCommand | ExecutionResult | WorkflowCompleted | ExecutionError | WorkflowCanceled | Notification |
---|---|---|---|---|---|---|---|---|---|---|---|
Receptionist | P | ||||||||||
Killswitch | P | ||||||||||
Observer | C | C | C | C | C | C | C | C | C | C | C |
Arranger | C | P | C | P | C | P | C | P | C / P | C / P | P |
Generators | C | P | P | ||||||||
Providers | C | P | P | ||||||||
Channels | C | P | P | ||||||||
Publishers | C | C | C | P |
Event formats¶
Workflow
¶
A Workflow
event is produced by the Receptionist
core service whenever it receives
a valid request.
The format of the Workflow
event is the order received, plus a metadata.workflow_id
entry and a metadata.creationTimestamp
entry. It otherwise respects the
Workflow schema.
{
"apiVersion": "opentestfactory.org/v1alpha1",
"kind": "Workflow",
"metadata": {
"Workflow_id": "...",
"creationTimestamp": "2021-09-10T13:48:49.582383"
},
// ...
}
GeneratorCommand
¶
For each generator job, a GeneratorCommand
event is produced.
The format of the GeneratorCommand
event uses the metadata of the Workflow
event,
the job definition containing the generator
block, as well as the evaluated
parameters specified in the job definition:
{
"apiVersion": "opentestfactory.org/v1alpha1",
"kind": "GeneratorCommand",
"metadata": {
"workflow_id": "...",
"job_id": "...",
"name": "...",
"creationTimestamp": "...",
// ...
},
"generator": "my/generator@v1",
"with": {
"test_plan": "foo",
"test_suite": "bar",
"test_iteration": "baz"
}
}
GeneratorResult
¶
For each GeneratorCommand
event considered by a generator service, a
GeneratorResult
event is produced.
The format of the GeneratorResult
event takes the metadata of the associated
GeneratorCommand
event, and contains a jobs
part which may contain one or more
job definitions, which will either have a steps
part or a generator
part:
{
"apiVersion": "opentestfactory.org/v1alpha1",
"kind": "GeneratorResult",
"metadata": {
"workflow_id": "...",
"creationTimestamp": "2021-09-10T13:48:49.582383"
// ...
},
// ...,
"jobs": {
"job-123": {
"name": "Persistence tests",
"needs": "ui-tests",
"strategy": {
"parallel": 4
},
"steps": [
// ...
]
}
}
}
ProviderCommand
¶
For each non-elementary step (directly specified in the command or obtained after
expansion), a ProviderCommand
event is produced.
The format of the ProviderCommand
event includes the metadata of the associated
Workflow
event, as well as the context elements (variables, resources, job parameters
excluding the other steps):
{
"apiVersion": "opentestfactory.org/v1alpha1",
"kind": "ProviderCommand",
"metadata": {
"workflow_id": "...",
"creationTimestamp": "2021-09-10T13:48:49.582383"
// ...
},
"step": {
// ...
}
// ...,
}
ProviderResult
¶
For each ProviderCommand
event considered by a provider plugin, a
ProviderResult
event is produced.
The format of the ProviderResult
event uses the metadata of the associated
ProviderCommand
event and contains a steps
part which may contain zero or more
steps, elementary or not:
{
"apiVersion": "opentestfactory.org/v1alpha1",
"kind": "ProviderResult",
"metadata": {
"workflow_id": "...",
"creationTimestamp": "2021-09-10T13:48:49.582383"
// ...
},
// ...,
"steps": [
// ...
]
}
ExecutionCommand
¶
For each elementary step defined directly or via a ProviderResult
event, an
ExecutionCommand
event is produced.
The format of the ExecutionCommand
event uses the metadata of the associated
ProviderResult
event, and contains a scripts
block which is the set of elementary
instructions that must be executed in the execution environment. If this script
block
produces deliverables that are intended to be published, the event will also contain a
reports
block, which is the set of elementary instructions that must be executed in
the execution environment to retrieve the produced reports. If the elementary
instructions of the scripts
block are executed asynchronously, the ExecutionCommand
event will also contain a status
block, which is the set of elementary instructions
which will make it possible to determine whether the execution ended, successfully or
not:
{
"apiVersion": "opentestfactory.org/v1alpha1",
"kind": "ExecutionCommand",
"metadata": {
"workflow_id": "...",
"creationTimestamp": "2021-09-10T13:48:49.582383"
// ...
},
"runs-on": ["windows", "robotframework"],
"scripts": ["ls -l"]
// ...
}
ExecutionResult
¶
For each ExecutionCommand
event that contains a reports
block, and which is
successfully completed, an ExecutionResult
event is generated.
The format of the ExecutionResult
event uses the metadata of the associated
ExecutionCommand
event. It also contains a list of document references and a list of
output values, as well as a numeric status.
The numeric status is 0 if no error occurred while running the corresponding command
script on the execution environment. It is the script returned error code otherwise. If no
continue-on-error: true
is set for the corresponding elementary step, the job it is part of
will fail.
{
"apiVersion": "opentestfactory.org/v1alpha1",
"kind": "ExecutionResult",
"metadata": {
"workflow_id": "...",
"creationTimestamp": "2021-09-10T13:48:49.582383"
// ...
},
// ...
"attachments": {
"name_1": "url_1",
// ...
},
"outputs": {
"key_1": "value_1",
// ...
},
"status": 0
}
ExecutionError
¶
At any time between the creation of a Workflow
event and until the production of a
WorkflowCompleted
event, one or more ExecutionError
events can be produced.
The format of the ExecutionError
event uses the metadata of the associated Workflow
event. It may include a details
block if such data is available.
{
"apiVersion": "opentestfactory.org/v1alpha1",
"kind": "ExecutionError",
"metadata": {
"workflow_id": "...",
"creationTimestamp": "2021-09-10T13:48:49.582383"
// ...
}
}
Given the asynchronous nature of some processing associated with commands, some events associated with the command may still occur after an execution error.
WorkflowCompleted
¶
When all the ExecutionCommand
events associated with a workflow have led to the
production of an ExecutionResult
event and no ExecutionError
event associated with the workflow has been produced, a WorkflowCompleted
event is
produced.
The format of the WorkflowCompleted
event uses the metadata of the associated Workflow
event:
{
"apiVersion": "opentestfactory.org/v1alpha1",
"kind": "WorkflowCompleted",
"metadata": {
"workflow_id": "...",
"creationTimestamp": "2021-09-10T13:48:49.582383"
// ...
}
}
WorkflowCanceled
¶
A WorkflowCanceled
event is produced to inform the interested services of the
unsuccessful end of a workflow.
There are two causes for those events. After the generation of an ExecutionError
a
WorkflowCanceled
event is produced with a “failed” status. And if the Killswitch
service is called, a WorkflowCanceled
event is produced with a “cancelled” status.
The format of the WorkflowCanceled
event reuses the metadata of the associated
Workflow
event, and can include a details
entry of the associated
ExecutionError
event:
{
"apiVersion": "opentestfactory.org/v1alpha1",
"kind": "WorkflowCanceled",
"metadata": {
"workflow_id": "...",
"creationTimestamp": "2021-09-10T13:48:49.582383"
// ...
},
"details": {
"reason": "...",
"status": "failed" // or "cancelled"
}
}
Given the asynchronous nature of some processing associated with commands, some events associated with the cancelled workflow may still occur after its cancellation.
Notification
¶
Services can produce notifications, that must only carry an informational value.
{
"apiVersion": "opentestfactory.org/v1alpha1",
"kind": "Notification",
"metadata": {
"name": "Report processor",
"workflow_id": "...",
// ...
},
"spec": {
// ...
}
}
Some notifications are standardized. Services may produce them, but they then should respect their semantics, as other services consume them and have some associated expectations.
Standardized entries are in the spec
section. They contain no dots (.
).
Non-standardized entries must include a dot (.
) but may not start with
opentestfactory.
or org.opentestfactory
.
Here are some examples of valid and invalid non-standardized entry names:
myentry invalid (does not contain a dot)
.myentry valid
com.example.notif valid
org.opentestfactory.notif invalid (starts with a reserved string)
The values associated with those non-standardized entries can be any valid JSON values.