If you’ve purchased the Notification package or are running the Premium Enterprise version of Step, you can subscribe to notifications about the status of test plan executions.

Currently, Step supports two notification mechanisms: E-Mail, and Webhooks.

E-Mail: Notifications are delivered by sending E-Mails to specific mail addresses. This is a simple and proven procedure, but the format of these messages is fixed.
Webhooks: Notifications are delivered by performing HTTP requests to an external URL. In this case, an HTTP server is required on the receiving end, and the actual data processing is the responsibility of the remote server. You can choose to have webhooks delivered in a standard format defined by Step, or in a format that you can freely specify according to your implementation.

Configuring notifications is a two-step procedure: First, a notification gateway is defined which contains the generic configuration (e.g., Mail server, or Webhook endpoint). Second, individual notification triggers are configured, where notification-specific settings (e.g., mail recipient, or custom payloads for webhooks) are defined.

Define your notification gateways

To create a notification gateway, navigate to Admin > Settings > Notification gateways and click on the New button on the top right :

E-Mail notification gateway

Create a new notification gateway of type “E-Mail”, and fill the fields according to the settings you want to use :

Name : the name of your gateway
Type : the gateway type (E-Mail).
From Address : the mail address that is set as the mail sender
From Name : the display name of the mail sender
SMTP Server host : the SMTP server hostname
SMTP Server port : the SMTP server port
Username : the username to access the email account
Password : the password to access the email account

Example

The example below is using a Gmail account in order to setup a gateway :

Follow this link for more information about Google SMTP servers configuration.

Webhook notification gateway (Step message format)

For this notification mechanism to work, the receiving endpoint must be able to handle messages in the specific JSON format that Step uses (more information on the format is provided below). In other words, “Step defines the rules, and you adhere to them”. For the opposite case, see the section on custom webhooks below.

To create a new notification gateway using this message format, fill the fields according to the settings you want to use :

Name : the name of your gateway
Type : Webhook (Step format)
HTTP(S) Endpoint : the URL where notifications are sent to. HTTPS is strongly recommended, but not required.
Secret : any string. This is included in every message sent to the webhook endpoint, and can be used (in the webhook implementation) as a simple authentication factor to validate the authenticity of received messages.
Payload Description: a user-friendly description of the kind of payload that is sent to the webhook. This is displayed to Step users when they define the notifications which are to be sent via this gateway. Attention: if this field is left empty, the notification payload will always be empty, and will not be user-configurable.

Examples

Use case: endpoint for “execution failed” events, no payload

Use case: endpoint for “execution ended” events, payload is an informational message

Use case: endpoint for “execution ended” events, payload is a list of mail addresses to inform

Webhook notification gateway (Custom format)

If you require a more flexible webhook format, Step can be configured to send webhook messages using any textual format that you provide. In other words, “You define the rules, and Step adheres to them”. This setup requires slightly more configuration.

To create a new notification gateway using a custom message format, fill the fields according to the settings you want to use :

Name : the name of your gateway
Type : Webhook (Custom)
HTTP(S) Endpoint : the URL where notifications are sent to. HTTPS is strongly recommended, but not required.
HTTP METHOD : GET or POST.
HTTP Headers : An arbitrary number of HTTP headers to send, in standard HTTP format.
Message Body: The raw HTTP message body to be sent to the remote server.

Variable substitution

The endpoint, headers, and body will be used verbatim as defined by you in the gateway configuration screen. However, always sending the exact same message is generally of limited usefulness, as one will usually require at least some dynamic information.

Therefore, the following substitutions will be performed before sending the message:

${executionId}: The identifier of the execution that has triggered the notification, in hexadecimal format (e.g. 63dbcb222e9efd3a01b09c4f).
${status}: The status of the execution, for instance PASSED or FAILED.
${eventType}: one of execution_ended or execution_failed. This is determined by the subscription which triggered the notification.
${tenant}: The name of the project in which the execution took place.
${executionDescription}: The name of the plan that was executed.
${executionUrl}: The complete URL of the execution, suitable for opening in a web browser.
${controllerUrl}: The base URL of the controller where the execution took place.
${executionParameters}: A JSON representation of all parameters of the execution.
${executionParameters[XY]}: The value of the execution parameter XY, if and only if such a parameter is present. For example, if the execution contains a parameter named env with value TEST, then ${executionParameters[env]} will be replaced by TEST. However, if no such parameter is present, the string ${executionParameters[env]} will be kept unchanged (it will not be removed!).

Notes:

Variable substitution is performed in the message body, as well as the headers and the endpoint URL. In the last case, the result of the substitution is URL-encoded in addition.
Only a textual substitution is performed, there is no full-blown expression evaluation. In other words, substitution will only be performed on the exact strings as shown above. For example, the string ${ status } (with added spaces) will not result in a substitution.

Example

The following definition for a custom webhook:

Will result in the following payload when sent (the actual values will of course depend on the environment):

{
	"step_server": "http://localhost:8080",
	"project" : "Common",
	"plan" : "Test Plan",
	"parameters": {"xray_reporting":"true","env":"TEST"},
	"id" : "63dd054ec9615c57fdbe64a9",
	"status" : "PASSED"
}

Setup your notification triggers

Once you have defined notification gateways, you can create notification triggers for the events you are interested in.

Create a test plan, execute it, open the last execution, and then click on the Add notification link on the bottom right :

Alternatively, you can add notifications directly via the Notifications view, where you can also manage existing notifications:

Fill in the fields according to the kind of notification you want to receive :

Event : currently, Execution ended and Execution failed events are supported.
Gateway : select a previously defined gateway
Additional fields:
- For E-Mail gateways, there will be a field named Mail Recipient where you can enter mail addresses.
- For Step Webhook gateways, if (and only if) a Payload description was defined, a corresponding field will be shown.

Examples

Note: these examples show configuration screens for gateways defined like in the examples above. As mentioned, the concrete labels of the displayed fields depend on the gateway configuration!

E-Mail notification

Webhook, Step format, payload with plain-text message

Webhook, custom format

Notification subscriptions view

All notification subscriptions are listed in the Notifications view, accessible from the Alerting section:

From this view, existing subscriptions can be modified or deleted, and new subscriptions can be defined.

Keyword “Notification_Send”

A LocalFunction keyword, pre-populated by default in the enterprise edition of Step, allows you to send personalized notifications using the gateways you have set up previously:

The expected inputs are:

gatewayId : the name of the gateway your previously defined
sendTo : the destination of the notification
subject: the subject of your message
message: the body of your message
sendExecutionLink (optional): set it to false to remove the execution link from the message body. By default, this link is added at the end of the message

Webhook invocations and payloads (Step format)

This section only applies to Webhooks in Step format and provides the information required for implementing a compatible endpoint.

Notifications via Webhooks follow a simple and standardized JSON format, and are sent to the URL defined in the notification gateway setup. Please ensure that the target endpoints are directly reachable from the machine where the Step controller is running.

Messages are sent using the HTTP POST method, with a Content-Type of application/json. It is expected that the receiving end of the implementation returns an HTTP 200 OK Status Code on success. If any HTTP status code other than HTTP 200 OK is received in response to sending a webhook notification, the status code, and, if provided, the message body, are logged by the controller as warnings.

If you want to test your side of the webhook implementation, you can simulate a request using a curl command like the following (replace values as needed):

curl -H "Content-Type: application/json" -X POST http://localhost:9999/test/post/ended -d '{"secret":"secret", "eventType":"execution_ended", "payload":"the payload", "controllerUrl":"http://step.host/", "executionId":"executionId", "executionDescription":"Step plan name", "executionParameters":{"env":"TEST"}, "status":"PASSED"}'

“Execution ended” message

{
  "secret":"The secret you defined for the gateway",
  "eventType":"execution_ended",
  "payload":"The payload you defined for the notification",
  "controllerUrl":"The URL you defined for the controller in step.properties",
  "executionId":"The execution ID",
  "executionDescription":"The name of the plan that was run",
  "executionParameters":{"env":"TEST"},
  "status":"PASSED"
}

The eventType is ALWAYS execution_ended.

The exhaustive list of all (theoretically) possible status values is: TECHNICAL_ERROR, FAILED, PASSED, INTERRUPTED, SKIPPED, NORUN, RUNNING.

“Execution failed” message

This message is similar to the above one, except for the “status” field being omitted (as it would always be “FAILED”):

{
  "secret":"The secret you defined for the gateway",
  "eventType":"execution_failed",
  "payload":"The payload you defined for the notification",
  "controllerUrl":"The URL you defined for the controller in step.properties",
  "executionId":"The execution ID",
  "executionDescription":"The name of the plan that was run",
  "executionParameters":{"env":"TEST"}
}

The eventType is ALWAYS execution_failed.

Define your notification gateways

E-Mail notification gateway

Example

Webhook notification gateway (Step message format)

Examples

Use case: endpoint for “execution failed” events, no payload

Use case: endpoint for “execution ended” events, payload is an informational message

Use case: endpoint for “execution ended” events, payload is a list of mail addresses to inform

Webhook notification gateway (Custom format)

Variable substitution

Example

Setup your notification triggers

Examples

E-Mail notification

Webhook, Step format, payload with plain-text message

Webhook, custom format

Notification subscriptions view

Keyword “Notification_Send”

Webhook invocations and payloads (Step format)

“Execution ended” message

“Execution failed” message

See Also