Automation Packages are defined in a declarative way using YAML alongside the automation code. The Step Automation Package YAML syntax is specified by the Automation Package JSON schema, for more convenient work you can configure the embedded file editor in your IDE (IntelliJ IDEA or Visual Studio) to associate this schema with the Step YAML automation packages.

For Java Automation Packages, it is possible to directly declare certain entities in the code in addition to the YAML descriptor.

YAML descriptor

The automation package description is specified in a file name automation-package.yaml by convention. The top-level nodes of this descriptor consists of the following fields:

• name - the name of automation package
• version - specifies the version of the automation package schema used by this descriptor; required to ensure compatibility with future versions of Step
• keywords - fragment containing the keywords specifications
• plans - fragment containing the plans specifications
• parameters - fragment containing the parameters specifications
• schedules - fragment containing the schedules specifications
• presets - fragment containing the notifications presets specifications
• alertingRules - fragment containing the alerting rules specifications

Each fragment is optional.

Example:

---
version: 1.0.0
name: "load-testing-k6-automation-package"
keywords:
  - K6:
      name: 'OpenCart home'
      vus: 1
      iterations: 1
      scriptFile: "opencart-test.js"
plans:
  - name: "Opencart load test plan"
    root:
      testScenario:
        children:
          - threadGroup:
              users: 5
              iterations: 10
              children:
                - callKeyword:
                    keyword: "OpenCart home"

Name

The name of the automation package is not only used when deploying it to Step for organization purposes, it is also used as unique identifier. When deploying a package to Step, if a package with the same identifier already exists, it will be updated instead of deploying a new one.

For more advanced use cases, Step supports deploying multiple versions of a same automation package.

Schema version

This field defines the version of the automation package schema used by this descriptor. The available versions of the YAML automation package schema are described in the below table.

Step supports multiple schema versions to maintain backward compatibility. To ensure your automation package remains compatible with future versions of Step, you must explicitly set this field to the schema version used when editing the package. In other words, an automation package using the version 1.0.0 can be deployed on a Step instance using the version 1.1.0 as long as the version field is set in your YAML descriptor.

Keywords

Automation Packages support all Step Keyword types in a declarative way except for Astra, QF Test and PDF Test keywords. This includes for instance Java-based keywords (Playwright, Cypress, Selenium, etc), JMeter keywords and Grafana k6 keywords. You may read the related documentation for details on the configuration properties related to each keyword type, the same are available in YAML format.

Following sections give some details for a subset of these keyword’s types

Java keywords

Java keywords are supported natively with the Java @Keyword annotation and do not have to be specified in the YAML descriptor. In other words, any Keywords defined in the Java code contained in the automation package will be part of the package.

Please note that Java Keywords can include any third-party library and can thus be used to declare Keywords relying on Selenium, Playwright, or any other library supporting Java.

JMeter

Existing JMeter test plans can be declared as Keyword in automation packages using the JMeter keyword type as follows:

keywords:
  - JMeter:
      name: "JMeter keyword from automation package"  
      description: "JMeter keyword 1"
      jmeterTestplan: "jmeterProject1/jmeterProject1.xml"

Fields:

• name: an arbitrary name for the Keyword. This name will be used in the plan to reference the Keyword
• jmeterTestplan: the relative path to the JMeter test plan (xml)

Refer to the automation package schema for an exhaustive list of supported fields.

Cypress

Cypress Keywords can be declared in automation packages using the Cypress type as follows:

keywords:
  - Cypress:
      name: eCommerce - Typical visit
      cypressProject: cypress-test/
      spec: "opencart.cy.js"

Fields:

• name: an arbitrary name for the Keyword. This name will be used in the plan to reference the Keyword
• cypressProject: the relative path to the Cypress project containing the cypress.config.js
• spec: the filename of the Cypress spec to be executed

Refer to the automation package schema for an exhaustive list of supported fields.

Read more about the Cypress integration here

Grafana k6

Existing k6 tests can be declared as Keyword in automation packages using the K6 keyword type as follows:

keywords:
  - K6:
      name: 'OpenCart home'
      scriptFile: "opencart-test.js"
      vus: 1
      iterations: 1

Fields:

• name: an arbitrary name for the Keyword. This name will be used in the plan to reference the Keyword.
• scriptFile: relative path to the k6 script file to be executed.
  • If scriptDirectory is not specified, the path to scriptFile is considered relative to the root of the automation package.
  • If scriptDirectory is specified, then scriptFile should be defined relative to that directory.
• scriptDirectory (optional): The directory containing the script file and any supporting modules. If not set, the directory containing the defined script file will be used as the script directory.
  • This directory is transferred to the agent at execution time.
  • You can place and import local modules here using standard import statements in your k6 script.
  • To ensure proper module resolution, all required modules must reside within the specified scriptDirectory.
• vus (optional): number of VUs to run concurrently (k6 option ‘vus’).
• iterations (optional): total number of script iterations to be executed across all VUs (k6 option ‘iterations’).

Refer to the automation package schema for an exhaustive list of supported fields.

Read more about the k6 integration here

.NET

.NET keywords can be declared in automation packages using the DotNet type as follows:

keywords:
  - DotNet:
      name: "Open_Chrome_and_search_in_Google"
      dllFile: "DotNet/SeleniumKeywords.dll"
      librariesFile: "DotNet/AllDlls.zip"

Fields:

• name: the name of the Keyword. This name will be used in the plan to reference the Keyword. It should match the name of the Keyword contained in the referenced DLL
• dllFile: the relative path to the DLL file containing the Keyword
• librariesFile: optional path to the folder containing the library DLLs

Refer to the automation package schema for an exhaustive list of supported fields.

Read more about the .NET integration here

Node.js (JavaScript / TypeScript)

JavaScript / TypeScript keywords can be declared in automation packages using the Node keyword type as follows:

keywords:
  - Node:
      name: MyNodeKeyword
      jsfile: nodejs-keywords/

Fields:

• name: the name of the Keyword. This name will be used in the plan to reference the Keyword. It should match the name of the Keyword contained in the referenced project
• jsfile: the relative path to the Node.js project containing the Keyword definitions. This project should contain the package.json file and the keyword definitions in the folder keywords/

Refer to the automation package schema for an exhaustive list of supported fields.

Read more about the Node.js integration here

Oryon

Oryon keywords can be declared in automation packages using the Oryon keyword type as follows:

keywords:
  - Oryon:
      name: OryonKeyword
      scriptFile: oryon-script.groovy

Fields:

• name: an arbitrary name for the Keyword. This name will be used in the plan to reference the Keyword
• scriptFile: the relative path to the Oryon script

Refer to the automation package schema for an exhaustive list of supported fields.

Composite keyword

Composite keywords can be declared in automation packages using the Composite type.

keywords:
  - Composite:
      name: "Composite Keyword"
      plan:
        root:
          sequence:
            children:
              - echo:
                  text: "In Composite keyword"
              - return:
                  output:
                    - compositeOutput: "Composite Keyword output value"

The node plan contains the composite plan and follows the same structure as normal plans.

Read more about the Composite Keywords here

Plans

Plans are fully supported by the YAML format meaning that you can define plans in YAML identical to the ones created with the visual editor in Step. To understand the purpose and configuration of the many plan controls, refer to the dedicated documentation

Plans syntax

The plans are represented by a tree structure as in the visual plan editor and must contain a root node defining the type of plans (test set, test case, test scenario …). The different nodes share at least the following common properties:

• nodeName - [optional] name this node
• categories - [optional] list of categories assigned to this plan (categories can be used to filter plans for local and remote executions. See Filtering Plans).
• description - [optional] some textual description for the node
• children - [optional] the list of children of this node

Refer to the Controls documentation for the documentation and examples of the supported controls.

For instance, the following YAML descriptor contains a single plan of type test case invoking a keyword named “My Keyword”

---
name: "My automation package"
plans:
  - name: "Simple plan"
    categories:
      - "FunctionalTests"
    root:
      testCase:
        children:
          - callKeyword:
              keyword: "My Keyword"

Convert visual plan to YAML

To convert existing plans to YAML or to help you develop plans in YAML syntax, you may generate the plan in YAML format from the visual plan editor.

Convert YAML to visual plan

You can create a visual plan by providing its YAML source in the UI:

Advanced keyword call

In order to pass inputs to a keyword or to route the execution of the said keyword to specific agents, following syntax is used:

---
name: "My automation package"
plans:
  - name: "Simple plan"
    root:
      testCase:
        children:
          - callKeyword:
              keyword: "My Keyword"
              inputs:
                - myInputNumber: 1
                - myInputBoolean: true
                - myInputString: "some input"
              routing:
                - agentOS: "windows"

Dynamic expressions

As in the visual plan editor many input fields for plan controls accept either a literal value or a dynamic expression evaluated with groovy, for instance to resolve the value of a variable. The same is possible with the YAML syntax.

Example of a thread group defined with literal values:

- threadGroup:
  users: 5
  iterations: 10

Example of a thread group defined with variables:

- threadGroup:
  users: 
    expression: "nbUsers"
  iterations: 
    expression: "nbIterations"

Controls - YAML specificities

While most controls properties are identical to the one of the visual plans or self-explanatory, there a few exceptions.

AssertMetric

In YAML, the metrics names are slightly different than in the UI, here are the mapping from YAML to UI:

• response-time: “Response time”
• executions/count: “Execution count”
• executions/failure-percentage: “Execution failure percentage”
• executions/failure-count: “Execution failure count”
• executions/failures-count-by-error-code: “Execution failure count by error code”

Agent provisioning settings

When running in Kubernetes, Step supports automatic provisioning of Agents. This section explains how to define the provisioning settings in Automation Packages.

The provisioning settings can be defined for each Plan using the agents field.

Refer to the Agent provisioning page for more details about this feature.

Enable automatic determination and provisioning of agents

To enable the automatic determination and provisioning of Agents, the field agents has to be set to “auto_detect” as follows:

---
version: 1.0.0
name: "my-automation-package"
plans:
- name: "OpenCart okhttp Load Test"
  agents: auto_detect
  ...
  

Manual specification of agents

In order to specify the agents manually, a list of pools can be defined in the agents field:

---
version: 1.0.0
name: "my-automation-package"
plans:
- name: "OpenCart okhttp Load Test"
  agents:
  - pool: java-enterprise-agent
    replicas: 2
    image: docker-dev.exense.ch/test:agent-java-custom
    

It supports 3 fields:

• pool: specifies the name of the pool template to be provisioned. You can get the list of available pool in the UI (see previous section).
• replicas: specifies the number of agents to be provisioned.
• image: optionally specifies the Docker image to be used for the pool. Refer to Creating your own Agent images for details on how to create custom images.

Plain text plans

Plain text plans can also be included in automation packages. Simply include the files in the automation package and reference them in the descriptor using following syntax (only the file property is mandatory):

plansPlainText:
  - file: "plans/this-is-a-plain-text-plan.plan" # relative path to the file, supports wildcard 
    name: "this is a plain text plan" # optional plan name (cannot be specified when using wildcard for the file) 
    rootType: TestCase # optional root type of the plan (TestCase by default)
    categories: # optionally assign categories which can then be used for the selection of plans when triggering executions 
      - myTestCategory

Parameters

Parameters can be defined in the YAML descriptor as they would be in the Step UI. They also support dynamic expression in YAML.

Example:

parameters:
  - key: "baseURL"
    value: "http://test.com"
    activationScript: "env == 'TEST'"
  - key: "baseURL"
    value: "http://prod.com"
    activationScript: "env == 'PROD'"
  - key: "fullURL"
    value:
      expression: baseURL + "/something"

Schedules

Schedules can be defined in the YAML descriptor too using the same configuration properties as in the Step UI. Note if you need help to define the CRON expression we recommend using the Step UI wizard.

Example:

schedules:
  - name: "Opencart synthetic monitoring schedule"
    #CRON expression for one execution per minute
    cron: "0 0/1 * * * ?"
    planName: "Opencart synthetic monitoring plan"

Notification Presets

Here is a snippet showcasing the definition of Notification Presets:

notificationPresets:
  - EmailNotification:
      presetName: "Sample mail"
      upstreamPreset: "Mail"
      subject: 
        data: "${eventSummary}"
        protection: READONLY
  - WebhookNotification:
      presetName: "Sample Webhook call"
      upstreamPreset: "Webhook"
      url:
        data: "https://example.org"
      method:
        data: "POST"
      body:
        data: "{\"Hello\": \"World\"}"
      headers:
        data:
          "X-Step-URL" : "${controllerUrl}"

Alerting rules

Alerting rules can also be defined in YAML as you would do it in the UI. Example:

alertingRules:
  - description: "Send email for incident created for Opencart synthetic monitoring"
    name: "Opencart synthetic monitoring email"
    eventClass: IncidentOpenedEvent
    conditions:
      - BindingCondition:
          bindingKey: "executionParameters"
          predicate:
            BindingExistsPredicate:
              {}
      - BindingCondition:
          bindingKey: "executionParameters[env]"
          predicate:
            BindingValueEqualsPredicate:
              value: "PROD"
    actions:
      - NotificationAction:
          notification:
            EmailNotification:
              upstreamPreset: "Sample Mail"
              from:
                data: "me@example.org"
              to:
                data:
                  - "you@example.org"
                  - "them@example.org"
              bodyPlaintext:
                data: "An incident was opened, technical information: %{bindings}"

Fragment files

The concept of fragment files enables the creation of modular automation packages.

A fragment file declares automation package entities using the same syntax as automation package files and can be included in a main automation package to build packages in a modular way.

Fragment files are referenced in the main automation package file using the fragments field:

version: 1.0.0
name: "My Automation Package"
fragments:
- "fragment1.yml"
- "fragment2.yml"

Each entry of the fragment list can be either the relative path to the referenced fragment or a wildcard expression referencing a list of fragments:

version: 1.0.0
name: "My Automation Package"
fragments:
- "fragments/*.yaml"

Example

Here’s an example of a main automation-package.yaml referencing 2 fragments that each define a plan to be included in the automation package:

version: 1.0.0
name: "My Automation Package"
fragments:
- "plan1.yml"
- "plan2.yml"

The plan1.yml file would only contain the definition of the plan 1:

plans:
  - name: "Plan 1"
    root:
      testCase:
        children:
          - echo:
              text: "Hello world 1"

Setup your IDE for auto-completion and validation

The JSON schema can be downloaded from your Step instance by going to the Automation Package view and accessing the help menu. The underlying REST service can also be used directly for integration: <Your_Step_instance>/rest/automation-packages/schema

In IntelliJ IDEA add the JSON schema mapping as follows

1. In IDEA setting open the JSON Schema Mappings in Languages & Frameworks section.
2. Find and select the downloaded JSON Schema from your Step instance (Automation Package Schemas are currently not publicly available on schemastore.org )
3. Choose the files you want to link with the selected JSON Schema

After that you can use the chosen schema in the embedded YAML editor:

1. When you type something in the YAML editor linked with JSON Schema, IDEA shows the on-the-fly suggestions and validates entered data according to the selected JSON Schema
2. For the current file you can change the link to the JSON Schema (switch schema from OS version to EE or update schema to the latest version)