Categories: DEVELOPER GUIDE CI CD DEV OPS ENTERPRISE

Automation Package descriptor

Automation Packages are defined in a declarative way using YAML alongside the automation code. The Step automation package YAML syntax is specified by a JSON schema, for more convenient work you can configure the embedded file editor in your IDE (IntelliJ IDEA or Visual Studio) to associate this schema with the Step YAML automation packages.

YAML descriptor

The automation package description is specified in a file name automation-package.yaml by convention. The top-level nodes of this descriptor consists of the following fields:

  • name - the name of automation package
  • schemaVersion - the version of the YAML automation package format (json schema). This field is optional, but recommended to keep the information about the json schema version you have used and to be notified in case of mismatch.
  • keywords - fragment containing the keywords specifications
  • plans - fragment containing the plans specifications
  • schedules - fragment containing the schedule specifications
  • alertingRules - fragment containing the alerting rules specifications

Each fragment is optional.

Example:

---
schemaVersion: 1.0.0
name: "load-testing-k6-automation-package"
keywords:
  - K6:
      name: 'OpenCart home'
      vus: 1
      iterations: 1
      scriptFile: "opencart-test.js"
plans:
  - name: "Opencart load test plan"
    root:
      testScenario:
        children:
          - threadGroup:
              users: 5
              iterations: 10
              children:
                - callKeyword:
                    keyword: "OpenCart home"

Using YAML fragment files

YAML fragments can be used to structure Automation Package descriptors using multiple YAML files.

For instance the main automation-package.yaml could only contain references to dedicated YAML fragments:

schemaVersion: 1.0.0
name: "My Automation Package"
fragments:
- "keywords.yml"
- "plans.yml"
- "schedules.yml"

The keywords.yml file content would then focus on defining the keywords of this automation package:

keywords:
  - JMeter:
      name: "JMeter keyword from automation package"  
      description: "JMeter keyword 1"
      jmeterTestplan: "jmeterProject1/jmeterProject1.xml"

Keywords

Automation Packages currently support Java-based keywords (Playwright, Selenium, etc), JMeter keywords and Grafana K6 keywords. You may read the related documentation for details on the configuration properties related to each keyword type, the same are available in YAML format.

Java keywords

Java keywords are supported natively with the Java @Keyword annotation and do not have to be specified in the YAML descriptor. In other words, any Keywords defined in the Java code contained in the automation package will be part of the package.

JMeter example

keywords:
  - JMeter:
      name: "JMeter keyword from automation package"  
      description: "JMeter keyword 1"
      jmeterTestplan: "jmeterProject1/jmeterProject1.xml"

K6 example

keywords:
  - K6:
      name: 'OpenCart home'
      vus: 1
      iterations: 1
      scriptFile: "opencart-test.js"

Plans

Plans are fully supported by the YAML format meaning that you can define plans in YAML identical to the ones created with the visual editor in Step. To understand the purpose and configuration of the many plan controls, refer to the dedicated documentation

Plans syntax

The plans are represented by a tree structure as in the visual plan editor and must contain a root node defining the type of plans (test set, test case, test scenario …). The different nodes share at least the following common properties:

  • nodeName - [optional] name this node
  • description - [optional] some textual description for the node
  • children - [optional] the list of children of this node

For instance, the following YAML descriptor contains a single plan of type test case invoking a keyword named “My Keyword”

---
name: "My automation package"
plans:
  - name: "Simple plan"
    root:
      testCase:
        children:
          - callKeyword:
              keyword: "My Keyword"

Convert visual plan to YAML

To convert existing plans to YAML or to help you develop plans in YAML syntax, you may generate the plan in YAML format from the visual plan editor.

Convert Step visual plans to YAML plans

Advanced keyword call

In order to pass inputs to a keyword or to route the execution of the said keyword to specific agents, following syntax is used:

---
name: "My automation package"
plans:
  - name: "Simple plan"
    root:
      testCase:
        children:
          - callKeyword:
              keyword: "My Keyword"
              inputs:
                - myInputNumber: 1
                - myInputBoolean: true
                - myInputString: "some input"
              routing:
                - agentOS: "windows"

Dynamic expressions

As in the visual plan editor many input fields for plan controls accept either a literal value or a dynamic expression evaluated with groovy, for instance to resolve the value of a variable. The same is possible with the YAML syntax.

Example of a thread group defined with literal values:

- threadGroup:
  users: 5
  iterations: 10

Example of a thread group defined with variables:

- threadGroup:
  users: 
    expression: "nbUsers"
  iterations: 
    expression: "nbIterations"

Controls - YAML specificities

While most controls properties are identical to the one of the visual plans or self-explanatory, there a few exceptions.

AssertMetric

In YAML, the metrics names are slightly different than in the UI, here are the mapping from YAML to UI:

  • response-time: “Response time”
  • executions/count: “Execution count”
  • executions/failure-percentage: “Execution failure percentage”
  • executions/failure-count: “Execution failure count”
  • executions/failures-count-by-error-code: “Execution failure count by error code”

Schedules

Schedules can be defined in the YAML descriptor too using the same configuration properties as in the Step UI. Note if you need help to define the CRON expression we recommend using the Step UI wizard.

Example:

schedules:
  - name: "Opencart synthetic monitoring schedule"
    #CRON expression for one execution per minute
    cron: "0 0/1 * * * ?"
    planName: "Opencart synthetic monitoring plan"

Alerting rules

Alerting rules can as well be defined in YAML as you would do it in the UI. Example:

alertingRules:
  - description: "Send email for incident created for Opencart synthetic monitoring"
    name: "Opencart synthetic monitoring email"
    eventClass: IncidentOpenedEvent
    actions:
      - NotificationViaGatewayAction:
          gatewayName: "Email notification gateway"
          gatewayParameters:
            - "changeme@changeme.com"

Setup your IDE for auto-completion and validation

The JSON schema will be published soon in a central repository (schemastore.org), for now they can be directly downloaded here and used in IntelliJ:

In IntelliJ IDEA verify that the use of remote JSON Schemas is enabled as follow

  1. In IDEA settings you need to open the Languages & Frameworks, and set the Use schemastore.org JSON Schema catalog flag in Remote JSON Schemas subsection there.

Then you can setup bindings to link the Step YAML plan files with the actual JSON Schema published to schemastore.org:

  1. In IDEA setting open the JSON Schema Mappings in Languages & Frameworks section.
  2. Find a choose the actual JSON Schema for Step YAML automation packages on schemastore.org (EE or OS version)
  3. Choose the files you want to link with selected JSON Schema

After that you can use the chosen schema in the embedded YAML editor:

  1. When you type something in the YAML editor linked with JSON Schema, IDEA shows the on-the-fly suggestions and validates entered data according to the selected JSON Schema
  2. For the current file you can change the link to the JSON Schema (switch schema from OS version to EE or update schema to the latest version)

See Also