Properties

• Key : the name of the variable to be defined or updated
• Expression : the value of the variable as string or as groovy expression to be interpreted (with groovy evaluation enabled)

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page. The following example shows how to assign a string to a variable in a YAML plan and read its content in an Echo afterwards:

---
version: "1.0.0"
name: "Plan example"
root:
  testCase:
    nodeName: "Plan example"
    children:
    - set:
        key: "myVar"
        value: "Hello world"
    - echo:
        text:
          expression: "myVar"

Following example shows how to assign a value to a variable using a groovy expression. The example would log “2” in the execution report.

---
version: "1.0.0"
name: "Plan example"
root:
  testCase:
    nodeName: "Plan example"
    children:
    - set:
        key: "myVar"
        value:
          expression: "1 + 1"
    - echo:
        text:
          expression: "myVar"

Properties

There is no specific property using this control with natural language, see below example for usage.

Examples

// Set - City to Basel
Set City="Basel"
// Echo - City
Echo City
// Set - City to Bern
Set City="Bern"
// Echo - City
Echo City

Not yet documented.

Properties

• Text : the data you want to display (can be a groovy expression if Groovy mode is toggled)

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

The following example shows how to print a simple string in a Plan:

---
version: "1.0.0"
name: "Plan example"
root:
  testCase:
    nodeName: "Plan example"
    children:
    - echo:
        text: "Hello world"

The following example shows how to print the result of an expression:

echo:
  text:
    expression: "1 + 1"

Properties

There is no special property in natural language.

Examples

// Echo - Hello World !
Echo "Hello World"

Not yet documented.

Properties

• Duration : the amount of time to sleep.
• Unit : ‘ms’ for milliseconds (default), ’s’ for seconds and ’m’ for minutes.
• Advanced :
  • Release Tokens : release tokens reserved in current session (details).

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

Following example shows how to sleep 1s before performing another action in a Plan:

---
version: "1.0.0"
name: "Plan example"
root:
  testCase:
    nodeName: "Plan example"
    children:
    - sleep:
        duration: 1
        unit: "s"
    - echo:
        text: "After 1s sleep"

Properties

• Duration : the amount of time to sleep.
• Unit : ‘ms’ for milliseconds, ’s’ for seconds and ’m’ for minutes.
• ReleaseTokens: (optional, default is false) release tokens reserved in current session (details)

Examples

// Sleep control with a duration of 3 seconds
Sleep Duration=3 Unit="s"
//Optional parameter 'ReleaseTokens' (false by default)
Sleep Duration=100 Unit="ms" ReleaseTokens=true

// Simplified syntax (ReleaseTokens option and Groovy String interpolation not supported)
Sleep 100ms
Sleep 1m
Sleep 1h

Not yet documented.

Properties

• Script: the groovy code to be executed

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

The following example shows how to use the Script control to add elements to an array:

---
version: "1.0.0"
name: "Plan example"
root:
  testCase:
    nodeName: "Plan example"
    children:
    # Declare an array
    - set:
        key: "myArray"
        value:
          expression: "[]"
    # Add elements to the array using the Script control
    - script:
        script: "myArray.add(\"Apple\")\nmyArray.add(\"Orange\")"
    # Output the array to the execution report
    - echo:
        text:
          expression: "myArray"

Properties

There is no special property in natural language.

Examples

-
// Script - A custom script
Script 
result = 5 + 7;
-

Not yet documented.

Properties

• Expression : the expression to be asserted

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

Following examples shows how to perform an assertion with the Check control. In this example we assert that the value of the variable ‘myVar’ is equal to ‘Hello’

---
version: "1.0.0"
name: "Example plan"
root:
  sequence:
    nodeName: "Example plan"
    children:
    - set:
        key: "myVar"
        value: "Hello"
    - check:
        expression: "myVar == 'Hello'"

Properties

• Expression : the expression to be asserted

Examples

 // DefineLocation
DefineLocation
// Check the return value
Check Expression=previous.location.toString().equals('Basel')

Not yet documented.

Properties

• Output attribute : the key of the Keyword output property to be validated. In order to perform assertions on complex Keyword outputs, this field also supports JSONPath to identify the output property.
• Operator : the type of verification
• Expected value : the value you want to check against
• Negate the assertion: tick the box to enable the negation of the value you want to check against
• Custom error message: display a custom error message when the assertion fails

Available operator values are :

• equals : asserts true if the specified output attribute value and the expected value are strictly equals
• begins with : asserts true if the specified output attribute value begins with the expected value
• contains : asserts true if the specified output attribute contains the expected value
• ends with : asserts true if the specified output attribute value ends with the expected value
• matches : asserts true if the specified output attribute value matches the expected value (regular expression can be used as expected value)
• less than (less than or equals): asserts true if the specified numeric output attribute value is less than (or equal to) the expected value
• greater then (greater than or equals): asserts true if the specified numeric output attribute value is greater than (or equal to) the expected value
• is null: asserts true if the specified output attribute value is null or the output attribute is missing

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

Following example shows how to assert the value of a Keyword output. After the call to the Keyword HttpRequest, the Assert control asserts that the Keywourd output StatusCode equals “200”. To test this example, first upload the Keyword package step-library-kw-http. Refer to Keyword deployment for instructions on deploying a Keyword package.

---
version: "1.0.0"
name: "Plan example"
root:
  testCase:
    nodeName: "Plan example"
    children:
    - callKeyword:
        keyword: "HttpRequest"
        inputs:
        - URL: "https://www.exense.ch"
        children:
        - assert:
            actual: "StatusCode"
            operator: "EQUALS"
            doNegate: false
            expected: "200"
            customErrorMessage: ""

Properties

There is no specific property using this control with natural language, see below example for usage.

Examples

 // FindCity
FindCity

// Assert - City equals "Basel"
Assert City = "Basel"
// Assert - City begins with "Basel"
Assert City beginsWith "Basel"
// Assert - City contains "Basel"
Assert City contains "Basel"
// Assert - City ends with "Basel"
Assert City endsWith "Basel"
// Assert - City matches "Basel" (regex)
Assert City ~ "Basel"
// Assert - Count is less than 10
Assert Count < 10
// Assert - Count is less than or equal to 10
Assert Count <= 10
// Assert - Count is greater than 10
Assert Count > 10
// Assert - Count is greater than or equal to 10
Assert Count >= 10
// Assert - City is missing or null
Assert City isNull

Not yet documented.

Properties

• Values : has to be “report.attachments” with groovy evaluation enabled
• File : the file or directory path where to export the attachments
• Prefix : prefix to be prepended to the exported file names
• Filter : regular expression to use in order to filter files to export on their name

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

This control is not supported using natural language.

Not yet documented.

Properties

• Continue sequence execution on error : specify if the sequence should continue until its end if an error occurred within
• Pacing : minimum execution time for the whole sequence to be executed (in milliseconds) as a Long

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

Following example shows how to define a simple sequence of 2 steps:

---
version: "1.0.0"
name: "Plan example"
root:
  sequence:
    nodeName: "Plan example"
    continueOnError: false
    children:
    - echo:
        text: "Step 1"
    - echo:
        text: "Step 2"

Properties

• ContinueOnError : <true|false> specify if the sequence should continue until its end if an error occurred within
• Pacing : minimum execution time for the whole sequence to be executed (in milliseconds) as a Long

Examples

// Sequence - Order Keywords execution
Sequence ContinueOnError = true Pacing = 1000l
    // MyFirstKeyword
  MyFirstKeyword
  // MySecondKeyword
  MySecondKeyword
End

Not yet documented.

Properties

• Continue sequence execution on error : specify if the sequence should continue until its end if an error occurred within
• Pacing : minimum execution time for the whole sequence to be executed (in milliseconds) as a Long

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Properties

• Continue sequence execution on error : specify if the sequence should continue until its end if an error occurred within
• Pacing : minimum execution time for the whole sequence to be executed (in milliseconds) as a Long

Examples

//Set - declare myMessage variable
Set myMessage=''
Sequence
  BeforeSequence
    //Set - init myMessage once
    Set myMessage="Hello World"
  End
  //Echo - myMessage
  Echo myMessage 
End

Not yet documented.

Properties

• Continue sequence execution on error : specify if the sequence should continue until its end if an error occurred within
• Pacing : minimum execution time for the whole sequence to be executed (in milliseconds) as a Long

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Properties

• Continue sequence execution on error : specify if the sequence should continue until its end if an error occurred within
• Pacing : minimum execution time for the whole sequence to be executed (in milliseconds) as a Long

Examples

//Set - declare myMessage variable
Set myMessage=''
Sequence
  BeforeSequence
    //Set - init myMessage once
    Set myMessage="Hello World"
  End
  //Echo - myMessage
  Echo myMessage
  //AfterSequence - cleanup myMessage variable
  AfterSequence
    //Set - empty myMessage
    Set myMessage=""
  End
End

Not yet documented.

Properties

• Condition : the expression evaluated to know if the content of the node is executed

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

if:
  condition:
    expression: "true"
    children:
      - echo:
          text: "True is always true :)"

Properties

No condition for this control used with natural language. See below example for usage.

Examples

If 1==1
  Echo "Condition met"
End

Not yet documented.

Switch property

• Expression : the expression to switch on to

Case property

• Value : the value to be potentially matched

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

Performs different actions depending on the value of a random variable randomValue using the Switch control

---
version: "1.0.0"
name: "Example plan"
root:
  sequence:
    nodeName: "Switch case example"
    children:
    - set:
        key: "randomValue"
        value:
          expression: "(new java.util.Random().nextInt(2)).toString()"
    - switch:
        expression:
          expression: "randomValue"
        children:
        - case:
            value: "0"
            children:
            - echo:
                text: "The value was 0"
        - case:
            value: "1"
            children:
            - echo:
                text: "The value was 1"

This controls are not supported using natural language.

Not yet documented.

Properties

• Lock name: the name of the lock, leave empty if you want this lock to synchronize on this plan node
• Global lock: select this check-box if you want this lock to be global across all the executions

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

In the following example, the Synchronized control ensures that the Echo control is never run in parallel

---
version: "1.0.0"
name: "Example plan"
root:
  threadGroup:
    nodeName: "Plan root"
    users: 10
    iterations: 10
    children:
    - synchronized:
        lockName: ""
        globalLock: false
        children:
        - echo:
            text: "This echo is never run in parallel"

Properties

• LockName: (optional) when unspecified, synchronization happens on this plan node
• GlobalLock: (optional) set this property to true if you want this lock to be global across all the executions

Examples

Synchronized LockName="myLockName" GlobalLock=true
  // MyFirstKeyword
  MyFirstKeyword
  // MySecondKeyword
  MySecondKeyword
End

Not yet documented.

Properties

• Start : the initial value of the for loop
• End : the final value of the for loop (exit condition)
• Increment : the increment value at the end of each iteration
• Counter variable (handle) : the name of the variable containing the value of the iteration counter
• Number of threads : the number of threads to use in order to parallelize the loop (i.e the execution of the children)
• Maximum number of failures : the number of failures before the loop is stopped. If empty, the loop is never stopped

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

---
version: "1.0.0"
name: "Example plan"
root:
  sequence:
    nodeName: "Plan root"
    children:
    - for:
        start: 1
        end: 10
        inc: 1
        item: "counter"
        threads: 1
        children:
        - echo:
            text:
              expression: "\"Counter: ${counter}\""

Properties

• Start : the initial value of the for loop
• End : the final value of the for loop (exit condition)

Examples

// For - display counter value
For 1 to 10 
  // Echo - counter value
  Echo counter
End

Not yet documented.

Properties

• Condition : a groovy condition to be evaluated at the beginning of each iteration
• Post Condition : a groovy condition to be evaluated at the end of each iteration
• Pacing: : minimum execution time in ms for the node content to be executed as an Long
• Timeout : stop the control execution after the defined value as a Long (serves as a security)
• MaxIterations : stop th control execution after the define amount of iterations (serves as a security)

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

In the following example, the While control loops as long as the value of the variable counter remains below 5.

---
version: "1.0.0"
name: "Example plan"
root:
  sequence:
    nodeName: "Plan root"
    children:
    - set:
        key: "counter"
        value:
          expression: "0"
    - while:
        condition:
          expression: "counter < 5"
        children:
        - set:
            key: "counter"
            value:
              expression: "counter + 1"
        - echo:
            text:
              expression: "\"Counter: $counter\""

Properties

• Condition : a groovy condition to be evaluated at the beginning of each iteration. Please mind the “|” surrounding the condition expression.
• Post Condition : a groovy condition to be evaluated at the end of each iteration. Please mind the “|” surrounding the condition expression.
• Pacing: : minimum execution time in ms for the node content to be executed as an Long
• Timeout : stop the control execution after the defined value as a Long (serves as a security)
• MaxIterations : stop the control execution after the define amount of iterations (serves as a security)

Examples

-
// While
While 
  Condition=|!"Hello World !".equals("Goodbye World !")|
  Pacing=1000l
  Timeout=5000l
  MaxIterations=5
-
// Echo - Hello World
Echo "Hello World"
End

Not yet documented.

Properties

Same as in a regular Keyword call, but a name should be set in order to tell Step which Keyword needs to be called. Used in conjunction with groovy, this can provide a powerful mechanism for Keyword Call templating.

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

The following example shows how to use the CallKeyword control to call a Keyword by name. To test this example, first upload the Keyword package step-library-kw-http that declares the Keyword HttpRequest. Refer to Keyword deployment for instructions on deploying a Keyword package.

---
version: "1.0.0"
name: "Plan example"
root:
  sequence:
    nodeName: "Plan example"
    children:
    - callKeyword:
        keyword: "HttpRequest"
        inputs:
        - URL: "www.exense.ch"

In the following example, a Keyword is called dynamically based on the value of the variable keywordToCall which specifies the name of the Keyword to call.

---
version: "1.0.0"
name: "Example plan"
root:
  sequence:
    nodeName: "Plan root"
    children:
    - set:
        key: "keywordToCall"
        value: "HttpRequest"
    - callKeyword:
        keyword:
          name:
            expression: "keywordToCall"

Properties

Same as in a regular Keyword call, but a name should be set in order to tell Step which Keyword needs to be called. Used in conjunction with groovy, this can provide a powerful mechanism for Keyword Call templating.

Limitations

You can only have String variable using natural language (i.e boolean, numbers and date are not supported). You can however use the Groovy String interpolation to resolve variable value directly in a String. Application prefixed keywords syntax is supported as well as Application / Keyword having spaces in their name, thus the keyword call needs to be double-quoted when using plain text (see example below).

Examples

// CallKeyword - MyFirstKeyword
MyFirstKeyword location="Bern" date="01.07.2019" contact="${John Doe}"
// My Application -> My Second Keyword
"My Application.My Second Keyword"

Not yet documented.

Properties

• Keyword routing : optional, for affinity patterns using selection criteria

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

In the following example, we assume that we have 2 stateful Keywords Login to MyApplication and Perform a specified action in MyApplication that share some session objects. The Session control ensures that these 2 Keywords are executed on the same agent token.

---
version: "1.0.0"
name: "Example plan"
root:
  sequence:
    nodeName: "Plan root"
    children:
    - session:
        children:
        - callKeyword:
            keyword: "Login to MyApplication"
        - callKeyword:
            keyword: "Perform a specified action in MyApplication"

Properties

• Keyword routing : optional, for affinity patterns using selection criteria (see below example for usage)

Examples

// Session 
Session key1=value1 key2=value2
  // InitWebDriver - And put it into session
  InitWebDriver
  // NavigateToUrl - But first get the driver from session
  NavigateToUrl
End

Not yet documented.

Properties

• keys & values : the plan you want to invoke, as for Keywords and CallKeywords, CallPlan can be used for templating Plan executions.

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

---
version: "1.0.0"
name: "Example plan"
root:
  sequence:
    nodeName: "Plan root"
    children:
    - callPlan:
        nodeName: "Example plan"
        planId: "66bc5df58307a91fca497056"

Properties

The only property exposed at the moment is the plan name to be called, passing inputs is not supported in natural language yet.

Examples

-
// Call Plan - mySubPlan
Call plan "mySubPlan" 
-

Not yet documented.

Properties

There is no specific property configurable for this control.

Examples

Main plan echoing the counter value while the for loop incrementing the counter is defined in a sub-plan as a placeholder :

Execution results :

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

This control is not supported using natural language.

Not yet documented.

Properties

Simply wrap your Test Plans with a test case

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

---
version: "1.0.0"
name: "Example plan"
root:
  testSet:
    nodeName: "Plan root"
    threads: 1
    children:
    - testCase:
        nodeName: "TestCase A"
        children:
        - echo:
            text: "Running TC A"
    - testCase:
        nodeName: "TestCase B"
        children:
        - echo:
            text: "Running TC A"

Properties

Simply wrap your Test Plans with a test case

Examples

 TestCase
// MyFirstKeyword
MyFirstKeyword location="Bern" date="01.07.2019" contact="${John Doe}"
End

Not yet documented.

Properties

Simply wrap your TestCase’s with a TestSet controller, parallelism is derived from the global variable tec.execution.threads

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

The following example shows how to define a test set that calls 2 test case plans by Id

---
version: "1.0.0"
name: "Example plan"
root:
  testSet:
    nodeName: "Plan root"
    threads: 1
    children:
    - callPlan:
        nodeName: "Call Test case A"
        planId: "66bc5df58307a91fca497056"
    - callPlan:
        nodeName: "Call Test case B"
        planId: "66bc5df58307a91fca497056"

Properties

Simply wrap your TestCase’s with a TestSet controller, parallelism is derived from the global variable tec.execution.threads

Examples

TestSet 
  TestCase
  // CallKeyword - MyFirstKeyword
  MyFirstKeyword location="Bern" date="01.07.2019" contact="${John Doe}"
  End
  TestCase
  // CallKeyword - MySecondKeyword
  MySecondKeyword 
  End
End

Not yet documented.

Wizard

With the wizard the values for number of threads and iterations per thread can be easily calculated. It requires the following information:

• Target throughput : the expected number of iterations produced over specified time during the execution
• Estimated iteration durations max : the duration to execute one iteration under load, considering to keep sufficient margin. If the actual iteration time is higher during the execution, the target throughput will not be reached.

Properties (Advanced)

• Number of threads : the number of threads to be started in parallel
• Number of iterations per thread : the number of iteration each thread will perform
• Pacing : defines the fix period of time that the ThreadGroup will try to keep between the start of each iteration. In combination with the number of threads, this allows to control the number of iterations per time unit (so that throughput remains independent of the actual duration of each iteration)
• Rampup : defines the duration of the ramp-up i.e. the time that the ThreadGroup will take to start all the threads. If not set, the pacing value is taken as default rampu-up
• Start offset : defines an optional offset of time before the ramp-up start
• Maximum duration : the maximal execution time for the whole thread group

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

Following examples defines a ThreadGroup with 10 threads, 100 iterations per thread and a pacing of 500ms. It will execute the child nodes (Keyword 1 and Keyword 2) 100 times per thread

---
version: "1.0.0"
name: "Plan example"
root:
  threadGroup:
    nodeName: "Plan example"
    users: 10
    iterations: 100
    pacing: 500
    startOffset: 0
    maxDuration: 0
    children:
    - callKeyword:
        keyword: "Keyword 1"
    - callKeyword:
        keyword: "Keyword 2"

Properties

• Threads : the number of threads to be started in parallel
• Iterations : how many times the node content should be executed per thread
• Pacing : defines a fixed time window in milliseconds, as an Integer, in order to better control the number of keywords executed per time unit (and so that throughput remains independent from keyword execution time)
• Rampup : the ramp-up value is designed to be automatically divided by the number of threads to start them progressively and “spread” them out evenly, it represents the total duration of the ramp-up phase. If the ramp up value is left empty, then it will default to the pacing value (all threads will be spread out evenly but started “quickly”, i.e in one single pacing period).
• Offset : the duration in ms to wait before the Rampup starts as an Integer
• MaxDuration : the maximal execution time in ms for the whole thread group as an Integer

Examples

-
// ThreadGroup
ThreadGroup 
  Threads=5
  Iterations=5
  Pacing=5000
  Rampup=1000
  Offset=100
  MaxDuration=25000
-
  // Echo - Hello World
  Echo "Hello World"
End

Not yet documented.

Properties

• Continue sequence execution on error : specify if the sequence should continue until its end if an error occurred within
• Pacing : minimum execution time for the whole sequence to be executed (in milliseconds) as a Long

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Properties

• Continue sequence execution on error : specify if the sequence should continue until its end if an error occurred within
• Pacing : minimum execution time for the whole sequence to be executed (in milliseconds) as a Long

Examples

//Set - declare myMessage variable
Set myMessage=''
// ThreadGroup
ThreadGroup Threads=2 Iterations=2
  BeforeThread
    //Set - init myMessage once
    Set myMessage="Hello World"
  End
  //Echo - myMessage
  Echo myMessage
End

Not yet documented.

Properties

• Continue sequence execution on error : specify if the sequence should continue until its end if an error occurred within
• Pacing : minimum execution time for the whole sequence to be executed (in milliseconds) as a Long

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Properties

• Continue sequence execution on error : specify if the sequence should continue until its end if an error occurred within
• Pacing : minimum execution time for the whole sequence to be executed (in milliseconds) as a Long

Examples

//Set - declare myMessage variable
Set myMessage=''
// ThreadGroup
ThreadGroup Threads=2 Iterations=2
  BeforeThread
    //Set - init myMessage once
    Set myMessage="Hello World"
  End
  //Echo - myMessage
  Echo myMessage
  //AfterThread - cleanup myMessage variable
  AfterThread
    //Set - empty myMessage
    Set myMessage=""
  End
End

Not yet documented.

Properties

There is no special property for this control.

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

Following examples defines a load test scenario with 2 thread groups:

---
version: "1.0.0"
name: "Example plan"
root:
  testScenario:
    nodeName: "Example plan"
    children:
    - threadGroup:
        users: 10
        iterations: 100
        children:
        - echo:
            text: "Running iteration from TG1"
    - threadGroup:
        users: 20
        iterations: 100
        children:
        - echo:
            text: "Running iteration from TG2"

Properties

There is no special property for this control.

Examples

TestScenario
  -
  // First ThreadGroup
  ThreadGroup 
  -
  // Echo - Hello World
  Echo "Hello World"
  End
  -
  // Second ThreadGroup 
  ThreadGroup
  -
  // Echo - Goodbye World
  Echo "Goodbye World"
  End
End

Not yet documented.

Properties

• Metric to assert : the measurement metric to check the value against.
• Measurement name : the measurement name (can be a custom measurement).
• Operator : the operator to use to compare the measurement metric against the specified value
• Value: the value to compare the measurement metric against

Available metrics to assert are :

• average : the average response time of the selected measurement
• min : the minimal response time of the selected measurement
• max : the maximal response time of the selected measurement
• sum : the sum of the response time of the selected measurement
• count : the number of time the selected measurement has been generated

Available operator values are :

• equals : asserts true if the selected measurement metrics value is equals to the defined value
• lower than : asserts true if the selected measurement metrics value is lower than the defined value
• higher than : asserts true if the selected measurement metrics value is higher than the defined value

Unitary example

In the below example, the PerformanceAssert validates that the average response time of the “testKeyword” execution is lower than 3000 ms.

Count example

In the below example, the PerformanceAssert validates that the “testKeyword” has been executed exactly 10 times.

Agreggation example

In the below example, the PerformanceAssert validates that the aggregation done on the average response time of the “testKeyword” over the 10 loop iterations is lower than 3000 ms.

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

Following example defines a ThreadGroup that runs 10 VUs (threads) in parallel and asserts that the average duraiton of the Keyword call “HelloWorld” is below 500ms

---
schemaVersion: 1.0.0
name: "Load testing plan with PerformanceAssert"
plans:
  - name: "Demo load test"
    root:
      threadGroup:
        # Run the sequence with 10 VUs
        users: 10
        # Perform 10 iterations per user
        iterations: 10
        children:
          # Call the keyword HelloWorld
          - sequence:
              children:
              - callKeyword:
                  keyword: "HelloWorld"
          # Assert that the response time of the measure 'HelloWorld' is below 500m
          - performanceAssert:
              measurementName: "HelloWorld"
              comparator: "LOWER_THAN"
              expectedValue: 500

Properties

• Metric (Optional): specifies the metric to be asserted
  • Type: String
  • Supported values: “Average”, “Max”, “Min”, “Count”, “Sum”
  • Default value: If not set the “Average” metric is used per default
• Measurement : specifies the name of the measurement to be asserted. This could be a regex matching several measurements. In this case all measurements will be grouped by name and each group will be asserted individually
  • Type: String
  • Supported values: any
• Comparator (Optional): the comparator to be used to compare the actual value with the expected value
  • Type: String
  • Supported values: “equals to”, “higher than”, “lower than”
  • Default value: If not set, “lower than” is used per default
• Value : the expected value in ms
  • Type: Number

Examples

Example 1:

For 1 to 10
  MyFirstKeyword
  MySecondKeyword
End
// Assert that the average response time of the Keyword 'MyFirstKeyword' is lower than 100ms
PerformanceAssert Metric = "Average" Measurement = "MyFirstKeyword" Comparator = "lower than" Value = 100

Example 2:

For 1 to 10
MyFirstKeyword
MySecondKeyword
End
// Assert that the average response time of the 2 Keywords 'MyFirstKeyword' is lower than 500ms
PerformanceAssert Measurement = ".*" Value = 500

Not yet documented.

The assert metric control is defined by following properties

• Metric available metrics for the assertion:
  • Execution count: count of execution
  • Execution failure percentage
  • Execution failure count
  • Execution failure count by error code
  • Response times
• Filters: add custom filters (for instance filter on a specific measurement name for Response times metrics)
• Rolling window: specify the time frame to be used when querying the metrics as a rolling window
  • The window starts from the end of the execution triggering the evaluation of the Assertion Plan
  • Per execution metrics, such as execution count and execution failures count, are ingested with the execution start time, therefore the windows must be at least larger than the execution duration
  • Using very small windows (i.e. close to the time series resolution which is by default 5 seconds) may give unpredictable results
• Time series aggregation: define which aggregator to use (i.e. average, min, max, sum, percentiles…)
• Time series group by (optional): to aggregate and assert grouping by specified attributes
• Comparator: used for the assertion, can be either below, above or matching threshold
• Threshold value: The values compared to aggregation result using selected comparator
• Error code: defien the code of the error that will be raised if the assertion fails

In below example, we assert that the number of failed executions in the last hour remains below 2:

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

Following example shows how to define an assertion plan in YAML that asserts that the executions of a schedule did not fail more than once in the last 10 minutes

schemaVersion: 1.0.0
name: "Assertion plan example"
plans:
  - name: "Opencart schedule assertion plan"
    root:
      assertionPlan:
        description: "Assert the scheduled executions did not fail more than once in the last 10 minutes"
        children:
          - assertMetric:
              description: "Make sure executions did not fail more than once in the last 10 minutes"
              comparator: LOWER_THAN
              aggregation: SUM
              expectedValue: 2
              slidingWindow: 600000
              metric: "executions/failure-count"

Not yet documented.

Not yet documented.

Common Properties

• Source type : define the source type object. Values could be : Excel, CSV, SQL, Flat file, Directory, Integer sequence, Json String, Google Sheet V4
• For write : define if the data source will be used to insert values
• Iterator handle : define a variable pointing to the dataset’s iterator
  • Rows can then be iterated and accessed with the groovy expression iteratorName.next(), see our example.
• Reset at end : define if once arrived at the end of the data source, the iterator handle should be reinitialized to the first row

Example - using DataSet in load testing plans

While designing load testing plans it is a common need to run a workflows with a set of data, picking up different rows across threads and iterations.. To run a workflow in parallel, Step provides the control ThreadGroup. Using ThreadGroup and DataSet in combination enables the execution of a workflow in parallel with different data.

You can directly import this sample plan in Step

Excel properties

• Excel file : the path to your source file
• Worksheet : the worksheet name in your source file (if blank, the first one in your sheet will be selected)
• Headers : check this box if your columns have header

CSV properties

• CSV file : the path to your source file
• Delimiter : the delimiter used in your CSV file

Example

SQL properties

• Connection string : the string used to connect to your database. For example, using jdbc to connect to a MySQL database on localhost : “jdbc:mysql://localhost:3306/dbname”
• Driver class : the driver full class name. For example, using jdbc for MySQL : “com.mysql.jdbc.Driver”
• Query : the SQL query to perform against your database to retrieve the data
• User : the username to connect to your database
• Password : the password to connect to your database. If you want to store your password in a Parameter and thus also protect it, you can use following syntax: “parameter:MyParameterName”. Example: if your password is store in a parameter called “datasource.sql.password”, you can refer to is as follow: “parameter:datasource.sql.password”
• Primary write key : the primary key to use in case you want to write back to your database

Flat file properties

• Flat file : the path to your source file

Directory properties

• Directory : the path to your directory

Integer sequence properties

• Start : value starting the sequence
• End : value ending the sequence
• Inc : increment value for the sequence

Json String properties

• Json string : the JSON document representing your data. The document has to represent of table of values with headers first, and then rows. For example :

{ "pagename" :
    ["Consulting", "Products"],
  "url" : 
    ["https://www.exense.ch/solutions",
      "https://www.exense.ch/products"]}

Google Sheet V4 properties

• Service Account Key File : the path to your service account key file. Refer to Google documentation about service account key file creation.
• File Id : the Google Cloud unique identifier of your file. It can usually be extracted from the URL locating your file in the Cloud.
• Tab Name : the name of the tab to use in your source file (if blank, the first one in your file will be selected)

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

Following example shows how to use a DataSet in combination with a ThreadGroup. In this example the dataSet uses a Json as source and is reset when the end of the data is reached. The DataSet is used within the ThreadGroup. In each iteration the method next() of the DataSet is called to move to the next line.

---
version: "1.0.0"
name: "Example plan"
root:
  sequence:
    nodeName: "Example plan"
    children:
    - dataSet:
        item: "dataSet"
        resetAtEnd: true
        dataSource:
          json-array:
            forWrite: false
            json: "[{\"Username\":\"User1\"},{\"Username\":\"User2\"}]"
    - threadGroup:
        users: 1
        iterations: 10
        children:
        - echo:
            text:
              expression: "'Username: ' + dataSet.next().Username"
    continueOnError: false

Common Properties

• SourceType : define the source type object. Values could be : excel, csv, file (for Flat file) or folder (for Directory). Other source type are not supported yet.
• File : the path to your source file
• ForWrite : define if the data source will be used to insert values
• IteratorHandle : define a variable pointing to the dataset’s iterator
• ResetAtEnd : define if once arrived at the end of the data source, the iterator handle should be reinitialized to the first row

Excel Properties

There is no specific property configurable for Excel sheet in natural language.

Examples

-
// DataSet - from Excel sheet
DataSet 
  SourceType = "excel"
  File = "myFile.xlsx"
  IteratorHandle = "myRows"
  ForWrite = true
  ResetAtEnd = true
-

CSV properties

There is no specific property configurable for CSV file in natural language.

Warning: Writing to a CSV file from a dataset is not supported

Examples

-
// DataSet - from CSV file
DataSet 
  SourceType = "csv"
  File = "myFile.csv"
  IteratorHandle = "myRows"
  ForWrite = true
  ResetAtEnd = true
-

File properties

There is no special property for “File” type in natural language.

Examples

-
// DataSet - from Flat file
DataSet 
  SourceType = "file" 
  File = "myFile.txt"
  IteratorHandle = "myRows"
  ForWrite = true
  ResetAtEnd = true
-

Folder properties

There is no special property for “Folder” type in natural language.

Examples

-
// DataSet - from Directory
DataSet 
  SourceType = "folder"
  File = "/myFolder"
  IteratorHandle = "myRows"
  ForWrite = true 
  ResetAtEnd = true 
- 

Not yet documented.

Common Properties

• Source type : define the source type object. Values could be : Excel, CSV, SQL, Flat file, Directory, Integer sequence, Json String, Google Sheet V4
• Number of threads : the number of threads to use in order to parallelize the loop (i.e the execution of the children)
• Maximum number of failures : the number of failures before the loop is stopped. If empty, the loop is never stopped
• Row handle : define a variable pointing to the current row

Excel properties

• Excel file : the path to your source file
• Worksheet : the worksheet name in your source file (if blank, the first one in your sheet will be selected)
• Headers : check this box if your columns have header

Example

CSV properties

• CSV file : the path to your source file
• Delimiter : the delimiter used in your CSV file

Example

SQL properties

• Connection string : the string used to connect to your database. For example, using jdbc to connect to a MySQL database on localhost : “jdbc:mysql://localhost:3306/dbname”
• Driver class : the driver full class name. For example, using jdbc for MySQL : “com.mysql.jdbc.Driver”
• Query : the SQL query to perform against your database to retrieve the data
• User : the username to connect to your database
• Password : the password to connect to your database
• Primary write key : the primary key to use in case you want to write back to your database

Flat file properties

• Flat file : the path to your source file

Directory properties

• Directory : the path to your directory

Integer sequence properties

• Start : value starting the sequence
• End : value ending the sequence
• Inc : increment value for the sequence

Json array properties

• Json : the JSON document representing your data. The document has to be an array of objects:

[{
		"name": "Dave Null",
		"age": 29,
		"profession": "SysAdmin"
	}, {
		"name": "Seg Fault",
		"age": 54,
		"profession": "Developer"
	}
]

You will then be able to use the Row handle variable as a Json object by, for example, accessing the name and age with row.name and row.age

Example

Google Sheet V4 properties

• Service Account Key File : the path to your service account key file. Refer to Google documentation about service account key file creation.
• File Id : the Google Cloud unique identifier of your file. It can usually be extracted from the URL locating your file in the Cloud.
• Tab Name : the name of the tab to use in your source file (if blank, the first one in your file will be selected)

Json String (Legacy) properties

• Json string : the JSON document representing your data. The document has to represent of table of values with headers first, and then rows. For example :

{ "pagename" :
    ["Consulting", "Products"],
  "url" : 
    ["https://www.exense.ch/solutions",
      "https://www.exense.ch/products"]}

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Example

The following example shows how to define a Plan that iterates over all lines of a CSV file delimited by ‘,’ and echo the value of the column “Column1” (the first row of the CSV is expected to contain the headers and thus one column with the text “Column 1”)

---
version: "1.0.0"
name: "Example plan"
root:
sequence:
  nodeName: "Example plan"
  children:
  - forEach:
      dataSource:
        csv:
          file:
            id: "/path/to/file.csv"
          delimiter: ","
      item: "row"
      threads: 1
      children:
      - echo:
          text:
            expression: "row.Column1"

Common Properties

• SourceType : define the source type object. Values could be : excel, csv, file (for Flat file) or folder (for Directory). Other source type are not supported yet.
• File : the path to your source file
• Threads : the number of threads to use in order to parallelize the loop (i.e the execution of the children)
• RowHandle : define a variable pointing to the current row
• MaxFailedLoop : the number of failures before the loop is stopped. If not set, the loop is never stopped

Excel Properties

• Worksheet : the worksheet name in your source file (if blank, the first one in your sheet will be selected)

Examples

-
// ForEach - row in excel sheet
ForEach 
  SourceType = "excel"
  File = "C:/myFile.xlsx"
  Worksheet = "myWorksheet" 
  Threads = 1 
  RowHandle = "myRow"
  MaxFailedLoop = 2
-
  Echo myRow.myColumn
End

CSV properties

• Delimiter : the delimiter used in your CSV file

Examples

-
// ForEach - row in file
ForEach 
  SourceType = "csv"
  File = "/home/myFile.csv"
  Delimiter = ";"
  Threads = 2 
  RowHandle = "myRow"
-
  Echo myRow.myColumn
End 

File properties

There is no special property for “File” type in natural language.

Examples

-
// ForEach - row in file
ForEach 
  SourceType = "file"
  File = "C:/myFile.txt"
  Threads = 2 
  RowHandle = "myRow"
-
  Echo myRow.myColumn
End

Folder properties

There is no special property for “Folder” type in natural language.

Examples

-
// ForEach - file in folder
ForEach 
  SourceType = "folder"
  File = "/myFolder"
  Threads = 2 
  RowHandle = "myRow"
-
  Echo myRow.myColumn
End 

Not yet documented.

Properties

• Max retries : number of retries if child nodes execution fails
• Grace period (ms) : duration to wait in ms after a failed attempt
• Timeout : maximum execution time in ms for the whole execution retries
• Advanced :
• Release Tokens : release tokens reserved in current session (details)
• Report only last iteration: when enabled, only the final iteration will be reported and persisted

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

The following example shows how to use the RetryIfFails to rerun a Keyword until its status is PASSED. In this example the RetryIfFails will try to re-execute the Keyword 2 times maximum and wait 1000ms after each retry.

---
version: "1.0.0"
name: "Example plan"
root:
  sequence:
    nodeName: "Example plan"
    children:
    - retryIfFails:
        children:
        - callKeyword:
            keyword: "DummyKeyword"
        maxRetries: 2
        gracePeriod: 1000
        timeout: 0
        releaseTokens: false
        reportLastTryOnly: false

Properties

• MaxRetries : number of retries if child nodes execution fails
• GracePeriod : duration to wait in ms after a failed attempt
• Timeout : maximum execution time in ms for the whole execution retries
• ReleaseTokens: (optional, default is false) release tokens reserved in current session (details)
• ReportLastTryOnly: (optional, default is false) when enabled, only the final iteration will be reported and persisted

Examples

RetryIfFails MaxRetries=3 GracePeriod=5000 Timeout=30000
// CallKeyword - MyFirstKeyword
MyFirstKeyword location="Bern" date="01.07.2019" contact="${John Doe}"
End

//Same example with all properties
RetryIfFails MaxRetries=3 GracePeriod=5000 Timeout=30000 ReportLastTryOnly=true ReleaseTokens=true
// CallKeyword - MyFirstKeyword
MyFirstKeyword location="Bern" date="01.07.2019" contact="${John Doe}"
End

Not yet documented.

Properties

• Event selection criteria: either select event by id or by group and name (name is optional)
• Event Id: the ID of the event to wait for
• Event Group: the event group to wait for
• Event Name: the event name to wait for
• Polling settings : adapt the polling frequency and control optionally the maximum wait time
• Polling frequency (ms): the interval to poll the event broker in ms as a Long
• Max number of polling: stop waiting for the event after the define amount of iterations (serves as a security)
• Maximum wait time (ms): stop waiting to the event after the defined value as a Long (serves as a security)
• Remote broker settings: to be used when the events are not stored in the local controller
• Remote URL: the Step remote controller URL where to listen to an incoming event
• Remote User: the Step remote controller username to use for connection
• Remote Password: the Step remote controller password to use for connection
• Advanced:
• Release Tokens: release tokens reserved in current session (details)

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

Following example shows how to use the WaitForEvent in a YAML Plan to wait for an event with the Id ‘myEvent1’ and perform an action upon event reception:

---
version: "1.0.0"
name: "Example plan"
root:
  sequence:
    nodeName: "Example plan"
    children:
    - waitForEvent:
        children:
        - echo:
            text: "Received event 1"
        eventId: "myEvent1"
        pollingSpeed: 500
        timeout: 0
        maxIterations: 0
        releaseTokens: false

Properties

• EventId: the ID of the event to wait for
• Group: the event group to wait for
• Name : the event name to wait for
• PollingSpeed : the interval to poll the event broker looking for the event in ms as a Long
• Timeout : stop waiting to the event after the defined value as a Long (serves as a security)
• MaxIterations : stop waiting for the event after the define amount of iterations (serves as a security)
• ReleaseTokens: (optional, default is false) release tokens reserved in current session (details)

Examples

-
WaitForEvent 
  Group="eventGroup"  
  Name="eventName" 
  PollingSpeed=1000l 
  Timeout=5000l 
  MaxIterations=5
-
// Set - myEvent
Set myEvent=report.getEvent()
// Echo - myEvent
Echo myEvent

Not yet documented.

Properties

No property. Just enter key values as you’d like them to be available in the output of a regular keyword.

Examples

The YAML syntax of Plan controls follows the schema documented on the Automation package descriptor page.

Examples

Following example shows how to return an output within the Plan of a Keyword composite. In this example an output with the key/value pair “output1” = “Output 1” would be returned:

---
version: "1.0.0"
name: "Example plan"
root:
  sequence:
    nodeName: "Example plan"
    children:
    - set:
        key: "myOutput"
        value: "Output 1"
    - return:
        output:
        - output1:
            expression: "myOutput"

Note

This control is not available as natural language as Composite Keywords cannot be created using natural language yet.

Not yet documented.