Setting up system monitoring with a Step agent

This article demonstrates how to set up distributed system monitoring using Keyword executions, and analyze the results as measurements.

This tutorial will demonstrate how to set up distributed system monitoring by leveraging Keyword executions on dedicated token pools, as well as how to analyze the results as numerical measurement in RTM views. This example is based on Windows performance counters using the standard typeperf command. The same approach can be used on Unix with vmstat and can easily be extended and customized to suit other platform needs.

Keyword setup

We will leverage the agent platform and execute a special Keywords at regular intervals in order to poll system metric values and feed them back to Step’s measurement database. Sample Keyword code is provided in our GitHub sample repository under “keywords/java/demo-system-monitoring”. The code contains a generic Keyword for executing managed processes from Java that is extended to cover the Windows typeperf process.

Setup your agent

We will create dedicated execution slots, called token pools, in order to isolate the execution of this technical Keyword from business keywords. You will need to modify the AgentConf.json configuration file in your existing agent setup.

Add an AGENT_TYPE “DEFAULT” to the main tokenGroups
Create a 2nd group with a unique AGENT_TYPE name and a capacity of 1

For the controller and database servers, you will install a dedicated agent with only one token group for the monitoring following the same rules, capacity of 1 and unique AGENT_TYPE value.

Example:

  {
"gridHost":"http://controller:8081",
"agentPort": 31001,
"registrationPeriod":1000,
"gridReadTimeout":20000,
"workingDir":"../work",
"tokenGroups":[
{"capacity":250,
"tokenConf":{
"attributes":{"AGENT_TYPE" : "DEFAULT"},
"properties":{"chromedriver":"../ext/bin/chromedriver/chromedriver.exe"}}
},
{"capacity":1,
"tokenConf":{
"attributes":{"AGENT_TYPE" : "MONITORING_agentName_001"},
"properties":{"chromedriver":"../ext/bin/chromedriver/chromedriver.exe"}}
}
],
"properties":{
"plugins.jmeter.home":"../ext/lib/jmeter",
"plugins.selenium.libs.2.x":"../ext/lib/selenium/selenium-2.53.1",
"plugins.selenium.libs.3.x":"../ext/lib/selenium/selenium-java-3.5.3",
"plugins.qftest.home":"C:/Users/jcomte/Programs/qftest-4.1.2",
"plugins.pdftest.gsexe":"/path/to/gswin64.exe"
}
}

Step controller parameters

Now that a distinct pool is available for monitoring purposes, we will tell Step to route all regular Keywords to the default pool. Later on, we will make sure to route the monitoring Keywords to the other pool in our plan.

Key: rout_to_AGENT_TYPE, value: DEFAULT

Metric configuration and postprocessing

We recommend updating the typeperf counters already defined in the included resource CSV file, “typePerf.csv”

  Metric_name,Typeperf_counter,groovy
CPU(%),\Processor(_Total)\% Idle Time,100-value
MemoryAvailableMB,\Memory\Available MBytes,

Metric name is the name of the metric which will be reported in RTM
Typeperf_counter is the typeperf counter to be retrieved. Refer to windows-commands/typeperf to learn more about typeperf.
groovy is a groovy expression which will be evaluated during the post processing; the resulting value will be stored as the measurement’s value.

The Step plan

The plan in the example below is designed to run for one hour, taking a sample for each defined agent each minute.

Note, the Keyword execution is configured to take place on the new token pool reserved for monitoring. Refer to the routing section of the Keyword call node in the provided plan.

Execution

You have the option to run the plan once and tweak it to match your desired duration and sampling interval or to schedule it to restart every hour as shown below:

Selecting the CRON expression to run every hour will result in the following entry in the scheduler

RTM

Unlike other test executions, all measurements from this Keyword are created using a common execution id, eld, unrelated to the actual Step execution. The common eld used is “OSmonitor” and is used in RTM to retrieve related measurements. If you keep the monitoring running in the background, it is recommended to use a time frame filter in your query. In addition to the metric name, the “hostname” of the monitoring agent is automatically added to your measurements and you may then use it as a group clause:

Which results in the following output:

Housekeeping

You may cleanup the measurement collection based on the OSmonitor eld and timestamp. Refer to our Housekeeping section.

Summary: This article demonstrates how to set up distributed system monitoring using Keyword executions, as well as how to analyze the results as numerical measurements in RTM views.

Using Step with Grafana

This article demonstrates how to connect Grafana to data generated by Step.

NET tutorials: Microsoft Office automation with Step

This tutorial demonstrates how to automate interaction with Microsoft Office applications using the Office Interop Assembly.

JUnit Plan Runner

This article provides documentation for how to integrate JUnit tests into Step.

How to monitor services availability and performance

This tutorial demonstrates how Step can be used to monitor services, availability and performance metrics.

.NET tutorials: AutoIt with Step

This tutorial demonstrates how to utilize the AutoIt C# binding to automate interactions with Windows applications.

Android Testing using Step and Appium

This article demonstrates the automation of mobile applications on Android using the Appium framework.

Browser-based automation with Step and Selenium

This article defines three Keywords which will be used in browser-based automation scenarios, using Step and Selenium, as general drivers.

Illustration for Load Testing with Cypress

Load Testing with Cypress - advanced

This tutorial shows you how to efficiently set up a browser-based load test using existing Cypress tests in the Step automation platform.

Adding and Configuring New Agents

In this short tutorial, we show how to quickly implement a simple browser-based load test based on Cypress scripts in Step.

Illustration for Load Testing with Playwright

Load Testing with Playwright using Step UI

This tutorial shows you how to set up a browser-based load test using existing Playwright tests in the Step UI.

Basic Keyword Development

This article explains Keywords in Step and demonstrates how to create simple ones.

Designing functional tests

This tutorial demonstrates the design, execution, and analysis of functional tests using the web interface of Step.

Robotic Process Automation (RPA) with Selenium

This tutorial will demonstrate how to use Step and Selenium to automate various browser tasks.

Robotic Process Automation (RPA) with Cypress

This tutorial demonstrates how to use Step and Cypress to automate various browser tasks.

Synthetic Monitoring with Selenium

This tutorial demonstrates how Selenium automation tests can be turned into full synthetic monitoring using Step.

Load Testing with Cypress

In this tutorial, you'll learn how to reuse existing Cypress tests to quickly set up and run a browser-based load test using the automation as code approach.

Load Testing with Serenity BDD and Cucumber

In this tutorial, you'll learn how to reuse existing tests written with Serenity BDD and Cucumber for load testing.

Synthetic Monitoring with Cypress

This tutorial demonstrates how Cypress automation tests can be turned into full synthetic monitoring using the automation as code approach.

Load Testing with Cypress using Step UI

In this tutorial, you'll learn how to reuse existing Cypress tests to quickly set up and run a browser-based load test using the Step UI.

Load Testing with Selenium

This tutorial demonstrates how to leverage existing Selenium tests to set up and execute browser-based load tests, following a full code-based approach.

Load Testing with Selenium using Step UI

This tutorial demonstrates how to set up a browser-based load test in the Step UI using existing Selenium tests.

Synthetic Monitoring with Playwright

This tutorial demonstrates how Playwright automation tests can be turned into full synthetic monitoring using Step.

Synthetic Monitoring with Cypress using Step UI

This tutorial demonstrates how Cypress automation tests can be turned into full synthetic monitoring using the Step UI.

Robotic Process Automation (RPA) with Playwright

This tutorial will demonstrate how to use Step and Playwright to automate various browser tasks.

Load Testing with Playwright for Java

In this tutorial, you'll learn how to reuse existing Playwright tests written in Java to quickly set up and run a browser-based load test using the automation as code approach.