Categories: RELEASE

Release notes

21.5 (minor)

Bug fixing

  • Tree view: Nodes are re-activated when modifying a parent
  • Performance of Monitoring view decreasing with high number of executions
  • Custom web context: missing few partial views

21.4 (minor)

Improvements

  • Support execution of single ADO Test Case as Step Test Case

Bug fixing

  • Unable to add output to the Return control
  • Cannot switch scheduled tasks on or off when the global switch is off
  • Agent custom attributes are ignored on the token group view
  • Notifications subscriptions are not filtered by project when triggered
  • Link to scheduler settings broken for non admin
  • The ADO plugin deletes attachments from ended executions
  • Tooltips are sometimes not showing and have two different designs
  • Table size preference not working properly
  • ExecutionParameters do not display description field as tooltip
  • Clean up step-api-function jar to remove sl4j dependencies

21.3 (minor)

Bug fixing

General UI/UX

  • Progress spinner is not displayed when adding / updating / removing keyword input
  • Analytics: incorrect total throughput when changing metric unit
  • UI Error message might displays http error instead of the actual error message
  • Big execution trees are not readable
  • Keyword routing is not displayed in execution view anymore
  • Filtering on some custom column (screen input) is broken
  • Export error not displayed in the UI
  • CallKeyword selection criteria contains only “name”
  • Start interactive execution of keyword fail with custom screen inputs
  • Incorrect project selection in edge cases

Plugins

  • Azure Devops: reporting results to azure devops fail with Java 17
  • Azure Devops: fails to delete temporary files
  • Dotnet agent: Binding issue with Dotnet 6 agent in docker
  • Dotnet agent: error when two keywords contains dependencies with the same file
  • Cypress: process might leak if errors occurs during startup
  • Cypress: add logging for resolved Cypress version
  • Managed process does not always cleanup the log folder

21.2 (minor)

Improvements

  • Implement plaintext support for PerformanceAssert

Bug fixing

General UI/UX

  • Minor UI layout issues
  • Missing UI customizations for parameter table (last modification date & user)
  • User preferences to set the default number of rows per table is ignored
  • Upgrading an older Step version does not create initial versions for plans and parameters
  • Navigating back and forward doesn’t work properly on all views
  • Unable to edit notification subscriptions for external repositories
  • Priority" filter from the Parameters page is not working properly
  • Maintenance messages from earlier Step versions not supported

Plan editor

  • Vertical scrolling of the plan tree of the visual plan editor is missing
  • Errors while setting keyword inputs for some edge cases (using output of executions, optional empty inputs, dynamic keywords…)
  • Tabs in the right sidebar can be unreachable

Execution view

  • Jumping from keyword node in execution to plan does not focus on the correct plan node
  • Execution tree not refreshed properly when an execution is running
  • Navigating back to Execution of TestSet displays an empty ‘keyword calls’ table

Devops (Step runner, artefact repository )

  • The update policy of the maven cache used by the artifact repository plugin is ignoring new snapshot versions of artifacts
  • Keyword timeout not taken into account for artifact repository
  • CallKeyword junit report throw NPE when the keyword is not found

Miscellaneous

  • Unable to add project member without specifying the role (i.e. to use the main user’s role)
  • Managing user (creating and editing name) is not disabled when userManagement is external
  • Missing audit logs

21.1 (minor)

Bug fixing

Plan editor

  • Various minor bugs around the keyword detail form (defining keyword’s inputs as json/dynamically, default values from schema,…)
  • First click on visual plan editor tree ignored
  • Line breaks in keyword descriptions not displayed properly on plan editor’s tooltips
  • Tree nodes get skipped unintentionally

Execution view

  • Auto-refresh cancel running requests
  • Execution view date filter uses incorrect timezone
  • Clicking on one testcase does not expand the Keyword calls panel
  • Execution tree looses its state when navigating between tabs
  • Opening the plan from an execution throw errors

Miscellaneous

  • Step Upgrade: Login not possible due to browser caching issue
  • Unable to assign role when users are managed externally, but roles are managed in Step
  • Unable to duplicate composite keyword
  • Parameter values with long text are not wrapped
  • Scheduler settings missing in open source

21.0 (major) - codename “Rigi” 13/02/2023

New features

UI/UX improvements

Continuation of the complete redesign of the UI/platform, bringing:

  • Modernization of the UI/UX for additional views (plan edition, execution results…)
  • Redesign of the call keyword form of the visual plan editor with enhanced support of Keyword schemas
  • Versioning of plans and parameters

Analytics

  • Enhancement of the execution performance view allowing complex filtering
  • Dedicated analytics view for complex analytics such as cross executions analysis

Access management

DevOps

  • CI/CD integration: Refactoring and improvements of the HTTP repository, renamed to Artifact repository
  • Development and local execution: Step runner improvements (Junit reporting and parametrization)

Keyword API

  • Step API enrichments, yet staying backward compatible, giving more configuration possibilities
  • New hooks available triggered before and after the keyword execution.

Plugins

  • Cypress plugin:
    • Support of Cypress versions up to 12
    • Improved dependency management
    • Support of TypeScript
  • New integration plugin for Astra enabling no-code automation
  • New integration plugin for SikuliX for GUI-based automation
  • Extension of the notification plugin to support custom payloads in webhooks

Miscellaneous

  • Improvement of the Check control allowing more complex expressions
  • New admin service to report the controller idle time is available
  • From the user list you may now display the user’s projects memberships

Bug fixing

  • Cloning plans imported from Test Management tools keep the source
  • Remove the entry “tenant” from the repository parameters when starting execution from the root/repository
  • Dynamic naming of controls wrongly toggled
  • Plans imported with the HTTP repository are not persisted
  • Keyword schema is not used in interactive mode
  • Dual plan editor - focus should be in the plan text editor rather than plan name
  • Modifying the number of items displayed per page is cancelled by the auto-refresh
  • Reimplement entity migration service using BulkOperationManager
  • FileWatchService fails to notice file changes for unknown reasons

Migration notes

For most users and setup upgrading from Step 20, no manual migrations tasks will be required. If you’re upgrading an older version, please consider following migration tasks

20.2 (minor)

Bug fixing

UI/UX

  • Disabling the scheduler globally does not reflect on the scheduler list view
  • Login view is not showing error messages
  • Unauthorized operations redirect to the login page
  • Filtering entities to move them to another project is not working
  • Dynamic naming of controls wrongly toggled for imported plans
  • Execution list empty after some time requiring a page refresh
  • Parameter table not reloaded after parameter edit
  • Navigating back to the plan list from the plan editor doesn’t work
  • Bulk operations apply only one column filter

Miscellaneous

  • Export execution steps to CSV does not work
  • Cypress project storage space should be optimized
  • Cypress error handling for project file structure needs improvements
  • Fix dotnet agent and libraries names
  • Web resources of custom made plugins are not available
  • FileWatchService fails to notice file changes for unknown reasons
  • Notifications of external repositories not properly triggered
  • The behaviour of the method used to read Keyword output changed

20.1 (minor)

Improvements

Analytics and visualization

  • Improved chart legends and contextual information
  • Add more attributes to the timeseries for future analytics view
  • Implement housekeeping for the timeseries
  • Add metric selection (TPH, TPM, TPS) for throughput chart
  • Indicate when queries are running in the background
  • Keep table pagination in auto-refresh

Bug fixing

Analytics and visualization

  • Performance Overview and Response Time charts use different rounding methods
  • Dragging of the ranger selection does not stop it exactly in the dropped position
  • Custom time range selection not working depending on browser local settings
  • ThreadGroups chart wrong for plans with multiple thread groups
  • ThreadGroups chart: Tooltip disappear between points

Miscellaneous

  • Terminated session is not redirecting to login page
  • Navigating back to a Plan doesn’t work
  • Dotnet agent fails with unclear error
  • Housekeeping not deleting grid monitoring measurements

20.0 (major) - codename “Pilatus” 14/10/2022

Step’s versioning is moved from 3.19 to 20 to correctly mark our quarterly updates as major. This process is ongoing and will be changed over all our libraries until the next major release.

New features

UI/UX improvements

Complete redesign of the UI/platform, bringing:

  • Modern layout
  • Set of new features (new tables, bulk operations, etc.)
  • Solid foundation for upcoming changes and improvements

Analytics and visualization

Complete analytics and visualization rewrite, bringing:

  • Modern interface with numerous new features for the reporting and analysis of Performance Tests and Synthetic Monitoring
  • In-house implementation of the analytics pipeline for a vastly superior performance for SaaS and On-Premise customers.

Integration

Plan and control flow improvements

  • After Sequence / After Thread controls are now also executed when interrupting an execution

Projects (multitenancy)

  • Keep page location when switching project
  • Specific access right for using the “all” project filter, it can now be assigned to any role

Plugins

  • [Documents comparison] Enable document comparison to use file on agents
  • [Azure Devops] support project names with spaces

Miscellaneous

  • Improve usability of import/export dialogs (progress bar, import all entities disabled by default)

Bug fixing

  • Execution Tree view doesn’t show on-going iteration
  • Execution Tree view pagination is broken
  • Legacy thread group execution chart maybe inconsistent, it should be replaced by the new visualization
  • Variable set in before thread are not accessible in the thread context
  • Step IDE cannot execute groovy keyword
  • Step controller not gracefully shutting down
  • [ALM] Multiple identical step import errors not displayed in UI
  • [REST API] FindManyByCriteria with no criteria throw exception
  • Duration of Cypress commands not measured properly in Cypress Keywords
  • Missing project attributes for plan inner functions cause error at runtime
  • Disabling authorization is broken
  • Incorrect OTP and authorization error handling with the LDAP authenticator

Migration notes

For most users and setup, no manual migrations tasks will be required. However, if you use complex groovy dynamic expressions, custom screen input with custom HTML or self developed Step plugins, please consider following migration tasks

3.19.0 (major) 02/05/2022

Note: this release is mainly a technical one with core frontend changes. This will allow us to do a full redesign of the UI/UX in the coming releases.

New features

UI/UX improvements

  • Technical refactoring of the Step web UI: the first milestone for an in-depth redesign of the UI/UX
  • Improved UI/UX of the ‘used by’ feature (pagination, contextual information)
  • Keyword call form: support default values in keyword schema
  • Replace copy/paste functions by duplicate
  • Execution view: add link from execution nodes to the corresponding plan’s node.

Projects (multitenancy)

  • Automatically switch projects when opening referenced entity

Cypress plugin

  • Support import of existing cypress projects/configurations
  • Support custom Cypress commands

Bug fixing

  • Stopping a test using thread group waits on the pacing
  • New DB index required for the execution list view performance
  • ConcurrentModificationException for the MeasurementControllerPlugin might appear in the logs
  • Open keyword editor from plan is broken
  • Reusing an existing resource as keyword package dependency is not possible
  • Exporting plan with slashes in name is not working
  • User with the role “tester” can rename plans and add/change nodes (but not delete)"

3.18.5 (minor)

Bug fixing

  • [Azure Devops] project names with spaces are not supported
  • [Azure Devops] Azure access rights errors are only displayed in the logs
  • Step client remote accessor might fail with some limit/skip combination
  • Project’s icons are not displayed on the Notifications subscription list view

3.18.4 (minor)

Bug fixing

  • Assignment of notification subscriptions to other project is not supported
  • Tenant is missing in webhook delivered data
  • Performance of import/export and find usage should be improved
  • Partial import from an export containing multiple type of entities is failing
  • Importing all entities by default is misleading
  • [Azure] Support shared steps

3.18.3 (minor)

Bug fixing

  • Multitenancy
    • Missing project entity icon in execution list
    • “Proceed filtered” wrongly displayed on user assignment
    • Migrating scheduler entities based on filter is not working
    • Missing scope attributes of parameters for entity migration
  • Deleting a tasks not yet executed cause a NullpointerException
  • Audit logs for session invalidation of unauthenticated user shall be removed
  • The controller hangs during a few seconds when aborting a test with large number of threads

3.18.2 (minor)

New Features

  • Multitenancy: support bulk assignment of entities by search criteria

Bug fixing

  • Switching project on repository plan should remain on the same page
  • Project assignment and find usage errors for Keywords using dynamic expressions in the Jar-Path

3.18.0 (major) 16/12/2021

New features

Core changes

  • Enforce Keyword package as leading Keyword management system
    • Customization per keyword is still allowed in the UI, but this will be overridden when updating the package
    • Single keyword (not part of a keyword package) are not impacted
  • Resolve keywords by name instead of id for all keywords references (i.e. when called from a plan)
    • greatly ease the import, export, migration from one environment to another…
    • the resolution is done on all keyword’s attributes such as the default name and project but also any additional custom attributes that you may have defined
    • Important: The keyword should be uniquely identifiable by its attributes (name, project and any additional custom attributes that you have configured)

Projects (multitenancy)

  • Improve and enforce the use of projects
  • Detect all unassigned entities and provide a wizard to properly assign them to a new or existing project
  • New feature to move all entities per type (plan, keyword,…) from one project to another in one go
  • New feature to move entities from one project to another recursively
  • Deprecate multitenancy deactivation

UI/UX improvements

  • Add a find usages feature for keyword, resources and plans
  • Improve data table pagination
  • Several other UI improvements (contextual information in dialogs, toolbars redesign…)

Analytics

  • Replace the legacy dashboard view with our grafana dashboard integration
  • Design a new grafana dashboard dedicated to synthetic monitoring (based on scheduled execution)

Integration

  • Add token based authentication for the REST calls
  • Document [REST services]({{ relref “rest-api” }}) with Swagger
  • First (beta) version of the LogicFLow integration plugin

Azure DevOps

  • Speed up retrieval of large test plans
  • Cache and restore last used project, provide per-project bookmarks

Miscellaneous

  • Modify user creation to define an initial and one time random password
  • Support password strength requirements
  • Implement Audit logs for Step rest services
  • Deprecate the legacy report view for admins
  • Exclude the log4j library (only used by the image OCR feature in the ImageCompare plugin) and replace it by the slf4j bridge

Bug fixing

  • [Execution] Synchronized artefact not executing skeleton phase
  • [Execution] Skeleton creation may fail when dynamic expression cannot be evaluated
  • [Execution] Some complex error message are not persisted due to serialization errors
  • [Execution] NullPointerException thrown in case of syntax errors in dynamic expressions
  • [Step client] The project is not set when creating a plan with the Step client
  • [Azure] Reporting uses wrong ID format, resulting in buggy reports in Web UI
  • [Dotnet] improve agent url discovery
  • [Dotnet]: Reusing the tmp folders is not robust
  • [Dotnet] The file App.config is copied to the distribution of the .NET 5 agent
  • [Dotnet 5] Missing runtimeconfig properties
  • [Admin] Admin users are allowed to delete themselves

3.17.2 (minor)

Improvements

Azure DevOps

  • Button to refresh plans of current project
  • Improve error handling while browsing projects
  • Add information about which user ran tests
  • Support execution reports in plain text format
  • Allow to preserve formatting in Action and Expected Results
  • Support custom product name in DevOps reports
  • Display Failure Reason in Execution Comment
  • Support disabling steps by commenting them out
  • Allow to disable DevOps reporting
  • Allow to display DevOps IDs in tree

Bug fixing

  • Execution context unexpectedly closed
  • Azure DevOps reporting uses wrong ID format, resulting in buggy reports in Web UI

3.17.1 (minor)

Improvements

Azure DevOps

  • Report test runs with errors/failures as “Investigation needed” instead of “Completed”
  • Prefix attachment filenames with step number
  • Unique report filenames
  • Shorten Step description in report.html

Miscellaneous

  • SoapUI: Manage environment variable
  • Add execution parameters to webhook notifications

Bug fixing

  • Unable to execute the .NET 5 agent from the mac finder
  • Filter HTML entities on import in Azure
  • Minor issues in the new .NET 5.0 agent
  • Performance beta view - restarting execution show data of previous execution
  • Performance beta view - execution panel not updated
  • Prometheus charts displaying wrong y-axis label
  • Prometheus transactions overview graph wrong min max
  • Controller shutdown hook is broken
  • Clean up of Groovy Errors in the controller logs
  • PrometheusHandler causing a memory leak
  • The attachment given by ResourceManager.download in the Step client has an invalid name

3.17.0 (major) 05/07/2021

New features

Analytics

  • Implemented brand new performance dashboard for the real-time visualisation and analysis of performance tests
  • Updated and enhanced Grafana dashboards. See our related tutorial

Step SaaS

Step features enabling the upcoming new SaaS solution:

  • On-demand scaling of Step agents
  • Graceful shutdown of agents
  • Full Kubernetes integration (incl. Grafana and Prometheus)

While waiting for the self-service portal, you can still order your cluster from our current cloud offering form

Development distribution

  • Introduced new standalone distribution of Step called IDE distribution (beta) to develop and test Plans and Keywords locally before pushing them to the Step server (Documentation upcoming)

.Net plugin

  • The dotnet agent and keywords support now the .NET 5.0 framework giving the possibility to run them on Linux and macOS in addition to the Windows OS. The legacy agents for .Net 4.x is still supported

Cypress plugin

  • First (beta) version of the Cypress.io plugin is available. You can now declare Step Keywords that run cypress commands and thus leverage the combined power of Step and Cypress as described here

Integration

  • Webhook support added to complement the email notifications
  • step Client API: new function to retrieve executions by interval and status
  • Azure DevOps Plugin: Test Suite/Plan execution are now opened in new tabs

UI/UX improvements

  • Further customization options added to change the navigation style as step CSS variable
  • Browser cache invalidation added to help with upgrade of Step versions

Plan and control flow improvements

  • Plaintext plans can have different type of root nodes and are not limited to testcases anymore

Miscellaneous

  • Multiple performance optimizations implemented to increase the maximum throughput of your executions
  • The routing criteria of keywords can be configured at the keyword package level

Bug fixing

  • Infinite loop when starting a plan with an infinite while
  • Wrong LDAP filter syntax and placeholder described in step.properties
  • Grid view unresponsive for very large number of tokens
  • The retryIfFail control’s tooltip is misleading
  • PDF and Image Masks can be modified without specific rights
  • Improve Syntax Error handling for repository plans (ALM,devops…)

3.16.3 (minor)

Bug fixing

  • Shutdown hook of the controller is unable to fully stop the process
  • Running load test with many concurrent testcases slow down the execution view
    • To execute load test plans using testcases inside a threadgroup, it is recommended to upgrade to the 3.17 release
  • Running load tests with the prometheus plugin active and large volume of transactions may cause stability issue of the controller

3.16.2 (minor)

Improvements

  • Azure DevOps Plugin: Open Test Suite/Plan execution in new tab
  • Allow Plaintext Plans to have root nodes other than TestCase

Bug fixing

  • Protected parameters cannot be accessed within plans for data source passwords
  • Monitoring dashboards export and config buttons are not clickable
  • Editing a disabled scheduler task make it to be executed

3.16.1 (minor)

Improvements

  • Grafana dashboards for step executions slightly reworked
  • Sampling interval for new gauge metrics can be configured as step property

Bug fixing

  • Document Compare Mask list is not displayed properly
  • Test cases added to an already scheduled test set are skipped by the scheduler
  • Table auto refresh is interrupted after (network) errors
  • Plan and Function locator invoked from the plan editor are not project aware
  • Connection aborted exceptions are thrown in controller logs

3.16.0 (major) 01/04/2021

New features

Analytics

  • New plugins available for advanced analytics and integration with external solutions
  • Prometheus plugin to expose execution metrics to your Prometheus server
  • Log measurement plugin to expose execution metrics to your log ingestion pipelines (Logstash, Splunk,…)
  • Jdbc measurement plugin to write the execution metrics to any compatible database
  • Advanced dashboarding: whether you use the new plugin or still rely on the RTM mongo database, we prepared a set of dashboard and tutorial to integrate with grafana
  • New metrics available (for all plugins)
  • Threads usage per test execution and thread group
  • Step’s grid usage
  • New feature to export RTM raw measurements from the interactive analysis view

UI/UX improvements

  • Support case insensitive filters in data table
  • Support operations on multiple nodes for the Visual plan editor (Copy-paste, delete…)

Plan and control flow improvements

  • Control of the execution flow on error has been improved. One can now control what to do in case of error for each and every node.
  • Enhanced ThreadGroup for better control on execution duration, reaching throughput goals and designing ramp-up

Protection of parameters

Integration

  • Azure DevOps Plugin: Implement test suite view as tree instead of list
  • Adding repository information to notification subscription list to distinguish between local and external plans (Alm,…)

Operations

  • Migrate to YAML format for agent configuration (AgentConf.yaml)
  • step’s controller and agent now require Java version 11.
  • Support of internet explorer has been dropped

Bug fixing

  • Plan editor: unexpected collapse tree
  • Report node details not visible in execution tree
  • Button “Select test case…” on Execution Steps table rows doesn’t work anymore
  • Performance issue on Parameters view
  • Performance dashboard width is overflowing
  • Table Export to CSV in Interactive Analysis doesn’t export all data it gets from the RTM
  • Azure DevOps Plugin: support pagination in calls to the Azure DevOps APIs.
  • PDF and Image Compare keywords are not properly initialized when Multitenancy is disabled
  • Improve error messages for document compare misusage

3.15.7 (minor)

Bug fixing

  • Parameter list is not displayed if it contains parameters created before version 3.6
  • Document Compare Mask list is not displayed properly

3.15.6 (minor)

Bug fixing

  • Displaying the parameter table is slow for large list
  • Table auto refresh is interrupted in case of errors
  • Test cases added to an already scheduled test set are skipped by the scheduler

3.15.5 (minor)

Bug fixing

  • Scheduler setting “Execution user” ignored by the scheduler
  • Table auto refresh broken when minimizing the browser

3.15.4 (minor)

New features

  • Support default values for check boxes in screen templates
  • Use “functionTable” screen definition instead of “compareMasksTable” for “Document Compare Masks”
  • Add repository information to notification subscription list

Bug fixing

  • Improve error messages for document compare misusages
  • Performance dashboard width is overflowing
  • Keywords annotated with @Attribute not found in JUnit Runner
  • Keyword input keys containing spaces not supported by the plain text plan syntax
  • Assertions with output attributes containing spaces not supported in plain text plan syntax
  • Controller unable to start when multitenancy is disabled
  • PDF and Image Compare KW not properly initialized when Multitenancy is disabled
  • Report node details not visible in execution tree
  • Automatic migration of notification subscriptions from 3.14 to 3.15 randomly fails
  • Toggle “Enable scheduler” toggled when setting “Execution user” on scheduler setting screen
  • Loading spin for data table not displayed

3.15.3 (minor)

Please do not use this version, it was mistakenly released from a wrong branch. It is superseded by the corrected version 3.15.4.

3.15.2 (minor)

Bug fixing

  • Automatic migration from STEP versions < 3.13 to 3.15 is broken

3.15.1 (minor)

New features

  • Azure: Add execution link to Azure DevOps report
  • SoapUI: Added beta support for SoapUI PRO and improved reporting (missed in 3.15.0)

Bug fixing

Import/export

  • Plan export failing if a Keyword referenced by attributes cannot be resolved
  • Import of single entities type (i.e. plan) from a large export fails
  • Import of parameters is failing
  • Import/export from windows to linux is failing for exports containing resources

UI

  • When editing a “callKeyword” control the focus on inputs was incorrect
  • Map entry order got lost after using “Map output to” in keyword calls

3.15.0 (major) 17/12/2020

New features

Integration

Plan edition and execution

Document comparison

ALM plugin

  • Added new switch to toggle off ALM reporting
  • Implemented asynchronous ALM reporting

SoapUI plugin

  • Added beta support for SoapUI PRO
  • Improved SoapUI plugin reporting

Miscellaneous

  • Enhanced JUnit runner

Bug fixing

  • Cannot enter a running execution started via Step Client (REST)
  • Click on testcase’s name to filter the keywords calls on execution view is broken
  • Monitoring dashboard - incorrect names in configuration mode

3.14.2 (minor)

New features

  • Improved responsiveness and accuracy of performance Dashboards
  • Processing of performance measurements to generate graphics and statistic tables improved drastically
  • Percentile approximation is now reduced to the millisecond in most cases (the actual precision reached is now displayed next to your statistics)
  • Import/export improvements:
  • Adding export of single keyword
  • Export recursively keyword and plan referenced by attributes (CallPlan and CallKeyword controls)

Bug fixing

  • Unable to edit project memberships after user deletion
  • Project admin page does not reload when changing project
  • Recursive export of resource from dataset is not working
  • Default Test case not selected when executing a Plain text from the execution view
  • Opened execution not being refreshed after test completion
  • Missing screen inputs related rights in the default role handler used by the step OS version

3.14.1 (minor)

New features

Bug fixing

  • Import of older plans with Assert to 3.14.0 is failing
  • Import keyword package failing for controller installation with spaces in path
  • Scheduler tasks are not executed within the right project after migration to 3.14.0
  • Unnecessary/duplicated keywords delivered in step EE out of the box

3.14.0 (major) 08/10/2020

New features

Projects (multitenancy)

  • Created new project admin rights enabling administration of single projects (without being a ‘global’ administrator)
  • Introduced project settings view for project’s administrator

Dashboards and Performance measurements

  • Plan and Task IDs added to RTM measurements enabling dashboarding & analysis across multiple executions (can be accessed through the monitoring view for example)
  • Keyword measurements moved closer to the keyword invocation (do not include the delegation time to the agent anymore)
  • Display Keyword measurements on execution view (see execution step details)
  • Self-Monitoring dashboard added to step (all metrics can already be colleted with step 3.14.0, dashboard and documentation are coming soon)

Import/Export

  • Refactoring and extension of Import/export features (more details)
  • Introduced recursive export and import supporting multiple type of entities (plans, keywords, resources…)
  • Reimporting in same controller can either overwrite or duplicate exported entities

Integration

  • Introduced JUnit Runner as part of the Step Testing SDK to seamlessly integrate Step Plans in plain text format into JUnit tests
  • Introduced Executable bundles (Plans, Keywords, and required dependencies) for self-contained executions on the STEP platform and seamless integration with CI/CD servers
  • Implemented Artifact repository connector for retrieval and execution of Execution bundles located on artifact repositories (Nexus, etc)
  • Added a Client API for managing parameters (see step client API’s service description)

Plan editor

  • Added ability to undo/redo or discard all changes in plan editors
  • Support dynamic names for any nodes (including test cases) used by the tree view and report nodes
  • Improved Visual Plan Editor layout

Execution view

  • Add Check to the execution tree & ALM reports
  • Report nodes of keyword called by ‘CallKeyword’ are now named after the keyword name
  • Optimization on performance dashboard

Controls

Plugins

  • Dotnet: Improved isolation of keyword execution allowing to use different versions of .NET libraries
  • Dotnet: Improved error handling and error messages for the .NET agent
  • SoapUi Keyword: add possibility to execute all test suites and test cases of a given project from one single keyword

Miscellaneous

  • Notification: Change recipient field to a dynamic value
  • Improving handling of keyword package import errors like “Invalid schemas”
  • Improve error handling of non-existing or restricted scoped variables accesses in groovy expressions

Bug fixing

  • Threadgroup - change rampup to start the 1st thread immediately
  • Button “Show this node on tree view” in keyword list doesn’t work in big trees
  • Improve interruption of test executions
  • Table filter “Package” of the Keyword view not working
  • Agent cannot be reached when using agentUrl
  • Step Agent: Issue with locked .dll
  • Plan attachments not deleted by housekeeping task
  • Creating kw library with step client imports the Echo kw
  • Execution list not showing up when root report nodes are missing
  • Tokens already reserved error in Interactive mode
  • Test cases defined under foreach create inaccurate reports
  • Columns missing in data table in case of slow network
  • Dashboard content displayed in the logs when saving
  • Editing project name remove all its members
  • Performance regression in Keyword call introduced in previous release
  • Threadgroup with extremely large number of iterations cause out of memory
  • Some tooltip are partially hidden
  • Cannot set back the Testcase field in a SoapUi keyword to null
  • Iterating over a CSV DataSet generates unnecessary tmp file
  • Each For or ForEach control creates a useless dataset-write-thread on the controller
  • FunctionPackage discovery not working when the java classpath of the controller process is relative
  • Fairness not guaranteed in Synchronized controls
  • Error in the logs when executing the Housekeeping job with no results
  • Filter out new plan templates
  • Oryon Keyword template adapted to latest API

Migration

Migration guidelines for this release are available here.

Main point:: Nashorn engine used to evaluate Javascript expressions and execute javascript (but not nodejs) keywords is deprecated starting with step 3.14. Support of nashorn will be removed in a future release, in 3.14:

  • The engine used to evaluate the “Activation script” of parameters and screen templates has changed to groovy
    • For now you may still revert to nashorn if required by changing the following property in ‘step.properties’ and restarting the controller: tec.activator.scriptEngine=nashorn
  • Likewise keywords of type script and using javascript as language are deprecated

3.13.5 (minor)

New features

  • Extend ‘RemoteExecutionManager’ and improve documentation

Bug fixing

  • Number of tasks displayed on scheduler list limited to 50
  • Empty execution description when starting plan using the step client
  • Plan of composite Keywords are not hidden
  • Simple expressions in Set controls not working in plain-text syntax

3.13.4 (minor)

Skipped (build deployment error)

3.13.3 (minor)

Bug fixing

  • Import of plans with special characters fails due to encoding issues
  • Issue when selecting existing jar at keyword creation
  • Plan attachments not deleted by housekeeping task
  • Plan is not directly saved after performing a Copy/Paste in plan tree

3.13.2 (minor)

New features

  • Adding “Notification_Send” local keyword to send custom notifications (i.e. emails…)
  • Prevent multi-threaded miss-usage of function group (sessions)
  • Improve error handling of non-existing or restricted scoped variables accesses in groovy expressions

Bug fixing

  • Sporadic IOExceptions observed in controller logs for server side data tables
  • Selecting a new Plan in a scheduler task doesn’t refresh the tasks table
  • NPE errors raised from the current operations view
  • RetryIfFail with report last node only option: last try is not reported if the timeout is reached
  • Javascript incompatibility with IE11 issue was introduced with step 3.13

3.13.1 (minor)

Bug fixing

  • Table filters might stop working under rare conditions
  • Execution view is not refreshed in specific context
  • Duplicated auto refresh of the execution list view causing unnecessary load
  • Editing a scheduler task from the monitoring view is not working
  • Opening the Keyword editor from the plan tree is broken
  • Migration breaks for plans with unexpected ID type
  • Import of plans from previous version is not handled
  • Sleep control is executed in Simulation mode which should not be the case

3.13.0 (major) 07/04/2020

New features

Projects (multitenancy)

  • Implement project-based role assigment
  • Implement global projects for common entities (Common Plans, commons Keywords, common Parameters, common Execution Parameters)
  • Implement project support for STEP Client (REST) API
  • Enable definition of project specific screen templates
  • Enable definition of project specific monitoring dashboards

Dashboards

Plan editor

  • Manage plain-text plans exactly like visual editor plans
  • Improve plan editor tree usability
  • Implement a function in the visual plan editor to disable nodes
  • Provide proper filename upon plan export
  • Support keyword with spaces in plain text syntax

Keywords

  • Support package managers (nexus, etc) as repository for Keyword Packages
  • Add field description/documentation to Keyword
  • Package embedded keywords as Keyword package

Execution view

  • Scrolling position of “Execution tree” reset after refresh
  • Redesign the auto refresh
  • Monitoring of current operations in test cases
  • Improve WaitForEvent reports
  • Optimize performance of the execution view
  • Show waiting controls in executions steps

Controls

  • Release Agent Tokens in waiting controls
  • For and ForEach: add the maxFailedLoops parameter to the gui
  • Implement named and global locks via Synchronized control
  • RetryIfFails control: Add switch to persist only last iteration
  • Add checkbox to choose to display failures to the RetryIfFails control
  • Implement data set reset for source type “Directory”
  • Add UserID handle to ForEach (analog to ThreadGroup)

Miscellaneous

  • Add “New” button in Resource view and Scheduler View
  • Disable closing of dialogs when clicked outside
  • Add link to open entity editor from table rows
  • Add tooltips to execution parameters
  • Assign dedicated URLs to “Admin” views
  • Implement move up & down to define order of Screen Template inputs
  • Make the title (STEP) like the favicon configurable
  • Custom HTML Templates for Keywords
  • Placeholder lookup in Keyword properties should also take Keyword Inputs into account
  • Call Timeout and Keyword not passed as property to the Keyword
  • Improve the usability of the Parameter list
  • Add field “Parameters” to the “Edit scheduler task” dialog and remove editable execution parameters from scheduler table
  • Improve configurability of ScreenTemplates
  • Replace legacy datatable directive by st-table
  • Standardize CRUD REST services and migrate Remote Accessors to AbstractCRUDRemoteAccessor
  • Report a business error when a PDF Mask is missing
  • Make scheduler task table configurable
  • Add system chooser to resource loader

Bug fixing

  • Parsing error in plain-text plans when separating key-values by carriage return
  • executionParameters with an activation expression are not properly evaluated
  • Concurrency issue with the Echo function
  • In .Net, the exception captured when a timeout occurs capture the wrong AppDomain stack
  • Common Keywords not listed under Keywords when a project is selected
  • Double clicking the refresh button of a FunctionPackage creates a duplicate entry of the FunctionPackage
  • Keywords not isolated properly between projects
  • Display errors in the “Current operation” table of the execution view
  • Skip/limit fields on Execution Tree view not working as expected
  • ForEach: infinit loop for empty json map
  • The notation “Application.Keyword” isn’t supported by the Parameter scope definition for the scope Keyword
  • CosmeticsHandler redirects to non secure connection
  • Plans cannot access parameters from which the scope has been changed from application to global
  • Parameters are displayed on 2 rows
  • Fix NPE in ReportNodeStatisticsView and RtmPlugin when a KW cannot be found
  • Items potentially processed several times in parallel controls (For, ForEach, TestSet, ThreadGroup)
  • Encoding issue in the execution errors view
  • Execution parameters sometime hidden
  • QuotaManager not blocking on acquire permit and not propagating errors
  • Escaping of $ in Set controls (Plain-Text) doesn’t work properly
  • Fix various regressions in dashboard presets
  • ConcurrentModificationException in service threadmanager/operations

Migration

Migration guidelines for this release are available here.

3.12.3 (minor)

Bug fixing

  • Items potentially processed several times in parallel controls (For, ForEach, TestSet, ThreadGroup)
  • Global Parameters cannot be used in Keyword Jar-Path placeholders
  • The notation “Application.Keyword” isn’t supported by the Parameter scope definition for the scope Keyword
  • Plans cannot access parameters from which the scope has been changed from “application” to “global”
  • Rename attribute keyword.attributes.app to keyword.attributes.application

New features

  • Implement data set reset for source type “Directory”
  • Placeholder lookup in Keyword properties should also take Keyword Inputs into account
  • Add field “Parameters” to the “Edit scheduler task” dialog and remove editable execution parameters from scheduler table

3.12.2 (minor)

Bug fixing

  • Controller not starting under certain circumstances due to conflicting libraries of the mongo driver

3.12.1 (minor)

Bug fixing

  • Column Package disappears from Keyword table after screen template update
  • Clicking the groovy button while adding a new KW input inconsistency
  • Multitenancy plugin switch off cause errors
  • Filter by IDs on entity does not work
  • Projects dropdown not automatically reloaded
  • Screen template admin view broken when using projects
  • Scheduler entries plan name disappear when migrating to 3.12.0
  • Parameters created using a previous version of STEP disappear from the parameter list after switching the Project
  • Plans aren’t imported to the current project
  • The feature “Export all plans” exports the Plans of all projects
  • When starting an execution in a specific project the parameters of other projects are also applied
  • Toolbar layout of the dashboard view not correct in IE
  • Notification gateways are not visible when selecting a specific (non global) project
  • html elements not aligned properly
  • All keywords referenced in a keyword package take its name when the package is edited
  • Composite keywords not working anymore on agent
  • Custom attributes of a new FunctionPackage are ignored when referencing an existing package
  • Classloaders are loaded twice on the agent side
  • JNI resource management issue as part of a java keyword
  • Remove unnecessary column in Keyword library view (Plan Editor)
  • Keyword name not displayed on “Keyword calls” table when running Plain-text plan (ALM, organizer, etc)

New features

  • Add config variable switch for root true vs false in ALM parsed plans
  • Updating to viz alpha-37+

3.12.0 (major) 05/12/2019

Community Edition

Core functionality

  • Introducing a new visualization framework for consolidating and displaying data into Dashboards, allowing for historical analysis, aggregation and real-time monitoring of data and events originating from step itself as well as from foreign data sources. The visualization framework is fully configurative, allowing users to design their own visualization dashlets and provides modular components which will be reused throughout exense’s products.
  • Multiple pre-populated Dashboard presets are provided
  • Introducing Dashboard session entities for built-in persistence and management of dashboard instances
  • Keyword Session artefacts used in Plans now implicitly support multiple token types (for instance, a session can now concurrently handle Java and .NET keyword executions)
  • Complete overhaul of the Performance tab in the execution view for monitoring keywords & transactions (both in a real-time way and summary-oriented way), users can now easily navigate from the default Keyword-based view to a Transaction view (to view their Custom “Measures”)
  • Parameters now support a “scope” which restricts access from keywords depending on the keyword’s context (Global, App-based, Keyword name based).
  • A global maximum number of threads can now be assigned to an execution via the variable execution_threads_auto, serving as a safety cap or garantee of “fair” resource allocation to concurrent executions
  • Fixed issues with search headers in various data tables (among which Keyword Package, Scheduler Task, Resources)

Operations

  • step’s controller and agent can now be run on Oracle’s JRE 11. Step itself is still compiled using JDK 8, thus still allowing the controller and agent to run on JRE 8.

Keyword API

Ergonomy

  • Fixed an issue where modal dialogs were shut prematurely on mouse release

Enterprise Edition

Operations

  • While Step’s controller and agent can run on JRE 11, the ALM plugin (more specifically, the ALM daemon) requires a 32 bit JVM, which is not available in Oracle’s Java 11 distribution.

Project Entity (multitenancy)

  • Introducing the Project entity for implicit isolation of almost all of step’s entities in the table views. New projects can now be created and entities can be assigned to a project. Lists are automatically listed based on the selector chosen in the top right corner of the screen. Plans, Keywords, Executions, Parameters, Scheduler Tasks, Resources, FunctionPackages are governed via projects. The following entities and views are not covered as of 3.12.0: Monitoring, Dashboards, Reports, Event Broker Monitor.
  • Project creation and migration workflows are made available in a new view under Admin -> Settings -> Projects
  • 2 default projects are pre-created: system (used for instance for housekeeping) and alm (for storing parsed plans and executions from alm)
  • Project management services have been secured and additional default rights created in the access matrix file

Dashboards

  • Specific additional Dashboard presets are made available by default to Enterprise customers: RealtimeSelfMonitoring (containing a Free token per agent dashlet and a Current on-going execution dashlet), UserExecutionDashboard (offering a breakdown of user activity over the last week) and ProjectOverview (providing an overview of the related plan and keyword executions over the last week, after applying the corresponding project entity)

Ergonomy

  • A viz-presets file has been added to the list of customizable cosmetics (conf/cosmetics) for easily modifying and extending dashboard presets
  • Various bugfixes relating to modal popup behavior and text overflows have been performed

Plugins

  • ALM: necessary updates have been made to the daemon and the parser in order to ensure compatibility with project management and keyword and keyword package multi-versioning (this includes path-based resources and multi-version scenarios)
  • EventBroker: services have been secured and additional default rights created in the access matrix file
  • .NET: new annotation-based functionality is supported in .NET (similar to Java)
  • FunctionPackage: redesigned FunctionDiscovery as a managed process, thus unlocking the jar file in path scenarios, preventing concurrency issues and not requiring to open extra network ports anymore (in .NET). Keyword packages now support path-based resources and multi-versioning in Java and .NET. A new field “Reference Package” field has been added in order to preserve (i.e re-apply) any attributes which may already have been added in a previous version of the package or in a different package.

Deprecation & backward compatibility

  • Backward compatibility to step-api 1.0.0 is preserved, however, in order to benefit from new API features (such as property validation), an upgrade to 1.1.0 is needed

Migration

Migration guidelines for this release are available here.

3.11.1 (minor)

Features

  • step Client API

Added new workflow based on the identification of keyword packages via resource name for simplified management.

In this scenario, the developer doesn’t have to worry about storing the FunctionPackage object or its id. The workflow is simple: if the resource changes, the keyword package is refreshed. This means that the old Keywords and Resources are updated (and if necessary, cleaned up). As a consequence, in this scenario, it is assumed that different resources have distinct names and that only one KeywordPackage instance is to be managed per Resource name. FunctionPackage objects can now also be looked up remotely by resource name.

See for example method resourceBasedKeywordPackageCreationAndUpdate in our demo class StepClientDemo

  • KeywordPackage

Upon editing a keyword package and changing its resource(s), old resources are now cleaned up automatically.

Bug fixing

  • .NET Agent

Fixed multiple issues related to error handling and the absence of attachments in cases, such as returned exceptions or in the event of a Timeout. The SetError which had disappeard during Api refactoring is now present again.

  • Performance

Fixed a significant performance issue caused by redundant calls made in the Resource Manager, between the controller and agent. Although this had virtual no functional impact as well as no perceivable latency impact on instances with sufficiant resources, the theoretical maximal throughput of keyword executions was affected (since 3.10).

  • ResourceAccessor

Fixed mismatch in service URL causing for the delete function to fail with an HTTP error.

  • Logging

Fixed minor issue causing warnings to be logged by the web container in Enterprise versions.

  • Ergonomy

Fixed display glitch causing for text to overflow in the Execution Tree view. Fixed an issue causing bootstrap dialogs to close unexpectedly. Fixed a glitch specific to Internet Explorer, causing for custom screen attributes not to be displayed.

3.11.0 (major) 13/09/2019

Community Edition

Core functionality

Keyword API

  • Full isolation of Keyword API dependencies to allow desynchronizing keyword dependency upgrades from controller upgrades (users choose when they upgrade their script’s dependencies)
  • Supporting bundling of properties and schemas in keywords

Ergonomy

  • Plan Editor: Resizable panes, modal for value editing, keyboard shortcuts, improved drag&drop support (multiple artefact move)
  • General: new Community style

Enterprise Edition

Operations

  • Automatic Housekeeping via built-in LocalFunctions and admin plan, switch in settings, and “retain execution” policy
  • Batch deployment workflow for Java and .NET via Keyword Package : keywords are now automatically created based on Jar and DLL inspection

Ergonomy

StepClient API

  • Improved performance in local execution
  • Fixed issues in which event payloads were serialized incorrectly or causing errors

Plugins

  • ALM: enabling copy of ALM plans as regular editable plans, worksheet argument in ForEach
  • EventBroker: allowing choice between unicity or duplicates in Group+Name scenario
  • Plan parser / Dual plan editor: support for escaped characters
  • RTM: allowing transaction logging for log-based integration (Splunk, ELK, etc)
  • .NET:
  • dynamic dependencies deployment is now supported just as in Java.
  • Log4Net support: the .NET agent will now log critical errors and keywords can log information into the agent’s log file. See example here.
  • PDB support (simply packed along with dependencies zip)
  • support of keyword call timeout & thread interruption mechanism, similar to Java
  • using multiple assemblies in the context of the local KeywordRunner is now supported

Deprecation / Deletions

  • Removing Selenium as a built-in Keyword type: native Keywords (Java, .NET, Node.JS) + libraries are now the standard approach to deploying Selenium Keywords, this has the major advantage of allowing users to configure their own custom version of Selenium easily and without having to worry about mismatches between their project dependencies and the platform’s dependencies

Migration

Migration guidelines for this release are available here.

3.10.2 (minor)

Features

  • step Client API: First implementation of the RemoteExecutionAccessor client. This includes the functions getAll(), findLastStarted() and findLastEnded()
  • Plain text syntax: Adding the “Delimiter” parameter in the step parser for the ForEach control
  • PDF-Plugin: Improving error handling in keyword PDF_compare
  • Web interface: Adding the field “Parameters” to the “Edit task” dialog

Bug fixing

  • ALM-Plugin: Fixing reporting of empty step names

3.10.1 (minor)

Features

  • Monitoring plugin: Implementing new monitoring plugin
  • ALM Plugin: simplifying installation
  • Plain text syntax: Adding worksheet parameter to ForEach in plaint text syntax
  • Other: Implementing logging of std-out and err-out for ManagedProcess

Bug fixing

  • Web interface: Fixing drag&drop issue when uploading resources in Firefox and IE
  • Web interface: Fixing edition of preferences under MyAccount for non-admin users
  • ALM Plugin: fixing reporting issue of failed-reason and tester fields
  • Plain text syntax: Fixing escaping

3.10.0 (major) 13/03/2019

Community Edition

Core functionality

  • Introduced the ResourceManager to allow the upload, management and tracking of central resources on the controller. The ResourceManager is also used to store attachments and thus replaces the old attachmentdir (See http://docs.exense.ch/wiki/step/view/Versions/3.10.x/Migration/ for migration details)
  • Upgraded to RTM 2.2.4
  • Implemented support for write operations in CSV DataSets
  • Added MaxDuration field to ThreadGroup
  • Added support for upload and zipped projects in Node.js plugin
  • Added support for Puppeteer

Enterprise Edition

API (Remote-Client)

  • Implemented JUnit report writer for local and remote executions
  • Renamed ControllerClient to StepClient and refactored the API

ALM-Plugin

  • Implemented reporting of testcase start
  • Implemented asynchronous testcase export
  • Fixed username (tester) reporting issue

See Also