Categories:
RELEASE
Roadmap
2021
Cloud
- New cloud offering of step as Kubernetes app on Google Cloud Market place
- Auto-scaling of step Agents in Kubernetes
- Public Helm chart for step OS
Dashboarding & analytics
- Deep rework of existing dashboards (improved configurability, user-friendliness and look&feel)
- Extension of reporting functionalities (e.g. generation of reports, better exporting)
- Integration with Grafana and Grafana Dashboards on central repository
- Integration with Prometheus and TimescaleDB
- Integration with JDBC and log ingestion pipelines (logstash, splunk, etc)
UI
- Further improvements of the visual plan editor
- Plan refactoring features (find Keyword usage, batch operations, etc)
- Bidirectionality between plain text plan and visual plan editors
- Improvement of tables (filtering, multi-selection, etc)
- Design and prototyping of STEP 4.0 with complete redesign of the UI
Core
- Redesign of the controller for enhanced scalability
- Support and documentation of REST APIs
- Protection (encryption) of sensitive business parameters (SUT passwords, etc)
Documentation
- Tutorials:
- How to setup large scale on-demand test automation project with step
- How to perform end-to-end load & performance testing with step and Selenium
- How to perform continuous performance testing with step
- How to perform synthetic availability & performance monitoring with step
2020
Initial Roadmap
step Core
- Deep improving ergonomy and out-of-the-box end-user, dev & admin support
- Redesign of the execution reporting views
- Change tracking of entities in STEP UI
- Additional analytics capability and new dashboards
- REST API enhancement
CI/CD
- Managed executions and stateless controller
- Deep exports/imports of entity bundles
- Testing SDK
Performance testing
- Performance testing specific assertions
- Continuous performance testing support
- New performance testing controls
- Extended performance test dashboards
- API for automatic performance test reporting
Integration plugins
- Azure devops integration
- Extension of the Jira Zephyr support
step Libraries & Samples
- Extension of the step Library
2019
Evaluation
Full evaluation of the 2019 will take place in Q1 2020
Initial Roadmap
step Core
- Project overview dashboard + labeling and/or coloring of executions in executions list
- Cloud-ready interface (resource upload via drag & drop and resource manager)
- Artefacts publication in Maven, NuGet, npm
- Batch keyword import via Annotation detection (@Keyword)
- Improved bundling (export & import of all step objects, including keywords)
- Keyword API decoupling / externalization
- New documentation in the form of a modern knowledge base + case studies, white papers, etc
- New error visualization tab in execution view
- Applicative Logout feature
- Google Sheets as dataset
Ops & Administration
- Business-aware housekeeping (deletion of specific executions & related objects, drop all except..)
- Admin view improvements
- Self-monitoring (first via add-on tutorial, then built-in directly)
- Moving config files to db
- Placeholder support in remaining config files
Monitoring
- Scheduler-based monitoring view: provides an overview of the execution status of each job
- User-customizable dashboarding
- Notifications plugin
- Cron presets for common use cases
Performance testing
- Performance report & dashboard with throughput, error rate, avg response times table, response time overview & error breadown
- Better RTM integration & dashboarding capability
Plugins
- Major refactoring of .NET agent with library transfer support, logging, PDB’s, timeouts and improved exceptions management
- Monitoring view for Event Broker
- Improved PDF file transfer in PDF comparison plugin
- On-the-fly report synchronization with ALM
Downloadable Bundles
- Microsoft Dynamics 365 keywords package & tutorial
- Android & Appium keywords package & tutorial
- AutoIt & Office integration keywords & tutorial
- exense’s internal Selenium toolkit
2018
Evaluation
Evaluation done as of March 31st
| Goal | Completion % | Comment/Reason | EE only |
|---|---|---|---|
| Devops/CI/CD | |||
| Common test plan format | 80 | Step Natural Language + PlanBuilder released. Only missing code or at least unit test generation for local debugging | |
| New classes for client-side triggers | 100 | “STEPClient” released | X |
| Mobile | |||
| Ability to test Android apps & browser | 100 | Accomplished Java agent + Appium, also offered as a Cloud service | |
| Appium integration | 100 | Tested and now officially supported. No need for a dedicated plugin though. | |
| HTTP | |||
| HAR Replay | 30 | Achieved at a client, not officially integrated or supported yet. Official HTTP keywords offered as an alternative | |
| Custom script generation | 0 | Abandoned for now. If anything happens in this department, it’ll be through the HAR replay(er). | |
| OOTB support of SSL contexts | 90 | Provided via our official HTTP keyword package. Just not officially released/OS’ed yet, but in use at clients. | |
| Gatling & co | 0 | Not a high priority at this point. | |
| Keyword dev | |||
| Helper classes | 60 | In use internally and at some clients, not officially released yet. | |
| API Refactoring | 70 | Major improvements have been done. Inputs & Outputs still expected to change. API has now be desync’d from STEP for easier maintenance | |
| Starter kits | 100 | Sample projects implemented | |
| Javadocs | 100 | ||
| Plans dev | |||
| Generic keywords | 80 | Mecanism implemented. Done in EE for broker, in progress for HTTP | X |
| Plan templates | 100 | Available for import under Libraries in our docs | |
| Refactoring of artefacts & new artefacts | 100 | ||
| Previous KW output reference | 100 | ||
| Init hooks | 0 | Can be handled in other ways, low prio | |
| Ops | |||
| Export/Import | 100 | Done for Plans | |
| Data retention strategy | 100 | Done but can be improved | |
| Admin view improvements | 0 | Could not do in time. | |
| Self-Monitoring | 50 | Not published yet but new features in use as beta version at key clients. | |
| Misc | |||
| Doc refactoring & migration | 60 | Should be released by May | |
| Github status & widget | 0 | Low prio. | |
| Artefact publication in Maven | 0 | Low prio. | |
| Not in roadmap but done anyways | |||
| Resource manager | 100 | ||
| Notifications | 100 | ||
| Pupeteer | 100 | ||
| Async support | 100 | ||
| & much more (see release notes)… |
Initial Roadmap
Additional DevOps-oriented functionality
- Introduction of a common format for test plans (with translation between Java code and persisted plans in the web app) to enable wider cross-practice collaboration
- Introduction of new client classes providing client-side triggers, remote test plan execution and remote keyword execution to provide developers with more flexibility, facilitate integration with build processes and allow developers to commit test plans as part of their projects
Mobile testing support
- Support of both native mobile applications and mobile-based browsers (as a simulation and on real devices)
- Integration with popular mobile testing frameworks, preferably based on the WebDriver API such as Appium (Selendroid a possibility)
Avanced HTTP Plugins
- Support of HAR replays
- Custom script generation
- Out of the box support of SSL contexts
- Possible integration with additional HTTP frameworks such as Gatling (to complete the existing library: JMeter, Grinder and SoapUI)
Easier keyword development
- Exposure of frequently used Helper classes for keyword input / output management and error handling
- API refactoring (including the replacement the javax json library)
- Publication of starter kits and examples of best practices for the most common types of projects
- Publication of detailed Javadocs and source jars
Easier test plan development
- Introduction of generic keywords for the most common technical use cases and out-of-the-box support of standard protocols (htttp/s, ftp/s, etc) without having to use any plugins (grinder, jmeter, etc)
- Introduction of test plan templates for the most common use cases (data-driven, load and monitoring scenarios)
- Refactoring of key artefacts such as “Set”, “Composite” KWs or “CallFunction” and introduction of new artefacts in order to enhance test plan modularity and reusability
- Automatic access to previous keyword outputs for finer linkage
- Addition of various Init/Pre and Exit/Post hooks, destined to be used primarily in parallel artefacts (For, ForEach, ThreadGroup), as a first step toward the support of a jointpoint-style Test plan API
New operational features
- Introduction of one-click import and export for most objects (in particular Test plans and Executions)
- Introduction of data retention strategies
- Enhancements in the admin view to publish all of the internal technical information (controller + agents)
- Self-monitoring (at system, middleware and client code levels)
Misc.
- Migration, refactoring and update of the documentation
- Improved project status on github (nightly buildwidget)
- Direct artefact publication to the Maven Central repo
2017
Evaluation
No thorough evaluation of the 2017 roadmap took place but most of the features have been shipped in v3.5 and v3.6. Roadmap was still very much subject to major changes and pretty volatile as of early 2017.
Initial Roadmap
- Plan export & import
- NodeJS agent
- New artefacts: While, Assert, Synchronized
- JSON schemas for keyword input validation and input keys autofill
- Add “previous” object reference to access the output of the last completed Keyword call
- Interactive Test Plan Sessions
- Autoclose on Session end and test end