Async packages

Async and event-driven functionality is implemented in step in the form of what we called the Event Broker, a central component allowing developers and plan users to manage the transmission of events, as well as a series of controls and built-in keywords.

Event Broker overview

The basic semantics and the different means to access the broker are illustrated on the following figure:

The private members of the Event are as follows. While the id, group and name are metadata allowing to track the event and get a handle on it in both the pool and exactly-once scenarios, timestamps are carried within the event itself for measurement purposes.

public class Event {
	
	private String id;
	private String name;
	private String group;
	private Object payload;
	
	private final long creationTimestamp;

	// For client side
	private long submitionTimestamp;
	private long receptionTimestamp;
	
	// Server side
	private long insertionTimestamp;
	private long deletionTimestamp;
	private long lastReadTimestamp;
  [...]
}

Full details on the class are provided here.

Event Broker API

Details regarding the Event Broker API are provided here.

Async Controls

The following controls have been added:

• WaitForEvent: forces the plan or current thread in the plan to natively wait for a specific eventId or eventGroup. Works locally as well as remotely, by providing a remote controller’s credentials.
• EBC_Put keyword: pushes a new event into the Broker’s collection.
• EBC_Get keyword: retrieves an event from the Broker’s collection, based either on its identifier or group & name.
• EBC_Peek keyword: browses events from the Broker’s collection, based either on its identifier or group & name.

Monitoring view

Basic documentation of the dedicated event monitoring view is also available here.

EventBroker Properties

Here are some of the properties which can be set centrally as part of step’s configuration (i.e step.properties).

  eventbroker.circuitBreakerThreshold

This property centrally sets the value for the circuitBreaker which is used as a protection to cap the event collection. Expected value: integer.

  eventbroker.advancedStatsOn

This property decides whether or not the EventBroker’s advanced statistics are active. Can be turned off for performance reasons in extreme scenarios. Expected value: [true|false].

  eventbroker.uniqueGroupNameOn

This property decides whether or not Events tagged with the same group and name should be unique. Expected value: [true|false].

Upcoming

Here are some additional components which may be developed in the future:

Continuous Keywords

Continuous Keywords would implement new methods such as start() and stop() and possibly handle() in order to enable the deployment of continuously running code. Continuous Keywords would leverage distribution across step’s agent grid as well as library packaging and other keyword-related services to simplify use cases such as the implementation and operation of service mocks in event-driven scenarios.

Queue persistency

Events are not currently backed by a persisted collection. Instead, they’re stored in memory. Introducing a persistent mode would enable users to resume operations after a controller reboot, provide higher robustness in the event of a potential crash or unexpected shutdown as well as allow for scenarios in which large volumes of events are to be loaded. Currently, this can only be done at the expense of memory by increasing the circuitBreaker’s threshold value.

Advanced automatic monitoring

Currently, basic monitoring is provided via the EventBrokerMonitor’s statistics and via the manual addition and scheduling of monitoring plans to poll and chart these statistics over time. One could leverage the existing timestamps attached to events to automatically monitor metrics such as the average time spent by events in the queue or the average amount of time required to send an event to or retrive an event from the queue.