Step details - feature overview

The following article describes the features of Step details located in the Test step analysis module in the Reporting section of the Analytics application. For an intro to Test step analysis see this article.

Overview

Step details provides deep level test step analysis and can be accessed from the Test step analysis report, the Process heatmap or directly from a UUT report. 

Step details is divided into two major parts, the menu and the view area. To better organize the functionality of Step details, it is divided into multiple views, each focusing on a different aspect of the analysis.

Top bar, menu bar and view area

The menu

From the menu bar the user can access the different views available, see and change the steps to analyze as well as access settings and information about the filter applied.

To access the steps click the icon at the top of the menu. The menu can be pinned by clicking the pin icon in the upper right corner of the menu.


 

To analyze a single step in the list, click the name of the step. To analyze multiple steps click the checkboxes next to the steps and click the green Analyze selected button.

Note that due to some step types being multi-steps, Step details refers to the selected steps as measurements. So if the selection consists of a numeric step as well as two out of four measurements of a multi numeric step, the analysis will indicate that the analysis is performed on three measurements, one from the single numeric and two from the multi numeric.

Alternatively Step details can automatically perform a selection of steps based on a few criteria. To perform a quick selection, click on the button to left of the green Analyze selected button to open the selection overview. This open a dropdown displaying the currently selected steps in the left hand side and the quick selection tool on the right.


Current selection and quick selection tool

Views

The following describes the different views available in Step details:

• Overview: The "Overview" serves as the central hub in Step Details. It offers a high-level view of your entire dataset, displaying key metrics aggregated by period at the top and detailed measurement data in various charts and graphical components below. The aggregated period chart also functions as a filtering tool, allowing you to easily identify and select relevant data slices for visualization in the rest of your analysis.
   
• Workbench: The workbench view is intended for detailed graphical analysis of your measurement data. It puts the chart in focus and offers a variety of controls for customizing the chart display, filtering data, and fine-tuning analysis. For a closer look at the workbench view, see this article.
  
• Step comparison: The step comparison view is designed to examine the relationship between two different steps. It features a chart where each point represents the measured values from the same test execution, with the x-axis showing the value from step 1 and the y-axis showing the value from step 2. A clear trend in the plotted points suggests correlation, while a scattered distribution indicates little to no relationship. This helps in identifying dependencies, trends, and anomalies in test results.
  
   
• Step chart: The Step Chart View enables you to analyse chart data for your selected steps with ease. It offers two primary modes—Aggregated and Individuals—to balance the need for detailed analysis with optimal performance when working with large datasets. This view allows you to compare discrepancies across multiple steps simultaneously while providing intuitive tools to highlight and control the visibility of the data most relevant to your analysis. For a closer look at the step chart view, see this article.
  
• Step correlation view: The Correlation View in WATS helps you quickly discover relationships between step measurements. By selecting a step measurement as the reference, the view displays other step measurements that behave similarly. This allows users to investigate variations and potential root causes more efficiently. To read more about the Step correlation view, see this article.
   
• Data view: The Data view presents measurement data in a grid format, enabling you to analyze dimensions and metrics of interest. It also allows exporting the data to Excel, CSV, or TSV files.
  

Full data set and loaded selection

Step details works with two sets of data:

• Full data set: All measures of the analyzed steps that match the filter applied in Test step analysis. The aggregated period chart at the top of the Overview, the aggregated chart in the Workbench and the step list in the menu always show the full data set.
• Loaded selection: The individual measures loaded for detailed analysis. These are used by the Individuals chart and the components below it in the Overview, the individuals chart in the Workbench, and by the Step comparison, Step chart, Step correlation and Data views.

To change the loaded selection, adjust the sliders in the aggregated period chart in the Overview, or click a single period, and click the Apply selection button. The sliders show how many measures the selected timeframe contains. If this is more than the maximum allowed, two numbers are shown, for example 10.000 / 50.000. In that case, only the most recent measures up to the maximum are loaded. Click Reset to return to the default selection.

When the loaded selection is smaller than the full data set, counts and other metrics in the step list can differ from those in the views that use the loaded selection.

Measurement selection settings

Two settings control the size of the loaded selection. To change them, click the Settings icon (gear) in the menu bar and go to the Measurement selection section:

• Default selected measurements: The number of measures selected when Step details opens, or when the selection is reset. The default is 1.000. The value can be up to 10.000, but never more than Max allowed selected measurements.
• Max allowed selected measurements: The highest number of measures a selection can load. The default is 10.000, and it can be raised to 100.000.

The settings are saved to your user profile, so they still apply the next time you open Step details.