Reflect Docs

Viewing a Test Run

A single view for examining a test run, as well as inspecting and modifying a test.

Clicking on the name of a test in the Test List view opens the test detail view and displays the test’s most recent test run. This view contains both information about the test and its execution, as well as controls for modifying the test definition and metadata.

Test Detail View

The test’s basic metadata appears at the top of the left panel. This metadata section includes the ability to rename a test with the pencil icon, and change the test’s associated tags. The “Run Now” button executes a new test run of the current test, and the “History” button opens a new panel that shows the historical runs of the test, with the most recent run at the top.

The right side of the page displays a video recording of the current test run. The video automatically begins playback when the page is loaded, and if the test run resulted in failure, the playback skips ahead to the execution of the final test step. The top of the right panel displays the URL at the current moment in the test execution (as displayed by the video), and below the URL is the video playback control button which allows skipping to an arbitrary moment in the video. The panel below the video displays the JavaScript logs printed by the page during execution. The bottom panel can be toggled to show the log of network requests issued by the page.

Back in the left panel, below the metadata section, is the test definition, which is displayed as a simple text description of the ordered test steps. The test steps are highlighted in-sync as the video plays so that you can easily identify the currently executing test step.

Test Step Detail View

Test steps are interactive and clicking on one opens a new panel with detailed information about the test step’s execution in the current test run.

Selectors

The first section lists the selectors that were captured at test recording time that uniquely identify the element that the test step interacts with. Additionally, this section shows the outcome of using these selectors to identify the element in the current test execution. The test executor will use the selectors one at a time to query for elements on the page until it finds a unique, visible element. The first selector that matches a unique, visible element displays a green check mark, and any selector that wasn’t used at all is grayed out.

As a page’s HTML changes over time, these selectors may no longer uniquely identify a visible element. If this happens, the selector list displays the reason why the selector failed to find an element. These include the selector not finding any elements (not found), the selector matching more than one element (not unique), and the selector matching an element that is out of the viewport (not visible).

Input Text

If the test step requires entering input text into an element on the page, either via an input, textarea or contenteditable element, then that text content is displayed in the Input Text section.

Expected Text

Any test step that interacts with an element that displays visible text will display the Expected Text section, and will show a ‘diff’ view comparing the expected text to the actual observed text during the current test run. In this example below, the expected text was “7:50 PM” but the actual text displayed was “8:57 PM”.

Visual Observe

Visual Observe test steps display both the baseline image of the element and the actual image of the element captured during the test run. If the images differ, then a third image, called the “delta” image, is displayed next to the images. The third image shows the pixel-by-pixel differences between the baseline and the actual image, with the different pixels highlighted in red.

Attributes

Reflect captures all of the attributes for the element that a test step interacts with. The attributes section displays the values both at test recording time, as well as at test execution time if the values differ. In this way, you can verify the presence of key attributes such as aria- attributes.

Test Step Editing

Several of the test step detail sections expose editable elements used to modify the configuration of the test step (and ultimately the test definition). See Test Editing for full information about the editing actions that Reflect supports.

Copyright © 2020 Reflect Software Inc. All Rights Reserved.