Editing / Re-Recording

Reflect’s philosophy on automated regression tests is that “coding” or “scripting” front-end tests is tedious and error-prone. As a result, Reflect does not support creating tests this way, and instead, derives test definitions automatically from instrumenting normal user web sessions. This eliminates some forms of user-error, which can lead to false positive test failures, and it produces a more consistent test definition, which fosters more efficient test execution. An added benefit is that anyone in your organization can create and manage tests.

Of course, it can be just as tedious to have to re-create a test from scratch when all you need is to modify one test step. Thus, Reflect supports several methods of editing an existing test, but does so in a way that is still rooted in Reflect’s cloud-based browser, rather than coding. All forms of test editing are available from the Test Detail page, while test configuration such as changing folders and adding a description can be done from the test definition page.

Accepting Changes

A simple way to modify a test definition is to accept the current state of the page’s content in place of what was recorded. This most often occurs when a test is failing a Visual Validation test step due to the visual state or text content not matching what’s expected. The natural evolution of a page’s content means that there will be legitimate changes in expected content over time, and Reflect makes it easy to accept these changes as the new baseline assertion values for the test.

To accept changes for a Visual Validation or other test step, click the test step to open its Test Step Detail view. From there, inspect the actual content displayed on the page compared to the baseline. If the actual content is correct, click “Accept Changes” in the specific sections.

After clicking “Accept Changes”, a Save button will appear at the bottom of the test definition panel.

Click the Save button to save the accepted changes.

Input Changes

If your test includes steps for entering text into the page under test, you can alter this input text from the Input Text section on the Test Step Detail view. Whether it’s an input element or textarea, the detail view is editable and accepts changes directly in-line. As with other edits, you’ll need to click the Save button when you are finished editing the text in order to save the edits to the test definition.

It’s important to remember that changing the input values in your test may result in subsequent test actions being invalid depending on how the new input values change the webpage. As before, click Save to save the changes to your test definition and trigger a new test execution.

Delete a Test Step

To delete a step from a test, click on the test step to open the Test Step Detail view. Scroll to the bottom of the panel and click Delete Step.

Multiple steps can be deleted at once by clicking the checkboxes in the left sidebar next to the test steps you wish to delete, and clicking the Delete Steps option in the toolbar that appears at the top of the page.

Re-recording a Test

The most flexible way to modify a test’s definition is to re-record the test through Reflect’s interactive test editor. Re-recording allows you to insert or delete test steps anywhere in the test definition without repeating actions that you have already recorded.

Initiate the interactive test editor by hovering between the two steps where you wish to insert new step(s), and click the option Re-record from this point.

This will navigate your browser to the regular Recording experience, except your existing test definition will already be populated on the left sidebar. The Reflect cloud browser will execute the test definition up until the test step that you previously selected. When playback is complete, Reflect will enable the browser for recording and begin capturing your actions and inserting them into the test.

When you have finished inserting new test actions, click “Resume Execution” to watch Reflect continue the execution of the test definition from the current location.

In some cases, you may wish to re-run a subset of test steps before saving the test. To do so, either click and drag the “execution pointer” to the desired location, or hover between the steps you wish to move it to and click “Move execution pointer”. If the new location is earlier in the test, Reflect will automatically insert a breakpoint at the previous location.

If you wish to interact with the browser session without recording additional steps, click the Recording checkbox in the top right of the browser window. Additional steps will not be recorded when the Recording checkbox is deselected.

When the test has completed, or if you simply wish to save the updated definition and leave, you can click “Save”.

Reflect will continue the execution of your test to its conclusion (if it’s not already there), and then save a test run for the updated definition. You can always end the re-recording session without saving any changes by clicking “Exit” or “Exit without Saving”.

Watch Live

When Reflect is executing a test, you have the option to watch the test run live via the “Watch Live” button that appears wherever the test run is displayed in Reflect. Clicking this button will take you to the same interactive editor and viewer that is used when re-recording. This means that in addition to watching the test execute, you can also pause and edit the test.

Interactive Editing

During a re-recording or when watching a test run live, it’s possible that steps in the original test definition will fail during playback when the Reflect test executor tries to perform them. By default, the executor will retry a failing step for up to 30 seconds before failing the test. During a re-recording session, Reflect allows the user to interrupt the retry process and take corrective action manually. When the executor begins retrying the test step, you’ll see a banner at the top of the recording session indicating the retry attempt.

Click “Stop Retrying” to interrupt the execution, or you can wait for the retry timeout to elapse (e.g., 30 seconds). Then, Reflect will display contextual information describing the failed test step.

From here, you can select the best option for fixing the failing test step. Often, you will want to click “Replace Step” to re-enable the recording experience and overwrite the failing test step with newly-recorded steps. Alternatively, if the failing test step is erroneous, you can delete it outright and resume execution. If the failing test step is a Visual Validation, you can click “Use latest visuals” to explicitly update the text and baseline image to the live values. If the failing test step is a Text Validation, the “Use latest text” option will let you update the step to use the latest text value.

Copying a Test

If you have a test that you don’t want to change, but you want to reuse some or all of the test’s existing test steps in a new test, you can use the “Make a Copy” feature. Since a Reflect test is defined by the sequence of test step actions, making a copy is a lightweight operation that creates an independent logical test with exactly the same test steps as the original. You can then execute, modify and delete the copied test without any impact to the original test.

From the test definition page, click on the button with the three dots on the top right of the page, and then click “Make a Copy” within the dropdown menu:

After making a copy, you’ll be redirected to the test definition page for the newly-created test. From there, you can execute the test (with the “Run Test” button), or modify the test definition, including through the re-recording feature described above.

Restoring a Previous Version

Reflect stores each version of a test’s definition independently, and associates each test run with a specific version of the test. The test definition page lists all historical runs for a test. You can update, or revert, a test’s definition to use a prior definition by clicking “Restore this Version” from the context menu on a previous test run.

Reflect does not immediately execute the test after restoring a previous version. This allows you to re-record from a specific step from the updated definition in the left sidebar, or otherwise edit the test before executing it. To run the restored test definition, click “Run Test”.