Automating Manual Tests

Running tests

Reflect’s free Chrome extension lets you run your manual test cases as automated Reflect tests. Tests can be run from our extension without even needing to have an existing Reflect account. Note that in order to view the results of a test run (including a video recording of the run, pass/fail details, and console/network logs), you’ll be prompted to create a free Reflect account if don’t already have one.

Reflect leverages Generative AI to interpret and execute a manual test script as an automated test. Each step within your manual test effectively becomes an AI “prompt” that serves as the input to our AI engine. At runtime, the Reflect AI engine takes these inputs and analyzes the current state of the application to determine what actions and/or assertions need to be performed in order to fulfill each test step.

Our Chrome extension displays test results in realtime, and provides the ability to watch tests run live within Reflect with a single click. You can also see the history of test runs for a given test, and click to view the results of any test run.

If you’re in the middle of a regression test pass (sometimes called a Test Run or Test Cycle depending on the tool you use), then the test runs you create will be named after the Test Run / Test Cycle. This makes it easier to understand the context of historical test runs when you view them later.

Updating tests

When running tests via our Chrome extension, the source-of-truth of the test becomes the test steps defined within your test case management tool. Any updates made to your test case will be captured and synced to Reflect the next time you run your test case via our Chrome extension.

Linking to an existing Reflect account

If you are already a Reflect customer, you can link the Chrome extension to your Reflect account via the following steps:

• Within the Reflect web app, navigate to the Settings section.
• Under Account Information, find the section entitled ‘Reflect Extension’. If an Extension API Key does not already exist, click the ‘Generate Extension API Key’ button to create one.
• Copy the Extension API Key to your clipboard.
• Next, open the Reflect Chrome extension by clicking the Reflect icon in the Chrome toolbar.
• Click the gear icon in the top-right of the sidepanel that is displayed.
• Paste the Extension API Key value into the input field, and click ‘Update Settings’.

Your Chrome extension is now linked to your Reflect account. Any tests executed from the Chrome extension will now appear in your existing Reflect account.

Supported Integrations

The following test case management tools are supported by our Chrome extension:

TestRail

Within TestRail, you can execute test cases automatically from both the Test Case view, as well as within the context of a Test Run.

Test Case

Go to the Test Cases section and click on a test case. Any test case that contains a ‘Steps’ section that has individual test steps (such as when using the ‘Test Case (Steps)’ template) can be executed by Reflect.

To run a test case automatically, click the ‘Run as Automated Test’ button that appears in the sidebar, or click on the Reflect extension icon within the Chrome toolbar. The sidepanel will open and provide options for executing the test case.

Test Run

Test Cases can also be executed automatically within the context of a Test Run.

To run a test case automatically, navigate to your Test Run and click the ‘Activity’ link in the sidebar to view the list of test cases in this run. Click the desired test case within the Activity section. To run the test case from this view, either click the ‘Run as Automated Test’ button, or click on the Reflect extension icon within the Chrome toolbar. The sidepanel will provide options for executing the test case.

Xray

To run an Xray test, simply navigate to any Jira issue that is an Xray test and either click the ‘Run as Automated Test’ button that appears above the list of test steps, or click on the Reflect extension icon within the Chrome toolbar when viewing the Xray test.

Please note that for an Xray test to be executable within Reflect, it must contain at least one test step. If you’re using Xray’s default settings, the test will need to have a Test Type of ‘Manual’ to allow individual test steps to be defined.

Zephyr Scale

Within Zephyr Scale, you can execute test cases from either the Test Case view, or within the Test Cycle player.

Test Case

Any Test Case that has a ‘Type’ of ‘Step by Step’ can be run automatically using the Reflect Chrome extension.

To run a test case automatically, first click on the ‘Test Script’ tab when viewing a test case. A ‘Run as Automated Test’ button will appear within the Test Script section. Options for running the test case will appear in the sidepanel after clicking the ‘Run as Automated Test’ button, or after clicking on the Reflect extension icon within the Chrome toolbar.

Test Cycle Player

Test Cases can also be executed automatically within the context of a Test Cycle.

To run a test case automatically, first navigate to your desired Test Cycle (e.g. by clicking Zephyr Scale on the sidebar, selecting the Test Cycles tab, and clicking a Test Cycle from the list).

Each Test Case within the cycle can be executed automatically by selecting the Test Case in the left sidebar and subsequently clicking on either the ‘Run as Automated Test’ button that appears, or by clicking on the Reflect extension icon within the Chrome toolbar.

Zephyr Squad

Tests can be executed when viewing an individual Test, or when viewing a Cycle Summary.

Test Case

First, navigate to the Jira issue that represents the Zephyr Squad Test that you wish to run.

Within the Test Details section, a ‘Run as Automated Test’ button will appear. Clicking this button or clicking the Reflect extension icon will reveal a sidepanel that displays options for running the Test automatically in Reflect.

Cycle Summary

Tests can also be executed automatically within the context of a Cycle.

First, navigate to the Cycle Summary by clicking Zephyr Squad in the left sidebar and clicking the Cycle Summary link. Next, select your desired Cycle and enable the ‘Detail’ view if it is not already active. A ‘Run as Automated Test’ button will appear within the Cycle Summary view. Clicking this button or clicking the Reflect extension icon will reveal a sidepanel that displays options for running the Test automatically in Reflect.