Variables & Parameters

Variables

Variables are values that are assigned within a test, and can be used at any point later on in that same test.

Here are a few examples of when a variable comes in handy:

1. Testing Your Registration Flow: Being able to register a user is often one of the most important end-to-end tests you can create. However there’s one big challenge with creating automated tests for this flow: once you’ve registered an email address/username in your system, you can’t register it again. By associating a variable to the username / email address in your test, and assigning it a dynamically generated value, you can ensure that your test registers a new user every time it runs.

2. Verifying values that change over time: Imagine you want to test a shopping cart workflow. The test likely consists of adding a specific product to the shopping cart, purchasing it, and verifying that the proper order total is shown. But what if the price of the product changes over time? In Reflect, you can assign the product price to a Variable so that you can assert that the order total (e.g. the product price multiplied by some quantity and tax rate) is correct no matter what price the product ends up having.

Defining A Variable

Input Text

Variables can be defined on both input text and extracted text. To assign input text to a variable, click on the Input step and locate the Execute Action section.

Click the Assign to Variable button and give the variable a unique name:

Extracted Text

To assign extracted text to a variable, highlight the portion of the extracted value that you wish to assign to a variable, select the Assign to Variable context menu option, and give the variable a unique name:

Parameters

Parameters are similar to Variables, and in fact behave identically to Variables once they have been assigned. Whereas Variables are for dynamic values that are captured somewhere within a test (or are defined globally as Account Variables), Parameters are values that are passed into the test.

Here are a few example usages of Parameters:

1. Testing Role-based access controls: Imagine you have a new feature in your application, and you want to verify that it works for all of the different user roles supported in your application. With Parameters, you can create a single test that verifies your new feature, and parameterize the login credentials used in that test so that you can run it as a data-driven test across all user roles.
2. Testing internationalized content: To test i18n content, you would first create a test that asserts all of the text content for particular scenario using some common locale. To test against other locales, you could modify the test to pass in the desired locale as a parameterized request header, and then parameterize all of the text assertions with the expected values in that particular locale.
3. Testing values that differ across environments: If you’re running automated tests in more than one environment, it’s likely that each environment has some differences in data that need to be accounted for in your tests. For example, if the username and password of your default test user differs depending on the environment, you can set up a single “Login” Segment that takes a username and password as Parameters, and pass in the credentials for your test user in that environment.

When defining a Parameter, you can optionally assign a default value. A Parameter’s default value will appear on the Test definition page, along with a link to all of the steps that reference the Parameter within the Test. Additional Parameters can also be added to the Test by clicking ‘Add Parameter’ on the Test definition page.

In order to delete a Parameter, you must first delete all of its references within the Test.

Using Parameters in Segments

In addition to defining Parameters for a Test, Parameters can also be defined for a Segment. When a Segment is added to a Test, any Parameters associated with the Segment are now associated with the Test.

Let’s consider a test that logs into an application, uses that application’s search function to search for an existing record, and finishes by doing some assertions on the data displayed for that record. If you were building out other tests for this application, probably a lot of those tests would start by logging in, and maybe a fair number of them would use the search box to search for a record (such as tests to find and delete a record, or find and edit a record).

When the same set of actions appear in multiple tests, that’s a good indication that those actions should be refactored into a reusable /docs/recording-tests/composition/. So in the case of this test, we’d extract two sets of test steps into Segments: one segment for logging in, and another segment for searching for a record.

Now let’s consider what we might want to change about these Segments if we wanted to use them in other tests:

• For the “Login” segment, we might have tests that require logging in as a different user than what we originally recorded.
• For the “Search” segment, we might have tests that require searching for a different search term than the exact search term we originally recorded.

If there is data within a Segment that should change depending on what test is using it, that’s a good indication that that data should be turned into a Parameter:

• For the “Login” segment, we’ll create two Parameters: “username” and “password”. We’ll keep the default test user’s credentials as the default values for this Segment, since most tests are going to be logging in as this user.
• For the “Search” segment, we’ll create one Parameter: “search term”, which is the input that’s passed into the search box before the search button is clicked. We might elect to not save a default value for this Parameter, since so many different kinds of search terms could be used, and we want to ensure whoever uses this Segment knows they have to define the proper search term for their test.

When inserting a Segment containing a Parameter into a Test, you will be prompted to define the Segment’s Parameter values.

Populating a Segment’s Parameters using Variables

In the example above, we described a test for an authenticated application that includes a Segment which uses the application’s search functionality. This Segment contains a single Parameter called “search term”, which has no default value.

Because this Parameter has no default value, if we want to use it in another test then we’re going to need to define its value somehow. Normally you would define some static value and save it as the Parameter’s default value for that Test, but an alternative would be to use some value that’s extracted earlier on in the test itself.

A good example would be a test that creates a new record, and then immediately searches for that record in the app. In this scenario, we’d want to extract the ID of the new record that was created, and search for that dynamic ID in the search box. We can do this by assigning the record ID to a variable called “search term”, and then inserting the Segment somewhere after we extracted the search term.

When inserting the Segment, you’ll notice that the “search term” parameter is shown as already populated with the value extracted earlier in the test, and the Parameter’s default value is not editable. Whenever this test runs, the “search term” parameter will be populated with whatever new record ID was generated by the application.

On the Test definition page, Segment Parameters will also appear with its default value disabled, indicating that the value is defined as part of the test. The location where the Parameter’s value is defined can be found by finding the Variable of the same name on the Test definition page, and clicking the ‘Assigned in Step X’ link.

Functions

To generate a dynamic value for a given variable, you can use one of several supported Functions. Support for generating random letters and digits comes from the following functions: alpha(len), num(len), and alphanum(len). Each of these functions takes a value that specifies how many random characters to generate. The range(min,max) function generates a random number between min and max, inclusive. Support for generating timestamps and dates comes from the following functions:

time(offsetMs) - returns an epoch timestamp after adding offsetMs milliseconds to the timestamp,

datetime(offsetMs) - returns a date time string of the form “Wed Oct 07 11:19:04 EDT 2020” after adding offsetMs milliseconds to the date,

date(format,offsetDays) - returns a formatted date string after adding offsetDays days to the date. The supported formats include only the characters: dDmMyY /, and are parsed using Java’s SimpleDateFormat.

Examples:

In the var examples, varOne is defined as value1 and varTwo is defined as value2. In the sec example, the account’s Secret, my-password, has the value abcd1234.

Assertions

Reflect automatically extracts text from the elements you interact with. In some cases this text will be static, such as the text on a Sign In button. But in other cases the text could change over time, or potentially on every test run. To assert against dynamic values, click on the test step containing the text you want to assert, and under Expected Text, highlight the text you wish to assert and choose the Add Assertion context menu item:

The following types of assertions are supported:

• match exactly: Does an exact string match against the specified value, which can reference an existing variable.
• match using math formula: Asserts that the value is a number matching the formula defined in the Expected Value input. Formula assertions support simple arithmetic, and can utilize both static and dynamic values.
• match number within range: Asserts that the value is a number within the range of values specified in the From and To fields.
• match a value in a list: Asserts that the value matches one of the discrete list of items defined in the list.
• is not empty: Evaluates to true so long as the value is not empty.

To assert that this dynamic portion of text matches an existing variable, choose the Match Existing Variable context menu option and select the variable you wish to match against:

In addition to adding these assertions, you can choose to simply ignore text by selecting the Ignore Selection context menu option.

Secrets

Reflect supports storing sensitive values as “Secrets”. Secrets are similar to Variables in that they serve as named values that can be referenced within tests and segments. However, secrets are stored in a separate database using a separate key to store the encrypted value.

Only Administrators can create or edit secrets. Apart from that, the secret value is only decrypted at the time that it’s used within the test at run time.

To insert a secret value into a test at runtime, use the same dropdown as for Variables, as outlined above. To insert a secret when manually constructing a dynamic input string, such as when integrating Reflect with a Test Case Management tool, use the following syntax in a dynamic input field: ${sec(my-secret)}.