Reflect Docs

Suites API

Execute suites and retrieve results via CI/CD and other external systems.

List all suites

List all suites in your account.

Request
GET https://api.reflect.run/v1/suites
Response
{
  "suites": {
    "data": [
      {
        "name": "Regression Tests",
        "suiteId": "regression-tests",
        "created": 1638416940
      }
    ]
  }
}
Response Fields
suites
object
A wrapped collection of abridged suites.
suites.data.name
string
The suite name.
suites.data.suiteId
string
The suite identifier.
suites.data.created
number
The suite's creation timestamp as an epoch in seconds.

List suite executions

List recent executions for a given suite.

Request
GET https://api.reflect.run/v1/suites/<suite-id>/executions
Response
{
  "suiteId": "regression-tests",
  "executions": {
    "data": [
      {
        "executionId": 47,
        "url": "https://app.reflect.run/suites/regression-tests/executions/47",
        "isFinished": false,
        "status": "pending"
      }
    ]
  }
}
Response Fields
suiteId
string
The suite identifier.
executions
object
A wrapped collection of abridged executions.
suites.data.executionId
number
The suite execution identifier.
suites.data.url
string
The in-app URL that displays information and results about the suite execution.
suites.data.isFinished
boolean
A boolean indicating whether the suite execution has completed.
suites.data.status
string
One of: pending, canceled, passed or failed. Summarizes the state of the execution from its test instances.

Execute a suite

Triggers an execution of the specified suite.

The suite-id value for your suite should match the ID value displayed on the Suite Detail page, including casing. (See Integrating via API.)

Reflect identifies certain properties of a scheduled execution as “reserved”. These reserved fields can be overridden with values provided by the API request:

  • hostnames: Allows you to specify a replacement hostname value for a given target hostname.
  • parameters: Allows you to specify a URL parameter key/value pair that will be included (or overwritten) on the test’s initial URL.
  • cookies: Allows you to specify cookies that will be set when loading the test’s initial URL. A cookie consists of the following fields:
    • name (required string): The name of the cookie to be set.
    • value (required string): The cookie’s value.
    • domain (optional string): Host to which the cookie will be sent. Defaults to the host of the test’s initial URL.
    • expires (optional number): Epoch timestamp (in milliseconds) indicating when the cookie should be deleted. If unspecified, the cookie becomes a session cookie.
    • httpOnly (optional boolean): Indicates the cookie should be inaccessible to JavaScript on the page and only sent to the server.
    • maxAge (optional number): Number of milliseconds until the cookie expires. If both expires and maxAge are set, maxAge has precedence.
    • path (optional string): A URL path that must exist in a request URL in order to include the cookie in a request.
    • secure(optional boolean): Indicates that the cookie should only be sent to the server when a request is made with https:.
  • headers: Allows you to specify headers that will be sent when loading the test’s initial URL.
    • name (required string): The name of the header.
    • value (required string): The header value.
    • persist (optional boolean): Specifies whether this header should only be set in the initial request (ex: when defining an authentication header) or for every HTTP request (ex: when overriding the User-Agent or Accept-Language header). Defaults to false.
  • localStorage and sessionStorage: Allows you to specify local and session storage values to be set on the test’s initial URL.

The variables field allows you to specify a definition for a variable used in the tests.

Finally, if your Reflect account has OAuthed with GitHub, you can specify a repository, its owner, and a commit SHA in the request to have Reflect automatically post a commit status to GitHub as the suite’s execution progresses.

Request
POST https://api.reflect.run/v1/suites/<suite-id>/executions
{
  "overrides": {
    "hostnames": [{
      "original": "prod.myapp.com",
      "replacement": "staging.myapp.com"
    }],
    "parameters": [{
      "key": "token",
      "value": "abcdef"
    }],
    "cookies": [{
      "name": "my-favorite-cookie",
      "value": "chocolate-chip",
      "domain": "myapp.com",
      "expires": 123456789,
      "httpOnly": false,
      "maxAge": 123,
      "path": "/",
      "secure": true
    }],
    "headers": [{
      "name": "X-Custom-Header",
      "value": "custom-value",
      "persist": false
    }],
    "localStorage": [{
      "key": "my-local-key",
      "value": "local-value"
    }],
    "sessionStorage": [{
      "key": "my-session-key",
      "value": "session-value"
    }]
  },
  "variables": {
    "username1": "user+${alpha(8)}@example.com",
    "password": "acompletelyunguessablepassword"
  },
  "gitHub": {
    "owner": "repository-owner",
    "repo": "my-repository-name",
    "sha": "0f4212320f2cb6734583ebef3a4928d78d4f3fde"
  }
}
Request Fields
overrides
optional object
Overrides to apply to properties of all tests executed in this scheduled execution.
variables
optional object
Collection of ('name', 'definition') variable overrides to apply to this scheduled execution.
gitHub
optional object
An object specifying the GitHub repository name, its owner, and commit SHA to post the status of the execution.
Response
{
  "suiteId": "regression-tests",
  "executionId": 3878,
  "overrides": {
    "hostnames": [
      {
        "original": "prod.myapp.com",
        "replacement": "staging.myapp.com"
      }
    ],
    "parameters": [
      {
        "key": "token",
        "value": "abcdef"
      }
    ]
  },
  "gitHub": {
    "owner": "repository-owner",
    "repo": "my-repository-name",
    "sha": "0f4212320f2cb6734583ebef3a4928d78d4f3fde"
  },
  "url": "https://app.reflect.run/suites/regression-tests/executions/3878",
  "isFinished": false,
  "status": "pending",
  "tests": {
    "data": [],
    "bookmark": ""
  }
}
Response Fields
suiteId
string
The identifier specified in the execution request.
executionId
number
The unique numeric identifier assigned to the suite execution.
overrides
optional object
Overrides applied to properties of all tests executed in this suite.
gitHub
optional object
An object specifying the GitHub repository name, its owner, and commit SHA to post the status of the execution.
url
string
The in-app URL that displays information and results about the suite execution.
isFinished
boolean
A boolean indicating whether the suite execution has completed.
status
string
One of: pending, canceled, passed or failed. Summarizes the state of the execution from its test instances.
tests
object
A wrapped collection of test instances (test definition & execution configuration) with individual test runs.
tests.bookmark
optional string
Reserved for future - used to read additional test instances.

Get execution status

Returns the completion status of the specified suite.

Request
GET https://api.reflect.run/v1/suites/<suite-id>/executions/<execution-id>
Response
{
  "suiteId": "regression-tests",
  "executionId": 36,
  "overrides": {
    "hostnames": [
      {
        "original": "prod.myapp.com",
        "replacement": "staging.myapp.com"
      }
    ],
    "parameters": [
      {
        "key": "token",
        "value": "abcdef"
      }
    ]
  },
  "gitHub": {
    "owner": "repository-owner",
    "repo": "my-repository-name",
    "sha": "0f4212320f2cb6734583ebef3a4928d78d4f3fde"
  },
  "url": "https://app.reflect.run/suites/regression-tests/executions/36",
  "isFinished": true,
  "status": "passed",
  "tests": {
    "data": [
      {
        "instanceId": 0,
        "testId": 683,
        "startingUrl": "https://staging.myapp.com/test-one",
        "status": "passed",
        "runs": [
          {
            "runId": 2641,
            "browser": "Chrome",
            "status": "passed",
            "timestamp": 1638821720037,
            "variables": {}
          }
        ]
      },
      {
        "instanceId": 1,
        "testId": 686,
        "startingUrl": "https://staging.myapp.com/test-two",
        "status": "passed",
        "runs": [
          {
            "runId": 2642,
            "browser": "Chrome",
            "status": "passed",
            "timestamp": 1638821739491,
            "variables": {}
          }
        ]
      }
    ],
    "bookmark": ""
  }
}
Response Fields
suiteId
string
The identifier specified in the execution request.
executionId
number
The unique numeric identifier assigned to the suite execution.
overrides
optional object
Overrides applied to properties of all tests executed in this suite.
gitHub
optional object
An object specifying the GitHub repository name, its owner, and commit SHA to post the status of the execution.
url
string
The in-app URL that displays information and results about the suite execution.
isFinished
boolean
A boolean indicating whether the suite execution has completed.
status
string
One of: pending, canceled, passed or failed.
tests
object
A wrapped collection of test instances (test definition & execution configuration) with individual test runs.
tests.bookmark
optional string
Reserved for future - used to read additional test instances.
tests.data.instanceId
number
The identifier for a logical test instance (definition & configuration) in the execution.
tests.data.testId
number
The identifier for the test.
tests.data.startingUrl
string
The initial URL used by the test, accounting for any user-specified overrides.
tests.data.status
string
One of: pending, passed or failed. Summarizes the outcome of the test instance from its runs.
tests.data.runs
object
An array of test runs for a single test instance, with completion timestamp and status.

Cancel an execution

Attempts to cancel the specified suite execution.

Request
PATCH https://api.reflect.run/v1/suites/<suite-id>/executions/<execution-id>/cancel
Response
{
  "suiteId": "regression-tests",
  "executionId": 3878,
  "success": true
}
Response Fields
suiteId
string
The identifier specified in the execution request.
executionId
number
The unique numeric identifier assigned to the suite execution.
success
boolean
A boolean indicating whether the suite execution was successfully canceled.

Get started with Reflect today

Create your first test in 2 minutes, no installation or setup required. Accelerate your testing efforts with fast and maintainable test suites without writing a line of code.

Copyright © 2022 Reflect Software Inc. All Rights Reserved.