Reflect Docs

Connecting to CI/CD

Automatically execute Reflect tests from within your Continuous Integration/Continuous Deployment pipeline.

Via the Reflect API, you can automatically execute tests associated with a given tag (e.g. a ‘smoke-test’ tag). Below we’ve outlined how to trigger a Reflect API request from within several popular CI/CD tools.

Azure Pipelines

Jobs in Azure DevOps can be defined either as agentless / Server jobs or as a Pipeline job that is invoked on a remote agent.

In both cases, you’ll first need to create a Service Connection which represents the request to the Reflect API which kicks off your tests.

Creating a Service Connection

  • Select ‘Project settings’ from the left nav.
  • Select ‘Service Connections’.
  • Click the ‘New service connection’ button.
  • Select the ‘Generic’ option.
  • Under Server URL, enter https://api.reflect.run/v1/tags/<TAG-SLUG>/executions, replacing with the name of your tag as it appears in the URL of the Tags page within Reflect. To run all tests, replace ‘’ with ‘all’.
  • Under ‘Service connection name’, enter ‘Reflect Service Connection’.
  • Click the ‘Save’ button.

See below for examples using both flavors of job definition:

Agentless

steps:
- task: InvokeRESTAPI@1
  displayName: 'Invoke REST API: POST'
  inputs:
    serviceConnection: 'Reflect Service Connection'
    headers: |
     {
      "x-api-key": "<your api key goes here>"
     }     
    body: |
          {}
  enabled: true

Pipeline job

steps:
- task: CdiscountAlm.rest-call-build-task.custom-build-task.restCallBuildTask@0
  displayName: 'Rest call POST'
  inputs:
    webserviceEndpoint: 'Reflect Service Connection'
    httpVerb: POST
    headers: |
     {
      "x-api-key": "<your api key goes here>"
     }     
    body: |
          {}
  enabled: true

For more information on Azure Pipelines jobs, please refer to Microsoft’s online documentation

Bitbucket Pipeline

To kick off Reflect tests as part of your pipeline, you’ll need to add an additional step in your existing bitbucket-pipelines.yml

pipelines:
  branches:
    master:
    ...
    - step:
      name: Regression Tests
      script:
      - curl -X POST --data "{}" -H "x-api-key: <API-KEY>" https://api.reflect.run/v1/tags/<TAG-SLUG>/executions

We recommend that you set up your Reflect API Key as a secure environment variable rather than including it directly in your bitbucket-pipelines.yml file.

For more information on setting up Bitbucket Pipeline, please consult Bitbucket’s online documentation.

CircleCI

To enable automated test execution, add an additional step in your CircleCI config.yml:

jobs:
  build:
    steps:
    ...
    - run:
      name: Regression Tests
      command: |
                curl -X POST --data "{}" -H "x-api-key: <API-KEY>" https://api.reflect.run/v1/tags/<TAG-SLUG>/executions

Your Reflect API key should be set outside of version control as an Environment Variable rather than hard-coded in your config.yml.

More information on calling REST APIs as part of a CircleCI build is available on the CircleCI blog.

GitHub Actions

Integrating with Github Actions

GitHub Actions allow you to execute commands after some event occurs in your repository. One example is to execute Reflect tests after deploying your application to a new environment. Triggering the tests is as simple as issuing an HTTP request to the Reflect API. Additionally, if you have OAuthed your GitHub account with Reflect, you can specify the commit SHA on your pull request to have Reflect post test results back to the pull request’s commit as the tests complete.

name: Merge-Tests
on:
  push:
    branches: [ master ]

env:
  KEY_HEADER: "X-API-KEY: <API_KEY>"
  PAYLOAD: "{ \"gitHub\": { \"owner\": \"<OWNER>\", \"repo\": \"<REPO>\", \"sha\": \"${{ github.event.pull_request.head.sha }}\" } }"

jobs:
  run-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Run Reflect tests
        run: curl
             -X POST
             -H "$KEY_HEADER"
             -d "$PAYLOAD"
             -s
             https://api.reflect.run/v1/tags/<TAG_SLUG>/executions
Integrating via Webhooks

Alternatively you can trigger Reflect tests by configuring a Webhook. For example, you can set up your Webhook to execute when a deployment status event with the value success is returned, meaning that a deployment of a git commit hash to a given environment has completed successfully.

For more information on configurating Webhooks, consult Github’s online documentation.

Heroku

To configure Reflect tests to execute after a Heroku deployment, add the following Procfile with the command release: chmod u+x release.sh && ./release.sh

#!/bin/sh
#
# Execute commands after Heroku environment is fully deployed.
#
if [[ -z "${DEPLOY_ENV}" ]]; then
    echo "Environment variable not set"
else
    if [ "${DEPLOY_ENV}" = "staging" ]; then
        Tag="<my-tag>"
        echo "=== Triggering Reflect tests ==="
        echo "Tag=${Tag}"
        echo "Response:"
        curl --silent -X POST \
        -H "X-API-KEY: ${REFLECT_API_KEY}" \
        https://api.reflect.run/v1/tags/${Tag}/executions
    fi
fi

Gitlab CI/CD

A Reflect API request can be issued from your Gitlab CI/CD pipeline by modifying your existing .gitlab-ci.yml file:

regression_tests:
  stage: deploy
  script:
  - curl -X POST --data "{}" -H "x-api-key: <API-KEY>" https://api.reflect.run/v1/tags/<TAG-SLUG>/executions

The Reflect API key should be set as a predefined environment variable rather than hard-coded into the .gitlab-ci.yml.

For more information, check out Gitlab’s online documentation.

Jenkins Pipeline

A request to the Reflect API can be trigger via Jenkins Pipeline’s http_request plugin.

pipeline {
  stages {
    stage('Regression Test') {
      steps {
        ...
        script {
          httpRequest(url: 'https://api.reflect.run/v1/tags/<TAG-SLUG>/executions', customHeaders: [[name: 'x-api-key', value: "${API_KEY}"]], requestBody: '{}')
        }
      }
    }
  }
}

For more information, please review the Jenkins Pipeline online documentation.

LayerCI

LayerCI creates ephemeral staging environments for every Pull Request, allowing you to execute Reflect tests on every PR.

To integrate Reflect with LayerCI, first add your Reflect API key as a SECRET ENV within your LayerCI account.

Next, add the following shell script to your repository:

#!/bin/bash
REQUEST_BODY="
{
  \"overrides\": {
    \"hostnames\": [{
      \"original\": \"qa.example.com\",
      \"replacement\": \"$EXPOSE_WEBSITE_URL\"
    }]
  }
}"

EXECUTION_ID=$(curl --location --silent --show-error --request POST 'https://api.reflect.run/v1/tags/all/executions' \
    --header "X-API-KEY: $REFLECT_API_KEY" \
    --header 'Content-Type: application/json' \
    --data-raw "$REQUEST_BODY" | jq -r '.executionId'
    )

echo "Running the tests... Execution id: $EXECUTION_ID"

STILL_RUNNING_TESTS=true
while [ "$STILL_RUNNING_TESTS" = "true" ]; do
    EXECUTION_STATUS=$( \
         curl --location --silent --show-error --request GET "https://api.reflect.run/v1/executions/$EXECUTION_ID" \
              --header "X-API-KEY: $REFLECT_API_KEY" \
              --header 'Content-Type: application/json' \
         )
    
    echo "Checking test status..."
    TESTS_FAILED=$(echo $EXECUTION_STATUS | jq -c '.tests[] | select(.status | contains("failed"))')
    STILL_RUNNING_TESTS=$(echo -n $EXECUTION_STATUS | jq -c '[ .tests[] | select(.status | contains("running") or contains("queued")) ] | length > 0')
   
    if ! [[ -z "$TESTS_FAILED" ]]; then
       printf "\e[1;31mSome tests has failed.\nReflect Execution ID: $EXECUTION_ID\nFailed tests: $TESTS_FAILED\n\n" >&2
       exit 1
    fi
    sleep 5
done

echo "All tests have completed"

Finally, access your Reflect API key from the Settings page and invoke the shell script in your Layerfile. (The example below assumes the file is named reflect-test.sh.)

...
SECRET ENV REFLECT_API_KEY
RUN bash reflect-test.sh

Travis CI

Kicking off Reflect tests after a successful deployment can be done by executing an API request in the after_deploy section of your .travis.yml file:

after_deploy:
  - "curl -X POST --data \"{}\" -H \"x-api-key: <API-KEY>\" \"https://api.reflect.run/v1/tags/<TAG-SLUG>/executions\""

Ready to accelerate your testing efforts?

Create comprehensive end-to-end tests for your web application, without writing a line of code.

Copyright © 2021 Reflect Software Inc. All Rights Reserved.