Reflect Docs

Connecting to CI/CD

Automatically execute Reflect tests from within your Continuous Integration/Continuous Deployment pipeline.

Via the Reflect API, you can automatically execute tests associated with a given tag (e.g. a ‘smoke-test’ tag). Below we’ve outlined how to trigger a Reflect API request from within several popular CI/CD tools.

Azure Pipelines

Jobs in Azure DevOps can be defined either as agentless / Server jobs or as a Pipeline job that is invoked on a remote agent.

In both cases, you’ll first need to create a Service Connection which represents the request to the Reflect API which kicks off your tests.

Creating a Service Connection

  • Select ‘Project settings’ from the left nav.
  • Select ‘Service Connections’.
  • Click the ‘New service connection’ button.
  • Select the ‘Generic’ option.
  • Under Server URL, enter https://api.reflect.run/v1/tags/<TAG-SLUG>/executions, replacing with the name of your tag as it appears in the URL of the Tags page within Reflect. To run all tests, replace ‘’ with ‘all’.
  • Under ‘Service connection name’, enter ‘Reflect Service Connection’.
  • Click the ‘Save’ button.

See below for examples using both flavors of job definition:

Agentless

steps:
- task: InvokeRESTAPI@1
  displayName: 'Invoke REST API: POST'
  inputs:
    serviceConnection: 'Reflect Service Connection'
    headers: |
     {
      "x-api-key": "<your api key goes here>"
     }
    body: |
     {}
  enabled: true

Pipeline job

steps:
- task: CdiscountAlm.rest-call-build-task.custom-build-task.restCallBuildTask@0
  displayName: 'Rest call POST'
  inputs:
    webserviceEndpoint: 'Reflect Service Connection'
    httpVerb: POST
    headers: |
     {
      "x-api-key": "<your api key goes here>"
     }
    body: |
     {}
  enabled: true

For more information on Azure Pipelines jobs, please refer to Microsoft’s online documentation

Bitbucket Pipeline

To kick off Reflect tests as part of your pipeline, you’ll need to add an additional step in your existing bitbucket-pipelines.yml

pipelines:
  branches:
    master:
    ...
    - step:
      name: Regression Tests
      script:
      - curl -X POST --data "{}" -H "x-api-key: <API-KEY>" https://api.reflect.run/v1/tags/<TAG-SLUG>/executions

We recommend that you set up your Reflect API Key as a secure environment variable rather than including it directly in your bitbucket-pipelines.yml file.

For more information on setting up Bitbucket Pipeline, please consult Bitbucket’s online documentation.

CircleCI

To enable automated test execution, add an additional step in your CircleCI config.yml:

jobs:
  build:
    steps:
    ...
    - run:
      name: Regression Tests
      command: |
        curl -X POST --data "{}" -H "x-api-key: <API-KEY>" https://api.reflect.run/v1/tags/<TAG-SLUG>/executions

Your Reflect API key should be set outside of version control as an Environment Variable rather than hard-coded in your config.yml.

More information on calling REST APIs as part of a CircleCI build is available on the CircleCI blog.

GitHub Actions

Integrating with Github Actions

GitHub Actions allow you to execute commands after some event occurs in your repository. One example is to execute Reflect tests after deploying your application to a new environment. Triggering the tests is as simple as issuing an HTTP request to the Reflect API. Additionally, if you have OAuthed your GitHub account with Reflect, you can specify the commit SHA on your pull request to have Reflect post test results back to the pull request’s commit as the tests complete.

Note: In the example below, you may wish to refer to github.sha instead of github.event.pull_request.head.sha if your workflow is triggered by events other than pull-requests.
name: Merge-Tests
on:
  push:
    branches: [ master ]

env:
  KEY_HEADER: "X-API-KEY: <API_KEY>"
  PAYLOAD: "{ \"gitHub\": { \"owner\": \"<OWNER>\", \"repo\": \"<REPO>\", \"sha\": \"${{ github.event.pull_request.head.sha }}\" } }"

jobs:
  run-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Run Reflect tests
        run: curl
             -X POST
             -H "$KEY_HEADER"
             -d "$PAYLOAD"
             -s
             https://api.reflect.run/v1/tags/<TAG_SLUG>/executions
Integrating via Webhooks

Alternatively you can trigger Reflect tests by configuring a Webhook. For example, you can set up your Webhook to execute when a deployment status event with the value success is returned, meaning that a deployment of a git commit hash to a given environment has completed successfully.

For more information on configurating Webhooks, consult Github’s online documentation.

Heroku

To configure Reflect tests to execute after a Heroku deployment, add the following Procfile with the command release: chmod u+x release.sh && ./release.sh

#!/bin/sh
#
# Execute commands after Heroku environment is fully deployed.
#
if [ -z "${DEPLOY_ENV}" ]; then
    echo "Environment variable not set"
else
    if [ "${DEPLOY_ENV}" = "staging" ]; then
        Tag="<my-tag>"
        echo "=== Triggering Reflect tests ==="
        echo "Tag=${Tag}"
        echo "Response:"
        curl --silent -X POST \
        -H "X-API-KEY: ${REFLECT_API_KEY}" \
        https://api.reflect.run/v1/tags/${Tag}/executions
    fi
fi

Gitlab CI/CD

A Reflect API request can be issued from your Gitlab CI/CD pipeline by modifying your existing .gitlab-ci.yml file:

regression_tests:
  stage: deploy
  script:
  - curl -X POST --data "{}" -H "x-api-key: <API-KEY>" https://api.reflect.run/v1/tags/<TAG-SLUG>/executions

The Reflect API key should be set as a predefined environment variable rather than hard-coded into the .gitlab-ci.yml.

For more information, check out Gitlab’s online documentation.

Jenkins Pipeline

A request to the Reflect API can be trigger via Jenkins Pipeline’s http_request plugin.

pipeline {
  stages {
    stage('Regression Test') {
      steps {
        ...
        script {
          httpRequest(url: 'https://api.reflect.run/v1/tags/<TAG-SLUG>/executions', customHeaders: [[name: 'x-api-key', value: "${API_KEY}"]], requestBody: '{}')
        }
      }
    }
  }
}

For more information, please review the Jenkins Pipeline online documentation.

LayerCI

LayerCI creates ephemeral staging environments for every Pull Request, allowing you to execute Reflect tests on every PR.

To integrate Reflect with LayerCI, first add your Reflect API key as a SECRET ENV within your LayerCI account.

Next, add the following shell script to your repository:

#!/bin/bash
REQUEST_BODY="
{
  \"overrides\": {
    \"hostnames\": [{
      \"original\": \"qa.example.com\",
      \"replacement\": \"$EXPOSE_WEBSITE_URL\"
    }]
  }
}"

EXECUTION_ID=$(curl --location --silent --show-error --request POST 'https://api.reflect.run/v1/tags/all/executions' \
    --header "X-API-KEY: $REFLECT_API_KEY" \
    --header 'Content-Type: application/json' \
    --data-raw "$REQUEST_BODY" | jq -r '.executionId'
    )

echo "Running the tests... Execution id: $EXECUTION_ID"

STILL_RUNNING_TESTS=true
while [ "$STILL_RUNNING_TESTS" = "true" ]; do
    EXECUTION_STATUS=$( \
         curl --location --silent --show-error --request GET "https://api.reflect.run/v1/executions/$EXECUTION_ID" \
              --header "X-API-KEY: $REFLECT_API_KEY" \
              --header 'Content-Type: application/json' \
         )
    
    echo "Checking test status..."
    TESTS_FAILED=$(echo $EXECUTION_STATUS | jq -c '.tests[] | select(.status | contains("failed"))')
    STILL_RUNNING_TESTS=$(echo -n $EXECUTION_STATUS | jq -c '[ .tests[] | select(.status | contains("running") or contains("queued")) ] | length > 0')
   
    if ! [[ -z "$TESTS_FAILED" ]]; then
       printf "\e[1;31mSome tests has failed.\nReflect Execution ID: $EXECUTION_ID\nFailed tests: $TESTS_FAILED\n\n" >&2
       exit 1
    fi
    sleep 5
done

echo "All tests have completed"

Finally, access your Reflect API key from the Settings page and invoke the shell script in your Layerfile. (The example below assumes the file is named reflect-test.sh.)

...
SECRET ENV REFLECT_API_KEY
RUN bash reflect-test.sh

Travis CI

Kicking off Reflect tests after a successful deployment can be done by executing an API request in the after_deploy section of your .travis.yml file:

after_deploy:
  - "curl -X POST --data \"{}\" -H \"x-api-key: <API-KEY>\" \"https://api.reflect.run/v1/tags/<TAG-SLUG>/executions\""

Jenkins-X

A request to the Reflect API can be made in a Task:

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
  name: launch-reflect-tests
spec:
  params:
    - name: apiKey
      type: string
      description: Your Reflect API key
    - name: tag
      type: string
      description: The tag to run. The value should match the tag as it appears in the URL of the Tags page within Reflect.
  steps:
    - name: launch-reflect-tests
      image: curlimages/curl # Can be replaced with a different image so long as it has curl installed
      env:
        - name: REFLECT_API_KEY
          value: $(params.apiKey)
        - name: REFLECT_TAG
          value: $(params.tag)
      script: |
        echo "=== Triggering Reflect tests ==="
        echo "Tag=${REFLECT_TAG}"
        echo "Response:"
        curl --silent -X POST \
        -H "X-API-KEY: ${REFLECT_API_KEY}" \
        https://api.reflect.run/v1/tags/${REFLECT_TAG}/executions

That Task can then be incorporated into your release Pipeline:

apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
  name: release
spec:
  tasks:
    ...
    - name: launch-reflect-tests
      taskRef:
        name: launch-reflect-tests
      params:
        - name: apiKey
          value: your-api-key
        - name: tag
          value: tag-to-run

AWS CodeDeploy

Kicking off Reflect tests after a deployment can be done with a hook on the AfterAllowTraffic lifecycle event.

ECS/Lambda deployments

For ECS and Lambda deployments, you can use a Lambda function to trigger your Reflect tests. The Lambda function can be written in whatever language you choose, calling the Reflect API with your HTTP library of choice.

Once you have a function written (named TriggerReflectTests in this example), add the following to your AppSpec file:

Hooks:
  - AfterAllowTraffic: "TriggerReflectTests"

An example Lambda function for CodeDeploy lifecycle events can be found here.

Note: The Lambda function must also call PutLifecycleEventHookExecutionStatus before exiting in order to ensure the CodeDeploy deployment is marked as successful.

EC2/On-Premises deployments

EC2/On-Premises deployments can specify scripts to be run on the AfterAllowTraffic lifecycle event. To do that, add the following to your AppSpec file:

hooks:
  AfterAllowTraffic:
    - location: Scripts/PostDeploy.sh

Where Scripts/PostDeploy.sh contains:

#!/bin/sh

Tag="<my-tag>"
echo "=== Triggering Reflect tests ==="
echo "Tag=${Tag}"
echo "Response:"
curl --silent -X POST \
-H "X-API-KEY: ${REFLECT_API_KEY}" \
https://api.reflect.run/v1/tags/${Tag}/executions

Alternative Option: SNS Trigger

Rather than using a hook, you can use a Lambda function that is triggered by an SNS notification for your CodeDeploy deployment group on the ‘Success’ deployment event. An advantage to this approach is that you do not need to call PutLifecycleEventHookExecutionStatus in your Lambda function.

Documentation for creating an SNS trigger for a CodeDeploy deployment group can be found here.

Ready to accelerate your testing efforts?

Create comprehensive end-to-end tests for your web application, without writing a line of code.

Copyright © 2021 Reflect Software Inc. All Rights Reserved.