Agile and CI/CD
Continuous delivery is the software engineering practice of releasing, or deploying, software changes to the production system immediately after they are reviewed and deemed ready. A related concept is to perform the review of the changes quickly to increase both application development release frequency and the efficiency of confirming that the changes are correct post-deployment. The goal is to create a tight feedback loop between developers and the state of the production system. These short cycles of collaboration reduce bugs that are introduced due to context-switching or unfamiliarity, and establish a means for continuous testing. Additionally, it forces the system infrastructure itself to provably maintain its flexibility in response to new features, changing usage, errors and rollbacks.
Continuous delivery falls into the later stages of Agile software development, and is often coupled with continuous integration in the software development lifecycle. See our other article on Continuous Integration in Agile Development for more information about these two concepts that are often implemented together.
Modern continuous delivery
One traditional software development approach is the “waterfall” methodology, wherein the development and release of software is broken into different phases of requirements gathering, design, development and testing or bug fixes; and each is distinct and irreversible. This approach is better suited to building software for a fixed, well-known environment.
Conversely, agile software development focuses on iterative development, relying on self-directed, highly communicative, cross-functional teams. These teams rapidly build new software by blurring the lines between design and development, and aim to release many small changes quickly rather than very large changes slowly. It’s within the context of the agile software development life cycle that modern continuous delivery fulfills an integral role.
A key characteristic of continuous delivery is that it be automated. The benefits of continuous delivery in this sense are that it reduces human error during the deployment process, and importantly, increases development velocity because new changes won’t sit idle waiting to be released. The automation acts as a forcing function to ensure software quality, and introduces testing automation into your deployment pipeline. After all, if changes will go live instantly after being reviewed, then the developer will take care to ensure that the software changes are correct.
Let’s consider an example software development life cycle to see where the CD pipeline and end-to-end testing fit in. The planning phase comprises requirements gathering and design. Next, software developers implement the design and write unit tests. (If they practice test-driven development, then the unit tests would come first.) After that, the code changes are proposed and submitted to code review by the development team. If the changes pass review, they are merged into the official code repository and deployed to a staging environment. End-to-end testing is performed in the staging environment with the proposed changes, and if all of the tests pass, the code in the staging environment is released to the live production system. Finally, a post-deployment test pass might be performed to ensure no hidden problems.
Ideally, in Agile development, everything after the code review stage above is automated. However, end-to-end tests are notoriously difficult to automate, and so the majority of end-to-end testing is still performed manually. We’ll focus on the challenges of end-to-end testing in continuous delivery and how Reflect meets those challenges a little later in this article. First, let’s review how you can deploy your code changes to staging and production environments as a part of your continuous delivery pipeline.
Staging environments with continuous delivery
A staging environment is a running software system that mimics the functionality of a production system for the purposes of testing code changes and gathering user feedback. Generally speaking, a staging environment costs less money than the production environment to run because there is less data and less user traffic. The data itself may be a subset of production data, or total synthetic data for the purposes of testing. Some teams will only run a staging environment while it’s in use to save money. Staging environments are a common feature of Agile software development teams, and given that they are an exact replica of the production system, serve as the perfect place for end-to-end testing before you release software.
Maintaining a staging environment was historically a lot of work because, well, it’s an entire second production-like system! But with the increased focus on DevOps build automation and its role in Agile development, modern tooling makes managing staging environments easy because it’s just a second instantiation of the automated infrastructure that already powers the production system.
The main difference between your staging and production environments is the URL.
So, if you access your application at https://app.reflect.run, for example,
then you might access your staging environment (with all of the same functionality)
at https://staging.reflect.run/.
Some teams might also restrict access to the staging environment to users on a VPN.
So, assuming your DevOps scripts handle instantiating your production environment, what tools exist to automate the instantiation of your staging environment?
1. Heroku
Heroku’s Review Apps integrate with your GitHub account to run a complete instantiation of your Heroku App with the code changes from a pull request. They are meant to be lightweight and disposable, and primarily used for testing your code changes in a live environment. Each staging environment comes with a unique URL, and like each of these on-demand staging environment tools, your application must be defined and configured in code.
2. Release
Release instantiates your application from a docker-compose file. Their primary use case is on-demand staging environments for testing, but their platform can also manage your production deployment and environment as well. Similar to Heroku, they require a GitHub integration to detect code changes and automatically provision a new environment with your latest code. They expose a general purpose computing environment so you can take custom actions/triggers (e.g., external API requests) throughout deployment.
3. LayerCI
LayerCI provides an end-to-end testing environment on every commit to GitHub. Similar to Release, it provisions the environment automatically based on triggers in your code, and it can pre-populate test data in the environment to make it easier to perform end-to-end testing. LayerCI’s focus is on speed in instantiating the staging environments in order to reduce the time it takes to perform end-to-end testing.
End-to-end testing in continuous delivery with Reflect
At this point, it’s clear that DevOps in Agile development has evolved and end-to-end testing in a throwaway fashion has become commonplace. But recall our goal from earlier:
ideally, everything after code review is automated.
This poses a challenge to our modern Agile development process: we’ve automated everything up to the point of end-to-end testing, but how do we automate that stage? For that, we use Reflect.
Reflect turns manual end-to-end tests into automated end-to-end tests, and it does so without requiring the developer or QA tester to write any code. Furthermore, it requires no installation and runs entirely in the cloud. Let’s take a look at how you can use any of the platforms above to execute end-to-end tests against a staging environment.
One of the primary components of end-to-end testing in staging is regression testing. The goal of regression testing is to confirm that existing functionality still works even after the new code changes are incorporated and released. Anytime you change one area of a codebase, you run the risk of breaking some other area unintentionally. As a result, your end-to-end testing comprises a lot of individual regression tests for different functionality in your application.
Suppose you’ve implemented and locally tested some new code changes. First, you push it up for review and while you’re waiting for code review, your staging environment provider automatically deploys the changes to a new ephemeral environment. Next, you record a new Reflect test using the unique URL where the staging environment lives. Once you’ve created your test and it runs successfully, this test becomes a part of your overall regression test suite, which will be run to confirm the functionality continues to work with any future change.
After that, your team members approve your changes and they are merged by your version control system into the main repository. The staging environment provider detects the updated codebase and automatically deploys the changes to your primary (longstanding) staging environment. Using Reflect’s hostname overrides, you can run the exact same test you just recorded against your ephemeral environment against your main staging environment which now has your changes and any other proposed changes. Additionally, you run the tests in your “regression” Folder to run all of your other, pre-existing regression tests.
Assuming the tests pass there, your CI/CD automation can deploy your changes (and any others) live to production. At that point, you might run a third batch of Reflect tests against the production site using hostname overrides again. Each of these phases can be run manually, of course, but the real power of Reflect is in triggering these tests through our API, as we integrate with and provide resources for just about any CI/CD platform.
With this approach, your Reflect regression test suite grows over time, and continues to stay up-to-date through the continuous deployment with every change.
Conclusion
The Agile and DevOps movements champion rapid iteration and automated procedures throughout the software development lifecycle. Modern tooling for ephemeral staging environments means that automated end-to-end testing can finally be realized in your CD process. Running at the speed of code with the ease and flexibility of manual testing, Reflect makes end-to-end tests fast to create and easy to maintain.
