Testing helps prevent customers from hitting bugs. We all know that. But why is the process of testing web apps so inefficient? If you were building an automated test suite today, the norm is to use a code-based tool. But code-based testing has been the norm for the past 15 years (!), and there are still some very fundamental problems that make them difficult to use:
1. Translating a user’s actions into code is a painstaking, time-consuming process.
Just think of a simple workflow in your app, like signing up for a new account. There are probably 20 or 30 separate steps involved. Translating each click and form field interaction into code is not something that most developers would consider the highlight of their day.
2. These tests break all of the time.
Any change you make to your app is at risk of breaking these tests. That’s because the typical approach is to use a single CSS selector or XPath to target each element in a test. Hidden dependencies on the structure and styling of the page is one major cause for tests failing even when no bug is present. Having to fix up tests all the time becomes a huge distraction and causes test suites to quickly fall out of use.
3. Existing tools have major limitations that prevent you from testing common workflows or catching common types of bugs.
Despite many person-years of development, the best-in-class code-based testing tools still have major limitations. Issues include the inability to test workflows that span multiple domains, poor support for common actions like drag-and-drop or file-uploading. and no facilities for targeting more modern web building blocks like Web Components and Shadow DOM. Beyond these issues, most popular tools cannot detect visual regressions: such as a button that’s the wrong color, or a modal that’s obscured. This means even if you had 100% test coverage, you’re still going to have a whole class of failures that you’re not going to catch. And, if you want coverage for visual issues, you’ll need to use a third-party tool that costs extra.
Our Approach
We’ve built Reflect to address these issues. With Reflect, you don’t write code to create tests. Instead, Reflect spins up a browser in the cloud and records your actions as you use your site. Those actions are automatically translated into tests. We believe the browser is the best interface for creating E2E tests, not your IDE. For one thing, it’s way faster. Workflows that would take half a day to write in a code-based tool take just a few minutes to build in Reflect. Since end-to-end tests replicate user behavior, any workflow that you’d want to test, by definition can be done in the browser directly. So in a sense we’re just cutting out the middle-man :)
We’ve also built what we believe is the best way of maintaining these tests: an online debugger for your tests. You can fast-forward into any part of a test and add additional steps, tweak settings, and watch the test run live to ensure it’s repeatable. If an issue is found with your test changes, you can take steps to fix it immediately. We even provide a set of recommendations for how to fix up the issue. And if you’re a developer fearing the loss of code reuse in a no-code tool, we have that covered too, with reusable sub-tests called Segments.
Reflect is a “batteries included” solution in that the infrastructure for running the tests is provided as part of our service. You can choose to schedule tests (e.g. run them daily at noon), run them on-demand, or hook it up to your CI/CD platform to kick tests off after a successful deployment. We also integrate with platforms like Vercel and Heroku so that you can use Preview Environments to run tests on every Pull Request.
Thanks for reading, and happy testing!
The Reflect Team