Selenium Testing is Expensive

Selenium test suites are tedious to build and time-consuming to maintain, whether in code or Selenium IDE. Reflect is the fastest testing tool to build and maintain automated tests because it requires no programming and runs in the cloud.

As your software business grows, the cost of a bug or regression increases. Whether it's e-commerce sales or simply customer goodwill, broken software hurts your bottom line. Manually testing end-to-end simply doesn't scale, so you turned to Selenium WebDriver to implement an end-to-end testing suite in Python, Ruby or Java, and assigned an engineer or two to build it out. After all, it supports Chrome, IE and Firefox, it's free and open-source, and your software team is full of programmers. Perfect fit, right?

Unfortunately, the actual costs of using Selenium as a testing-framework to build and maintain an end-to-end test suite are not immediately apparent. Manual testers cannot contribute — it requires a developer; it has to change with every change in your product; and it has to run somewhere in your infrastructure. We love Selenium for automating web-browser tasks and cross-browser testing, but using it for end-to-end testing of a constantly evolving piece of software—such as typical modern web applications—is just too expensive.

Reflect is the end-to-end testing platform that runs at the speed of code without the time investment or maintenance burden of writing and maintaining code.

Why is end-to-end testing with Selenium so expensive?

Building Selenium tests seems great at first because any of the programmers building your software product can also write a Selenium test. However, if you had a choice of having those developers build new features or write Selenium tests, you'd always choose the former, right? But ignoring the opportunity cost of assigning developers to build Selenium tests, there are other challenges:

It's a whole second codebase

Every line of code written is a line of code that has to be maintained. It's challenging enough keeping your product's codebase clean, understandable and efficient, so why would you want to double the task by adding more code to the repository? Additionally, there might be a tendency for developers to be less interested in following best practices for their test code because, well, it's not the product! Writing test code can feel like important work (even if that's the wrong mentality).

Coded tests are too fragile

Code is a precise set of instructions, and to write code that handles ambiguity requires more code. So, your Selenium UI tests are generally going to be very specific to the current state of your webapp. This makes them fragile to just about any change in the UI or functionality. Consider this Selenium Java snippet:


  // Run the test

  WebElement booksLink = driver.findElement(By.cssSelector("#navigation > div:nth-child(3) > a"));;

  WebElement booksHeader = driver.findElement(By.cssSelector("h1"));
  assert(booksHeader.getText() == "Books Section");

This test will break if:

Obviously, a developer can update the test for any of those changes, and if the header text changes that's probably a good test failure to have, but should they have to update the test for the others? Ideally, your test would only depend on properties of your site that are required to be present. It's possible that you want to assert that the link is an A tag and not a styled DIV, but it's probably more likely you just want to click on the navigation link with "Books" in the text. Expressing that in code would require a lot more work.

No support for visual testing

Think about the bugs you've found in your app. Chances are that some of these bugs are purely visual problems. Things like modals being misplaced, elements getting the wrong styling, or elements on the page getting shifted or obscured. Conventional end-to-end test automation tools don't allow you to find these bugs because they operate on DOM elements and are unable to detect visual changes in your app.

Reflect is the efficient investment into end-to-end testing

It seemed like no matter how much time we spent perfecting our Selenium test suite, we always had to invest more whenever our webapp changed. Our insight was that it's quick and easy to perform a manual test but costly to codify it. So, Reflect's only input is just that: a manual test. From there, Reflect automatically extracts the user's intentions about what to test, and countless signals about how to execute it. Your only investment into Reflect, then, is knowing what you want to test and this makes it different from other automation tools. Here are some of the reasons why the best Selenium alternative is Reflect.

No infrastructure to maintain

Reflect is a fully managed solution that requires nothing other than a URL to create a test. Instead of configuring machines, installing and updating drivers, or checking on cron scripts in your selenium grid or jenkins, you just point and click through your webapp to build up your Reflect tests. But more importantly, you get all of the powerful features you would expect from a fully managed platform, including:

Isn't that better than losing an afternoon upgrading libraries on your grid of test machines?

Algorithmically driven element selection

To specifically address the challenge of targeting the correct element outlined in the Selenium snippet above, Reflect employs a custom algorithm to capture numerous signals about the target element, which means you'll never have to produce your own xpath or write custom JavaScript. Furthermore, Reflect captures multiple selectors for identifying an element, which means you'll see a list like the following for every interaction:

    nav.header-nav > #site-navigation > a:nth-of-type(3)
    nav > #site-navigation > a:nth-of-type(3)
    nav > div:nth-of-type(2) > a:nth-of-type(3)
    header #site-navigation > a:nth-of-type(3)
    nav:nth-of-type(2) > div:nth-of-type(2) > a:nth-of-type(1)

But, most importantly, Reflect can find elements by any visible text or keyword, including dynamic text produced at runtime. So, if all you want to do is click on the link with the text, "Books", then just click on the link with the text, "Books" :)

Visual testing as a first-class feature

Selenium has no built-in visual testing capability, which means you have to forego visual testing entirely or spend more time configuring a third-party plugin such as Appium. With Reflect, Visual Regression Testing is a first-class feature, which includes support for updating visual assertions in a single click when your webapp changes. This lets you detect true visual regressions in the scope of your test without getting notified about innocuous changes elsewhere. Furthermore, Reflect supports mobile and tablet emulation so you can confirm your site's visuals on different devices and meet your acceptance testing requirements without manual inspection every time.

Advanced features

If you're coming from Selenium, then you might be concerned that moving to a no-code platform would force you to give up the power of code reuse and deduplication in your test scripts. Reflect supports Test Composition to reuse test snippets in other tests, and Re-recording & Editing so that you never have to start from scratch to fix up a test.

Get started with Reflect today

Create your first test in 2 minutes, no installation or setup required. Accelerate your testing efforts with fast and maintainable test suites without writing a line of code.

Copyright © Reflect Software Inc. All Rights Reserved.