How YC Companies Test React Apps

November 9th, 2020

React has a great ecosystem not just for making development easier, but for making testing easier as well. We surveyed YCombinator founders to understand what libraries, frameworks, and methodologies they use to test their React applications.

The data from this article is based on a six-question survey of 46 technical founders of YCombinator-funded companies. The raw results are included below along with commentary and additional stats (courtesy of Openbase).

Unit Testing

Our first two survey questions were about unit testing, specifically what we categorized as unit testing frameworks and test runners. This ended up being an imperfect categorization, as some frameworks like Jest take an all-in-one approach, whereas libraries like Chai really are better described as an assertion library. So in hindsight it’s hard to lump them all together.

Despite the lack of nuance in these questions, clear trends have emerged, foremost being that a majority of respondents prefers the all-in-one approach of Jest. Given it’s heritage as a library maintained by Facebook, as the test framework used by the React core team, and as the testing framework recommended in React’s own documentation, it’s no huge surprise that Jest is popular among React developers.

One surprising trend however is that over 34% of respondents are doing no unit testing at all! The survey was anonymous so we can’t correlate this answer to things like the size of an organization or age of company. And YCombinator-funded companies run the gamut from the largest privately-held tech startup in the US to companies that are pre-launch and pre-revenue. But as a broad statement, we can say that mandatory unit testing and, more strictly, TDD, are not a given at a tech startup.

Other Testing Utilities

Beyond unit testing frameworks and test runners, there are a number of other libraries that can assist in testing React applications. We asked YC founders to select which libraries, if any, they used from the list below:

The three most popular libraries among our respondents are:

  1. Enzyme - A utility library that makes it easier to test React component output.
  2. React Testing Library - An alternative to Enzyme that also makes it easier to test React component output.
  3. Sinon - A mocking library for Javascript.

An interesting takeaway here is that Enzyme is more popular with respondents despite React Testing Library (RTL) being recommended within React’s own documentation. Looking at their historical growth rate, RTL has enjoyed faster growth since it’s release but both have relatively the same usage and popularity as of this article.

Enzyme historical stats via Openbase

Enzyme historical stats via Openbase

React Testing Library historical stats via Openbase

React Testing Library historical stats via Openbase

Five out of the seven respondents who use Sinon also use Jest, indicating that even though Jest has built-in support for mocks, stubs, and spies, several respondents find Sinon’s approach to mocking superior to Jest’s.

Snapshot Testing

Snapshot Tests are tests that render a React component, save its output to a snapshot file, and compare it to a known-good output from a previous run. The test fails if the two snapshots do not match. This type of testing is built into Jest, but is also supported in other libraries like Storybook. The practice is controversial, with proponents asserting that it provides coverage for errors that are hard to catch otherwise, whereas critics say that it can lead to noisy tests that fail due to innocuous changes in output, and not due to actual bugs.

Sentiment among respondents is mixed, with most respondents having heard of the practice but only 35% currently using it.

Visual Testing

Visual Testing is similar to Snapshot Testing, but instead of comparing DOM output, it instead compares the visual state of the component to a known-good baseline. The idea behind Visual Testing is that it (1) seeks to provide coverage for UI regressions that a typical test wouldn’t catch, and (2) is closer to testing how actual users perceive an app.

Visual Testing sees less use among YC founders compared to Snapshot Testing, with a fairly even split between the four possible answers.

End-to-end Testing

Finally we come to end-to-end testing, which is the practice of testing entire workflows and is meant to simulate how users actually use an application.

The big takeaway here is clear, the majority of respondents don’t do any end-to-end testing! And this is for a good reason - end-to-end tests are notoriously difficult to create and maintain. This is why we built special React testing features into Reflect, our no-code test automation product.

Here are a few ways Reflect is different from other end-to-end testing tools:

Interested in getting end-to-end test coverage for your React app? Sign up for a free Reflect account and get started in minutes.

Copyright © 2020 Reflect Software Inc. All Rights Reserved.