The data from this article is based on a six-question survey of 46 technical founders of YCombinator-funded companies. The raw results are included below along with commentary and additional stats (courtesy of Openbase).
Unit Testing
Our first two survey questions were about unit testing, specifically what we categorized as unit testing frameworks and test runners. This ended up being an imperfect categorization, as some frameworks like Jest take an all-in-one approach, whereas libraries like Chai really are better described as an assertion library. So in hindsight it’s hard to lump them all together.
Despite the lack of nuance in these questions, clear trends have emerged, foremost being that a majority of respondents prefers the all-in-one approach of Jest. Given it’s heritage as a library maintained by Facebook, as the test framework used by the React core team, and as the testing framework recommended in React’s own documentation, it’s no huge surprise that Jest is popular among React developers.
One surprising trend however is that over 34% of respondents are doing no unit testing at all! The survey was anonymous so we can’t correlate this answer to things like the size of an organization or age of company. And YCombinator-funded companies run the gamut from the largest privately-held tech startup in the US to companies that are pre-launch and pre-revenue. But as a broad statement, we can say that mandatory unit testing and, more strictly, TDD, are not a given at a tech startup.
Other Testing Utilities
Beyond unit testing frameworks and test runners, there are a number of other libraries that can assist in testing React applications. We asked YC founders to select which libraries, if any, they used from the list below:
The three most popular libraries among our respondents are:
- Enzyme - A utility library that makes it easier to test React component output.
- React Testing Library - An alternative to Enzyme that also makes it easier to test React component output.
- Sinon - A mocking library for Javascript.
An interesting takeaway here is that Enzyme is more popular with respondents despite React Testing Library (RTL) being recommended within React’s own documentation. Looking at their historical growth rate, RTL has enjoyed faster growth since it’s release but both have relatively the same usage and popularity as of this article.
Enzyme historical stats via Openbase
React Testing Library historical stats via Openbase
Five out of the seven respondents who use Sinon also use Jest, indicating that even though Jest has built-in support for mocks, stubs, and spies, several respondents find Sinon’s approach to mocking superior to Jest’s.
Snapshot Testing
Snapshot Tests are tests that render a React component, save its output to a snapshot file, and compare it to a known-good output from a previous run. The test fails if the two snapshots do not match. This type of testing is built into Jest, but is also supported in other libraries like Storybook. The practice is controversial, with proponents asserting that it provides coverage for errors that are hard to catch otherwise, whereas critics say that it can lead to noisy tests that fail due to innocuous changes in output, and not due to actual bugs.
Sentiment among respondents is mixed, with most respondents having heard of the practice but only 35% currently using it.
Visual Testing
Visual Testing is similar to Snapshot Testing, but instead of comparing DOM output, it instead compares the visual state of the component to a known-good baseline. The idea behind Visual Testing is that it (1) seeks to provide coverage for UI regressions that a typical test wouldn’t catch, and (2) is closer to testing how actual users perceive an app.
Visual Testing sees less use among YC founders compared to Snapshot Testing, with a fairly even split between the four possible answers.
End-to-end Testing
Finally we come to end-to-end testing, which is the practice of testing entire workflows and is meant to simulate how users actually use an application.
The big takeaway here is clear, the majority of respondents don’t do any end-to-end testing! And this is for a good reason - end-to-end tests are notoriously difficult to create and maintain. This is why we built special React testing features into Reflect, our no-code test automation product.
Here are a few ways Reflect is different from other end-to-end testing tools:
- There’s no coding required - Instead of writing code to simulate user actions, Reflect simply records you performing those actions yourself and translates them to a complete end-to-end test that you can run anytime. This drastically reduces the time it takes to create tests (we estimate a 10x - 100x time savings compared to conventional tools).
- Resilient selectors that use underlying components - Reflect is the only testing tool that can utilize the underlying React component names as selectors. This is important because CSS-in-JS frameworks like Styled Components make it hard to find stable selectors to target elements in a test since they inject random classes that can change on every build. By simply installing our open-source Webpack plugin, you provide Reflect with everything we need to extract the underlying component names for the elements you interact with over the course of a test, and automatically generate selectors that use those component names.
- Best in class test recorder - Because Reflect has complete control of the testing environment, we can do things that no other tool can, such as detect and exactly replicate complex interactions like drag-and-drops, hovers, and file uploads. For React apps, our recorder is resilient to changes synced via the Virtual DOM, which means no more flaky tests.
Interested in getting end-to-end test coverage for your React app? Sign up for a free Reflect account and get started in minutes.
