Blog Series:

• Fixing Cypress errors part 1: chromium out of memory crashes
• Fixing Cypress errors part 3: miscellaneous and additional notes

Error: Random timeout issues due to flaky tests

This problem was almost as common as the chromium crashing error. Cypress E2E tests would randomly error and complain it cannot find certain elements. The weird thing is that most of the time, all tests pass, and when tests fail, it is different every time. Finally, rerunning the build often fixes them, to the point people who maintain the project told me this is fine.

Note: interestingly for me, the flaky tests stop after implementing the memory crashes mentioned in Part 1. So before you try any of these solutions, give that a try first.

Solution 1: Increase timeout time

Back in Selenium days, it was common to use commands like wait(10000) in between assertions to wait up to 10 seconds in case network was unstable or slow. It would reduce tests' flakiness in a lot of situations.

Cypress has a similar API cy.wait(500), but we don't want to use it because it is an anti-pattern as explained by Unnecessary Waiting - Cypress best practices.

Instead, we are looking at using a more elegant solution. Since Cypress already has a default timeout duration of 5000ms (5s), sometimes we may need to extend the duration slightly without interfering too much with what Cypress is doing.

We can do this by passing an extra argument to cy.get() API. Here is an example of how to do this:

cy.get('[data-testid="modal"]', { timeout: 10000 }).should('be.visible');

Of course, if you have timeouts all over the place as I mentioned above, then maybe, we need a more drastic approach and extend the global timeout.

The simplest way to do this is by using the defaultCommandTimeout flag or in this case, updating it in the cypress.json config file.

// cypress.json
{
  "baseUrl": "http://localhost:8080/",
  "defaultCommandTimeout": 10000,
}

Personally, this didn't help much with the errors I saw. They still appeared just as frequently after I made these changes.

Solution 2: Retrying failed tests

The alternative to increasing the timeout duration is to run the failed tests again. Based on what I saw as part of my investigation, a lot of people were recommending this approach as well.

The plugin is called cypress-plugin-retries, and it will do what the name suggests, rerun individual failed tests. After implementing this, the timeout errors all disappeared from the pipeline.

The plugin's README has clear instructions on how to implement it so that I won't repeat it here.

Final words

My conclusion is that solution 1 is worth trying if you always see the same tests failing due to assertion timeout. If the tests are failing randomly, I think solution 2 is a better bet. But as a colleague of mine correctly pointed out, these solution 2 doesn't remove the root problem in your application.

Lastly, if you are still having issues, maybe check out the Part 1 and Part 3 of this blog series.