Skip to content

bZichett/jest-retry

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

53 Commits
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Jeust Retry

Motivation

E2E test failures due to reasons outside the domain of source code, including:

  1. Network timeouts
  2. Flaky endpoints etc.
  3. Login issues
  4. Flaky selenium selectors (bad code which is hard to diagnose)
  5. Whatever other mishaps, etc.

Setup

Create a run-config to pass into jest-retry

Example run

require('jest-retry')({
    debug: false,
    testDir: 'src/tests',
    testFilter: 'table-tests',
    // jest-junit is currently the only option
    outputTestResults: true,
    testResultsOutput: '.',

    /* Test retry options */
    flakyTestMock: false,
    flakyTestMockDir: 'src/mocks',
    flakyFailureMessages: ['Network timeout'],
    
    // Matches to test failures `testFilePath` 
    // These are still marked as fail, but does not contribute to jest exiting with failure status
    knownIssues: ['src/test/known-issue.test'],
    
    flakyNumRetries: 2, // 0 for disable retry pattern or Infinity for continue running until getting same results
       
    flakyMarkAll: false,
    flakyWaitBeforeRerun: 1000,
    // Jest options
    setupTestFrameworkScriptFile: 'jest-config.js',
    maxWorkers: 1,
    noStackTrace: false,
    silent: false,
});

Test

npm run test runs a short 2 test suite in which a failure occurs and is forced to pass on the second retry

Test output

☁  jest-retry [master] npm run test

> [email protected] test /code/jest-retry
> ./test/run.js

 PASS  test/always-pass.test.js
 FAIL  test/conditional-fail.test.js
  ● mocks/conditional-fail › conditionally fails

    expect(received).toBe(expected) // Object.is equality

    Expected value to be:
      false
    Received:
      true

      1 | describe('mocks/conditional-fail', async () => {
      2 |   it('conditionally fails', async () => {
    > 3 |     if (!process.env.SKIP) expect(true).toBe(false);
      4 |   });
      5 | });
      6 |

      at Object.it (test/conditional-fail.test.js:3:41)

Test Suites: 1 failed, 1 passed, 2 total
Tests:       1 failed, 1 passed, 2 total
Snapshots:   0 total
Time:        0.28s, estimated 1s
Ran all test suites.

flakyDictionaryCount: {
  "Expected value to be": 1
}

Retrying the following test suites:  [
  "/code/jest-retry/test/conditional-fail.test.js"
]
===============================================

 PASS  test/conditional-fail.test.js
  mocks/conditional-fail
    ✓ conditionally fails

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        0.099s, estimated 1s
Ran all test suites.

All failures have now passed

Test retries result: true