Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): Reduce e2e attempts from 3 to 2 in CircleCI #24531

Merged
merged 1 commit into from
May 31, 2024

Conversation

legobeat
Copy link
Contributor

@legobeat legobeat commented May 14, 2024

Description

This reduces:

  • E2E CI retries from 2 to 1 (ie attempts 2 times instead of 3 times)
  • The average cost and time to run individual CI jobs
  • Feedback loops for e2e errors
  • Success rate (and therefore hopefully introduction rate) of flaky tests
  • Success rate (and therefore hopefully introduction rate) of inconsistent application behavior which relies on specific CI conditions to pass E2E tests

Non-E2E CI jobs have their existing retry count unchanged.

Open in GitHub Codespaces

Related issues

Manual testing steps

Screenshots/Recordings

Before

After

Pre-merge author checklist

  • I’ve followed MetaMask Coding Standards.
  • I've completed the PR template to the best of my ability
  • I’ve included tests if applicable
  • I’ve documented my code using JSDoc format if applicable
  • I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Copy link
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@legobeat legobeat added area-CI team-security e2e-test End to end test for the MetaMask extension labels May 14, 2024
@legobeat legobeat marked this pull request as ready for review May 14, 2024 22:08
@legobeat legobeat requested review from kumavis and a team as code owners May 14, 2024 22:08
hjetpoluru
hjetpoluru previously approved these changes May 16, 2024
Copy link
Contributor

@hjetpoluru hjetpoluru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

seaona
seaona previously approved these changes May 23, 2024
@legobeat legobeat requested a review from hjetpoluru May 23, 2024 11:25
@legobeat legobeat force-pushed the e2e-decrease-retries branch 9 times, most recently from fa9d9c1 to fb946ff Compare May 23, 2024 19:13
@legobeat legobeat force-pushed the e2e-decrease-retries branch 12 times, most recently from 8508a6d to 7951645 Compare May 31, 2024 10:16
@legobeat legobeat force-pushed the e2e-decrease-retries branch 7 times, most recently from c02098b to 81a6632 Compare May 31, 2024 14:56
@legobeat legobeat merged commit 3983762 into MetaMask:develop May 31, 2024
77 checks passed
@github-actions github-actions bot locked and limited conversation to collaborators May 31, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-CI e2e-test End to end test for the MetaMask extension team-security
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

8 participants