Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests: overhaul to update to 1.10.x patterns and enforce better consistency #349

Open
parkedwards opened this issue Dec 26, 2024 · 0 comments
Assignees

Comments

@parkedwards
Copy link
Contributor

parkedwards commented Dec 26, 2024

too often, I find that I'm re-creating acceptance test suites from scratch, so the lead time to do any sort of TDD becomes too long. also this leads to test cases being a bit different from file to file, and then debugging failed tests becomes an archaeological dig. my hypothesis is that this is because we have multiple ways to write tests / fixtures / helpers, and we've also probably got some repeated code that is causing authors to have to reinvent the wheel repeatedly.

since we have a good chunk of test coverage now, I propose that we're able to do a little refactoring to do the following:

  1. Starting 1.10.x, the testing framework marked the currently used Check func that we use as "Legacy" + indicates that we should use the new ConfigStateChecks attribute as well as State Check implementations
  2. Explore Plan Checks / State Checks
  3. implement standardized Import checks (where necessary) - may depend on standardize import parameters schema #335
  4. standardize test case formats - naming
  5. standardize fixture instantiation - we have a couple of cool methods using go templating thanks to @mitchnielsen , so there should be a way to make the naming and patterns consistent (eg. when do we create new resources for update assertions?)
  6. standardize and share custom test checks - we have a couple of custom check functions mainly used to verify if the remote API has the resource object created in the TF test. These are pretty repetitive and basically re-implement Get() logic. We should look to standardize these common ones into shared helpers, but also find a way to codify a pattern so that test authors know exactly how/when to write a new custom check function
  7. Add in a TF version check
  8. Explore sweepers
@parkedwards parkedwards self-assigned this Dec 26, 2024
@parkedwards parkedwards changed the title tests: refactor so that we have consistent fixtures / test utils tests: overhaul to update to 1.11.0 patterns and enforce better consistency Dec 26, 2024
@parkedwards parkedwards changed the title tests: overhaul to update to 1.11.0 patterns and enforce better consistency tests: overhaul to update to 1.10.x patterns and enforce better consistency Dec 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant