Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(webhook,work-pool): example for jsonencoding file input #350

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

parkedwards
Copy link
Contributor

@parkedwards parkedwards commented Dec 27, 2024

resolves #259

in #340, @mitchnielsen updated our baseJobTemplate handling to rely on the json.NormalizedType semantic equality checking. By definition, this type should do the following:

Semantic equality logic is defined for Normalized such that inconsequential differences between JSON strings are ignored (whitespace, property order, etc). If you need strict, byte-for-byte, string equality, consider using ExactType.

however, there is an edge case with file() + resource imports, as our provider logic's equality checking only kicks in after the provider framework computes plan diffs (diff check = compare HCL to State object). Therefore, when importing a resource + using file(), you will still get plan diffs from inconsequential whitespace changes:

➜ terraform import prefect_work_pool.import c63d708d-b4d7-49a7-92b1-e067e65c99d7,import-me
prefect_work_pool.import: Importing from ID "c63d708d-b4d7-49a7-92b1-e067e65c99d7,import-me"...
prefect_work_pool.import: Import prepared!
  Prepared prefect_work_pool for import
prefect_work_pool.import: Refreshing state...

Import successful!

The resources that were imported are shown above. These resources are now in
your Terraform state and will henceforth be managed by Terraform.

➜ terraform plan
prefect_work_pool.import: Refreshing state... [id=7ccace8a-ea08-405b-8625-c3a67156f140]

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # prefect_work_pool.import will be updated in-place
  ~ resource "prefect_work_pool" "import" {
      ~ base_job_template = jsonencode( # whitespace changes
            {
                job_configuration = {
                    cluster_config            = "{{ cluster_config }}"
                    command                   = "{{ command }}"

# ...

when we use file(), the framework interprets this input as raw, non-normalized byte strings. so when we import a resource:

  1. TF will Read() the existing API object, save the value to State - this includes the base_job_template string for example as a json.NormalizedType
  2. Subsequent terraform plan will read our HCL, which uses file() - this is pulled in as the raw byte string, without any knowledge that this is normalized JSON
  3. The framework will detect a plan diff before stepping into our provider logic, where the json.NormalizedType semantic equality check can be performed and deemed a no-op

This diff behavior is better explained by this comment in the TF core repo, stating that file() inputs can only be evaluated via exact character matching due to how different OS-es handled file encodings

hashicorp/terraform#23928 (comment)

If we want to be able to treat file() as semantically equal to their JSON representations in a remote environment, there's a few ways to do this via PlanModifiers. But, I'd like to first expand our documented cases to suggest wrapping file() with a jsonencode(jsondecode(file(...))) chain, which achieves this same effect for our users. This is a pattern surfaced in one of the AWS providers

hashicorp/terraform-provider-aws#25340 (comment)

Essentially, we're coercing the file input to be normalized JSON via jsonencode() -- this will allow our underlying type system to do what we'd expect moving forward, and result in a no-op after an import

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Imported prefect_work_pool's base_job_template is overwritten on initial plan/apply when it's the same
1 participant