Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Terraform crashes when updating a spinnaker_project in-place. Destroying a project fails with Error: 500 #112

Open
wwalters12 opened this issue Mar 11, 2021 · 2 comments
Open

Terraform crashes when updating a spinnaker_project in-place. Destroying a project fails with Error: 500 #112

wwalters12 opened this issue Mar 11, 2021 · 2 comments

Comments

@wwalters12
Copy link

wwalters12 commented Mar 11, 2021 •
edited
Loading

WHAT

crash.log

Creating a spinnaker_project with terraform-provider-spinnaker is working, but managing that project afterwards is failing.

When trying to update a project in-place, terraform crashes:

Terraform will perform the following actions:

  # spinnaker_project.projects["corporate"] will be updated in-place
  ~ resource "spinnaker_project" "projects" {
      ~ email = "[email protected]" -> "[email protected]"
        id    = "corporate"
        name  = "corporate"
    }

  # spinnaker_project.projects["customercare"] will be updated in-place
  ~ resource "spinnaker_project" "projects" {
      ~ email = "[email protected]" -> "[email protected]"
        id    = "customercare"
        name  = "customercare"
    }

Plan: 0 to add, 2 to change, 0 to destroy.

----------------------------------------

spinnaker_project.projects["customercare"]: Modifying... [id=customercare]
spinnaker_project.projects["corporate"]: Modifying... [id=corporate]

Error: rpc error: code = Unavailable desc = transport is closing

Error: rpc error: code = Unavailable desc = transport is closing

panic: interface conversion: interface {} is nil, not int
2021-03-11T13:51:23.041Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0: 
2021-03-11T13:51:23.041Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0: goroutine 58 [running]:
2021-03-11T13:51:23.041Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0: github.com/mercari/terraform-provider-spinnaker/spinnaker/api.NewCreateApplicationTask(0xc000654c80, 0x49fa6b, 0xf9dfa0, 0xc0006fb100)
2021-03-11T13:51:23.041Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0:    github.com/mercari/terraform-provider-spinnaker/spinnaker/api/application.go:53 +0x11a5
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0: github.com/mercari/terraform-provider-spinnaker/spinnaker.resourceSpinnakerProjectUpdate(0xf9e020, 0xc000659020, 0xc000654c80, 0xd9ae20, 0xc00018e070, 0xc000397bd0, 0xacc86a, 0xc000331100)
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0:    github.com/mercari/terraform-provider-spinnaker/spinnaker/resource_project.go:128 +0x45
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).update(0xc000181380, 0xf9dfa0, 0xc0006fb100, 0xc000654c80, 0xd9ae20, 0xc00018e070, 0x0, 0x0, 0x0)
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0:    github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:309 +0x1ea
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*Resource).Apply(0xc000181380, 0xf9dfa0, 0xc0006fb100, 0xc0000b1570, 0xc000331100, 0xd9ae20, 0xc00018e070, 0x0, 0x0, 0x0, ...)
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0:    github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/resource.go:404 +0x390
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0: github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).ApplyResourceChange(0xc00029eda0, 0xf9dfa0, 0xc0006fb100, 0xc000666410, 0xc0006fb100, 0x0, 0xfa8f40)
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0:    github.com/hashicorp/terraform-plugin-sdk/[email protected]/helper/schema/grpc_provider.go:955 +0x8cf
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0: github.com/hashicorp/terraform-plugin-go/tfprotov5/server.(*server).ApplyResourceChange(0xc000653e00, 0xf9dfa0, 0xc0006fb100, 0xc0000b13b0, 0xc000653e00, 0xc00065eea0, 0xc0004dbba0)
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0:    github.com/hashicorp/[email protected]/tfprotov5/server/server.go:331 +0xae
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0: github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_ApplyResourceChange_Handler(0xe37fe0, 0xc000653e00, 0xf9e060, 0xc00065eea0, 0xc000658480, 0x0, 0xf9e060, 0xc00065eea0, 0xc0006b0d20, 0xef)
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0:    github.com/hashicorp/[email protected]/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:380 +0x214
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0: google.golang.org/grpc.(*Server).processUnaryRPC(0xc0001fea80, 0xfa6dc0, 0xc0004ee600, 0xc00046cf00, 0xc000080c00, 0x146c8a0, 0x0, 0x0, 0x0)
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0:    google.golang.org/[email protected]/server.go:1210 +0x522
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0: google.golang.org/grpc.(*Server).handleStream(0xc0001fea80, 0xfa6dc0, 0xc0004ee600, 0xc00046cf00, 0x0)
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0:    google.golang.org/[email protected]/server.go:1533 +0xd05
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0: google.golang.org/grpc.(*Server).serveStreams.func1.2(0xc000036050, 0xc0001fea80, 0xfa6dc0, 0xc0004ee600, 0xc00046cf00)
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0:    google.golang.org/[email protected]/server.go:871 +0xa5
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0: created by google.golang.org/grpc.(*Server).serveStreams.func1
2021-03-11T13:51:23.042Z [DEBUG] plugin.terraform-provider-spinnaker_v0.3.0:    google.golang.org/[email protected]/server.go:869 +0x1fd
2021/03/11 13:51:23 [DEBUG] spinnaker_project.projects["corporate"]: apply errored, but we're indicating that via the Error pointer rather than returning it: rpc error: code = Unavailable desc = transport is closing
2021/03/11 13:51:23 [TRACE] EvalWriteState: recording 0 dependencies for spinnaker_project.projects["corporate"]
2021/03/11 13:51:23 [TRACE] EvalWriteState: writing current state object for spinnaker_project.projects["corporate"]
2021-03-11T13:51:23.045Z [WARN]  plugin.stdio: received EOF, stopping recv loop: err="rpc error: code = Unavailable desc = transport is closing"
2021/03/11 13:51:23 [TRACE] EvalApplyProvisioners: spinnaker_project.projects["corporate"] is not freshly-created, so no provisioning is required
2021/03/11 13:51:23 [TRACE] EvalWriteState: recording 0 dependencies for spinnaker_project.projects["corporate"]
2021/03/11 13:51:23 [TRACE] EvalWriteState: writing current state object for spinnaker_project.projects["corporate"]
2021/03/11 13:51:23 [TRACE] vertex "spinnaker_project.projects[\"corporate\"]": visit complete
2021/03/11 13:51:23 [DEBUG] spinnaker_project.projects["customercare"]: apply errored, but we're indicating that via the Error pointer rather than returning it: rpc error: code = Unavailable desc = transport is closing
2021/03/11 13:51:23 [TRACE] EvalWriteState: recording 0 dependencies for spinnaker_project.projects["customercare"]
2021/03/11 13:51:23 [TRACE] EvalWriteState: writing current state object for spinnaker_project.projects["customercare"]
2021/03/11 13:51:23 [TRACE] EvalApplyProvisioners: spinnaker_project.projects["customercare"] is not freshly-created, so no provisioning is required
2021/03/11 13:51:23 [TRACE] EvalWriteState: recording 0 dependencies for spinnaker_project.projects["customercare"]
2021/03/11 13:51:23 [TRACE] EvalWriteState: writing current state object for spinnaker_project.projects["customercare"]
2021/03/11 13:51:23 [TRACE] vertex "spinnaker_project.projects[\"customercare\"]": visit complete
2021/03/11 13:51:23 [TRACE] dag/walk: upstream of "meta.count-boundary (EachMode fixup)" errored, so skipping
2021/03/11 13:51:23 [TRACE] dag/walk: upstream of "provider[\"registry.terraform.io/mercari/spinnaker\"] (close)" errored, so skipping
2021/03/11 13:51:23 [TRACE] dag/walk: upstream of "root" errored, so skipping
2021-03-11T13:51:23.048Z [DEBUG] plugin: plugin process exited: path=.terraform/providers/registry.terraform.io/mercari/spinnaker/0.3.0/linux_amd64/terraform-provider-spinnaker_v0.3.0 pid=953 error="exit status 2"
2021/03/11 13:51:23 [TRACE] statemgr.Filesystem: creating backup snapshot at terraform.tfstate.backup
2021/03/11 13:51:23 [TRACE] statemgr.Filesystem: state has changed since last snapshot, so incrementing serial to 52
2021/03/11 13:51:23 [TRACE] statemgr.Filesystem: writing snapshot at terraform.tfstate
2021/03/11 13:51:23 [TRACE] statemgr.Filesystem: removing lock metadata file .terraform.tfstate.lock.info
2021/03/11 13:51:23 [TRACE] statemgr.Filesystem: unlocking terraform.tfstate using fcntl flock
2021-03-11T13:51:23.078Z [DEBUG] plugin: plugin exited



!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Terraform crashed! This is always indicative of a bug within Terraform.
A crash log has been placed at "crash.log" relative to your current
working directory. It would be immensely helpful if you could please
report the crash with Terraform[1] so that we can fix this.

When reporting bugs, please include your terraform version. That
information is available on the first line of crash.log. You can also
get it by running 'terraform --version' on the command line.

SECURITY WARNING: the "crash.log" file that was created may contain 
sensitive information that must be redacted before it is safe to share 
on the issue tracker.

[1]: https://github.com/hashicorp/terraform/issues

!!!!!!!!!!!!!!!!!!!!!!!!!!! TERRAFORM CRASH !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Also, destroying a project fails with Error: 500:

Terraform will perform the following actions:

  # spinnaker_project.projects["corporate"] will be destroyed
  - resource "spinnaker_project" "projects" {
      - email = "[email protected]" -> null
      - id    = "corporate" -> null
      - name  = "corporate" -> null
    }

  # spinnaker_project.projects["customercare"] will be destroyed
  - resource "spinnaker_project" "projects" {
      - email = "[email protected]" -> null
      - id    = "customercare" -> null
      - name  = "customercare" -> null
    }

Plan: 0 to add, 0 to change, 2 to destroy.

-----------------------------------------

spinnaker_project.projects["customercare"]: Destroying... [id=customercare]
spinnaker_project.projects["corporate"]: Destroying... [id=corporate]

Error: 500 

Error: 500

WHY

We discovered this issue after trying to update a project with new applications after it was created. Without projects, we find that it's difficult to discern which applications belong to a particular development squad in the UI.

This is an example of the relevant code being used when we encountered this issue:

# customercare.tf
locals {
  project = {
    customercare : {
      email        = local.email
      applications = keys(local.applications)
    }
  }
}

# corporate.tf
locals {
  project = {
    corporate : {
      email        = local.email
      applications = keys(local.applications)
    }
  }
}

# main.tf
locals {
  corporate_project = module.corporate.project
  customercare_project = module.customercare.project

  all_projects = merge(
    local.corporate_project,
    local.customercare_project,
  )
}

resource "spinnaker_project" "projects" {
  for_each = tomap(local.all_projects)

  name = each.key
  email = each.value.email
  config {
    applications = each.value.applications
  }
}
@KeisukeYamashita
Copy link
Contributor

Sorry for the late reply, I will take a look today 🙇

@wwalters12
Copy link
Author

@KeisukeYamashita That's great!

Please let me know if you need any more information. We're still creating projects with the provider, but using a lifecycle block to ignore changes to them afterwards. Would be great to keep them updated with the provider as we add and remove applications 🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@KeisukeYamashita @wwalters12