Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for secondary_allocation_ids on nat gateways #1005

Open
darkxeno opened this issue Nov 3, 2023 · 9 comments
Open

Add support for secondary_allocation_ids on nat gateways #1005

darkxeno opened this issue Nov 3, 2023 · 9 comments
Labels
Milestone

Comments

@darkxeno
Copy link

darkxeno commented Nov 3, 2023

Is your request related to a new offering from AWS?

Is this functionality available in the [AWS provider for Terraform]
(https://registry.terraform.io/providers/hashicorp/aws/latest/docs)? See CHANGELOG.md, too.

  • No 🛑: please wait to file a request until the functionality is avaialble in the AWS provider
  • Yes ✅: please list the AWS provider version which introduced this functionality

Yes: https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/nat_gateway#public-nat-with-secondary-private-ip-addresses

Is your request related to a problem? Please describe.

I need to add secondary public ips to my NAT gateways in order to scale my egress bandwidth.

https://aws.amazon.com/blogs/networking-and-content-delivery/attach-multiple-ips-to-a-nat-gateway-to-scale-your-egress-traffic-pattern/

Describe the solution you'd like.

Add an input variable, of type list where an user can enter an arbitrary number of secondary allocation ids, the list length should match the number of NAT devices selected by the user, each list element references 1 to 7 EIPs, similarly to the current input field: external_nat_ip_ids

A maximum of 8 IPs can be associated with each NAT gateway, 1 primary and 7 secondary ones.

Describe alternatives you've considered.

There are no alternatives as my NAT gateways and VPC are managed by this terraform module. Maybe adding a lifecycle ignore on the secondary_allocation_ids field of the VPC module?

Additional context

@pdrastil
Copy link

pdrastil commented Nov 5, 2023

I've tried to add this capability as I have similar need, however I'm running into following issue:

│ Error: Provider produced inconsistent final plan
│
│ When expanding the plan for aws_nat_gateway.this[2] to include new values learned so far during apply, provider "registry.terraform.io/hashicorp/aws" produced an invalid new value for .association_id: was
│ known, but now unknown.
│
│ This is a bug in the provider, which should be reported in the provider's own issue tracker.

Could some of the maintainers take a look what might be wrong with this piece of code? Maybe there might be a better way to write this. When code is applied for the second time EIPs that have been created are also correctly associated with NAT Gateway.

Inputs:

variable "secondary_nat_ips" {
  description = "The number of secondary private IPv4 addresses assigned to NAT Gateways"
  type        = number
  default     = 0
}

main.tf

locals {
  nat_gateway_count         = var.single_nat_gateway ? 1 : var.one_nat_gateway_per_az ? length(var.azs) : local.max_subnet_length
  nat_gateway_ips           = var.reuse_nat_ips ? var.external_nat_ip_ids : try(aws_eip.nat[*].id, [])
  nat_gateway_secondary_ips = try(chunklist(aws_eip.nat_secondary[*].id, local.nat_gateway_count), [])
}

resource "aws_eip" "nat_secondary" {
  count = local.create_vpc && var.enable_nat_gateway && var.secondary_nat_ips > 0 ? local.nat_gateway_count * var.secondary_nat_ips : 0

  domain = "vpc"

  tags = merge(
    {
      "Name" = format(
        "${var.name}-%s-ext-%s",
        element(var.azs, var.single_nat_gateway ? 0 : count.index),
        floor(count.index / (var.single_nat_gateway ? 1 : length(var.azs))) + 1
      )
    },
    var.tags,
    var.nat_eip_tags,
  )

  depends_on = [aws_internet_gateway.this]
}

resource "aws_nat_gateway" "this" {
  count = local.create_vpc && var.enable_nat_gateway ? local.nat_gateway_count : 0

  allocation_id = element(
    local.nat_gateway_ips,
    var.single_nat_gateway ? 0 : count.index,
  )

  secondary_allocation_ids = [ 
    for id in local.nat_gateway_secondary_ips : element(id, count.index) 
  ]
  // omitted for brevity
}

@darkxeno
Copy link
Author

Thank you @pdrastil for the help. I just tried the implementation you provided on the fork, but i am getting another error too, when trying to update a NAT gateway that had already 1 public ip set as primary.

module.networking.module.vpc.aws_nat_gateway.this[0]: Still creating... [3m30s elapsed]
╷
│ Error: waiting for EC2 NAT Gateway (nat-0df5ec3ea661a14aa) create: unexpected state 'failed', wanted target 'available'. last error: Resource.AlreadyAssociated: Elastic IP address [eipalloc-0f95df00baa48f688] is already associated
│
│   with module.networking.module.vpc.aws_nat_gateway.this[0],
│   on .terraform/modules/networking.vpc/main.tf line 1080, in resource "aws_nat_gateway" "this":
│ 1080: resource "aws_nat_gateway" "this" {
│
╵

not sure if you have any idea about how to overcome this.

@pdrastil
Copy link

@darkxeno This might be related to issue in AWS provider I've bumped into. When testing the snippet I've posted I was hitting errors where provider didn't know if resource should be updated or replaced. To keep it simple I haven't proceed with the rest of implementation with providing custom IPs and left the fork for upstream so they can do the repro.

@flaviomoringa
Copy link

We really need this too, please prioritise it.
Thanks

@scamp
Copy link

scamp commented Feb 29, 2024

We definetely need this as well.

@rajcheval
Copy link

We also need this. We added secondary IP address manually a while ago with hopes that module will support it.

@tkatrichenko
Copy link

No updates?

@flaviomoringa
Copy link

Seems some work is being done here: #1109 but has not moved for some weeks ;-(

@velkovb
Copy link

velkovb commented Oct 31, 2024

https://github.com/terraform-aws-modules/terraform-aws-vpc/pull/1109/files#r1823054641 - I did some testing and it seems this is not really supported in a nice way by the AWS provider.

hashicorp/terraform-provider-aws#34268

hashicorp/terraform-provider-aws#33964

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

8 participants