Replies: 6 comments 1 reply
-
|
Hi @tpidor This module uses the AzureRM resource_group_template_deployment to deploy the exemption JSON as an ARM template so we're limited by the arguments available. So would recommend sticking with passing in a name, not an id, to the resourceGroupName variable. I'm also interested in knowing what the expected behavior is when using this module in the scenario you mentioned. Will test that out myself 😄
|
Beta Was this translation helpful? Give feedback.
-
|
HI @jesseloudon, This may not be directly related to this ticket but thought of asking you here. Error: validating Template Deployment "PROD_ABC-Ex-123" (Resource Group "rg-name"): requesting validating: resources.DeploymentsClient#Validate: Failure sending request: StatusCode=404 -- Original Error: Code="ResourceGroupNotFound" Message="Resource group 'rg-name' could not be found." We did looked at the prod SPN permissions but everything was configured correctly. All SPNs include the required permissions as you suggested in #6. Prod SPN does not have issues deploying assignments with scope to different subscriptions or resource groups from different subscriptions. Automation is only failing with the policy exemptions deployment with the above error. |
Beta Was this translation helpful? Give feedback.
-
Hi @tpidor this sounds like the prod SPN needs to use the relevant subscription context. Because the error suggests the SPN can't find the RG in its current subscription context. From your Terraform calling module can you give me more info of how are you switching/targeting multiple subscriptions and linking the exemption code (via this module) to the relevant subscriptions? |
Beta Was this translation helpful? Give feedback.
-
|
Hi @jesseloudon We have one automation that deploys assignments, initiatives, policies and exemptions into Azure Tenant. The SPN used by this automation is configured and assigned at tenant level, which means it has access to all management groups, subscriptions and resource groups within the tenant. This automation can create assignments with scopes to different subscriptions or resource groups from different management groups. There's no issue from here. With the exemption part, we have this main.tf with fields as per the readme here https://github.com/globalbao/terraform-azurerm-policy-exemptions. There are more RGs here but only give 2 examples. These 2 resource groups are from different subscriptions and could be from different management groups but all in the same tenant. This main.tf is called by a parent calling module (shown below) which passes multiple assignment ids list or map. This map contains the full assignment id created during the automation. This also means that the exemption module will wait for all assignments to be created first before the exemptions going to be deployed. During the automation, Terraform apply picks up all the assignment config and apply them into tenant root level, management group or subscription level depending on what assignment scope is defined. policy/exemptions/main.tf calling module I did try to replicate the issue on another test tenant today. I setup the SPN at tenant level, put assignment config with scope at tenant root level, then add 2 exemptions on 2 different resource groups (1 exemption each group) from different subscriptions. The result is that, it successfully created the exemption on one RG while it failed on another RG with the same error - Error: Code="ResourceGroupNotFound" Message="Resource group 'rg-name' could not be found.". Then I thought maybe the exemption module picks-up first resource group and creates it successfully while the rest will fail since they're on a different subscription, so I tried switching over the RG names but still fail on the same RG name. With creating exemption using az cli, it accepts scope with full resource group id containing subscription. How does this exemption Terraform module knows the subscription id? Do you get the subscription from the active session and dynamically generate the resource group id? If yes, what if the assignment id is scoped on a different subscription as per my above explanation? If not, how can we tell to the exemption module which subscription to create the exemption? Keen to hear your analysis on this ticket and what would be the behaviour looks like. Thanks for responding to my queries again, and good job on this exemption module. |
Beta Was this translation helpful? Give feedback.
-
|
Hi @tpidor - really happy you've brought this to my attention and thank you for the detail provided! As this exemption module creates an exemption at resource group scope I recommend the following 2 steps in your particular use case.
# default provider block
provider "azurerm" {
features {}
}
# new provider block for subscription A
provider "azurerm" {
alias = "subA"
subscription_id = "xxxxx-xxxxx-xxxxx-xxxxx-xxxxx"
features {}
}
# new provider block for subscription B
provider "azurerm" {
alias = "subB"
subscription_id = "xxxxx-xxxxx-xxxxx-xxxxx-xxxxx"
features {}
}
# assignment variable
variable "assignment_ids_map" {
type = map(any)
}
# exemption module for subscription A
module "policy_exemptions_subA" {
source = "globalbao/policy-exemptions/azurerm"
version = "0.1.1"
providers = {
azurerm = azurerm.subA
}
policyExemptions = {
PROD_ABC-Ex-123_01 = {
deploymentMode = "Incremental"
name = "PROD_ABC-Ex-123_01"
displayName = "PROD_ABC-Ex-123_01 - Exemption for preventing ...."
description = "Exemption from blocking new ...."
resourceGroupName = "rg-name-01"
policyAssignmentId = var.assignment_ids_map["PROD_ABC-As-123_00001"]
policyDefinitionReferenceIds = []
exemptionCategory = "Waiver"
expiresOn = "2021-11-30T23:59:00.0000000Z"
metadata = {
"requestedBy": "Requester",
"approvedBy": "Approver",
"ticketRef": "Ticket-01"
}
}
}
}
# exemption module for subscription B
module "policy_exemptions_subB" {
source = "globalbao/policy-exemptions/azurerm"
version = "0.1.1"
providers = {
azurerm = azurerm.subB
}
policyExemptions = {
PROD_ABC-Ex-123_02 = {
deploymentMode = "Incremental"
name = "PROD_ABC-Ex-123_02"
displayName = "PROD_ABC-Ex-123_02 - Exemption for preventing ...."
description = "Exemption from blocking new ...."
resourceGroupName = "rg-name-02"
policyAssignmentId = var.assignment_ids_map["PROD_ABC-As-123_00001"]
policyDefinitionReferenceIds = []
exemptionCategory = "Waiver"
expiresOn = "2021-11-30T23:59:00.0000000Z"
metadata = {
"requestedBy": "Requester",
"approvedBy": "Approver",
"ticketRef": "Ticket-02"
}
}
}
}You can research and read more about this pattern in the official Terraform docs here: https://www.terraform.io/docs/language/modules/develop/providers.html Hopefully I haven't made any mistakes/errors in my example above and that this helps you in your journey 😄 Cheers |
Beta Was this translation helpful? Give feedback.
-
|
Hi @jesseloudon Thanks for your recommendations. I did POC the addition of providers block into the policy-exemptions module as you explained, and yes it is working. I would suggest to maybe mention in the readme about adding providers block if resource groups are from different subscriptions. I'm now implementing to adapt into our automation which requires a refactor to the initial implementation which only accepts a list of resource groups without subscription ids. Basically, our automation is customer (or policy developer) facing. This means, the terraform modules and resources are not directly available to policy developers to modify. Policy developers are only writing up json config for assignments, initiatives, policies and exemptions. Pipeline automation will then picks up these json configs and feed into our automation to deploy Azure policies. In the case of exemptions, json config contains resource group lists to exempt without subscription (causing the issue of RG not found). This json config will be used to generate the main.tf for exemption modules. Previously, the sample below has only exemptions element in the config. This time, I'll be adding subscriptions with aliases which will be used to generate the 'provider "azurerm" {}' blocks. Subscription aliases will be needed this time beside the resource group name which will generate the 'module "policy_exemptions" {}' blocks. Meanwhile question to you, is it possible to add (if only doable) the subscription id into the "PolicyExemptions" block like below? This way, "globalbao/policy-exemptions/azurerm" module can use that subscription id to identify where the resource group is and avoid the use of multiple 'provider "azurerm" {}' blocks. I understand the provider azurerm block is a standard practice from https://www.terraform.io/docs/language/modules/develop/providers.html, but just thought of asking you in case it is doable for "globalbao/policy-exemptions/azurerm" to accept subscription id as well in "policyExemptions = {}" block. This way, we could minimize declaring many exemption modules if there are many resource groups to exempt from different subscriptions. The other question though is that, do you have a threshold on how many resource groups can "policyExemptions = {}" block handles? If it crashes on certain resource group count assuming that all resource groups exemptions will be created all at the same Terraform apply runtime, then maybe adding subscription id in "policyExemptions = {}" and putting all resource groups in the same block in only one 'module "policy_exemptions" {}' may not be a good idea. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the suggestion! Have updated the README #14 to show usage for multiple subscriptions! For your subscriptionId request would you be able to create a new issue/feature request? https://github.com/globalbao/terraform-azurerm-policy-exemptions/issues/new I'm not aware of any limitation/threshold on the number of RGs a policyExemptions block can handle. Cheers |
Beta Was this translation helpful? Give feedback.
-
Although it was mentioned in https://github.com/globalbao/terraform-azurerm-policy-exemptions#variables that resourceGroupName is the name of the Resource Group. I thought of trying to put the full resource group id but this error came up:
Error: "resource_group_name" may not exceed 90 characters in lengthIf an assignment is at tenant or management group level and the exemption needs to be applied to 2 or more resource groups of the same exact name but of different subscriptions, what is the expected behaviour? I would suggest to add the expected behaviour in the documentations if that's Ok.
Additionally, it would be good enhancement if the resourceGroupName field can accept resource group id which gives us ability to specify a specific resource group from a subscription to exempt.
Beta Was this translation helpful? Give feedback.
All reactions