Real-world usefulness of azd infra synth w/ Aspire

[rudiv]

I know this is somewhat Aspire related, but it's azd that is providing the limited options when it comes to infra synth. Yes, I know it's also an alpha. Yes, I know this is a tool targeted at developers.

In a relatively large project that will be extended further and further as time goes on, the usefulness of synthesising a production infrastructure-as-code becomes limited to the first run and subsequent runs only for new projects.

I'm talking specifically about this resulting confirmation after the first run:

? What would you like to do with these files? [Use arrows to move, type to filter]
Overwrite with the synthesized versions
Keep my existing files unchanged

In the real world, the generated files are (somewhat) useless, in so far as for a production environment, proper naming, extra resources and further configuration of the generated files becomes very important.

The options presented are a simple brute force all or nothing (well, new file only) approach, with no change detection baked in to the tool (say for example if a containerApp.tmpl.yaml wasn't changed as a result of this synth it's still overwritten).

Whilst it's great that for brand new services the files are still output, if the resources.bicep was changed (consider the Redis cache which by default it deploys the ACA Service but as a container, yet in the real world that would likely point to an Azure Cache for Redis instance so would be configured) there is no way to see what differences would actually be made.

I'm not saying I have a solution for this, but something as simple as separating out the resources.bicep in to different files could be beneficial. In the above scenario if there were 2 caches, 1 at the time of the first synth and another added later, having cache-name.bicep and cache-name2.bicep, allowing them to be configured would work as the cache-name2.bicep could be generated whilst untouching the first. Granted I know Bicep has limitations in what can be shared, but putting everything that isn't a separate project in to one file just seems wrong.

Another option could be presenting the diff of the files directly so they can be manually applied in an easier way? Again, not ideal, but better than all or nothing.

There are relatively simple workarounds such as moving the configured resources to separate files and replacing the "wrong" bits or re-adding additional references in the newly overwritten resources|main.bicep files.

Just curious as to what the stance is on this and if there are any more plans given the early stages? Maybe it's just intended as a starting point and everything is meant to be manual after this point?

[cfarrugia]

I totally agree with your comments. I had to make ingress changes on the ACA, which get wiped out every time. I need to have multiple environments with exactly the same code, yet i need to deploy the ACR multiple times.

Aspire has been amazing for local use, and to quickly put something up in test/stage/demo. However, we need more control in real life production stuff.

I would suggest to leave the option to be able to leave the provisioning as a different process, and Aspire can configure specific ACAs/ACRs that have been previously created through another IaC process.

[rudiv]

Ironically with the addition of AddBicepTemplate to Aspire, this has gotten better/worse.

Obviously it's better as we can now include Bicep within the app host, and have it in the actual deployable files by calling azd infra synth, however any change to the Bicep requires another synth which, as you've probably guessed - overwrites everything.

In our scenario, we have a bicep file that ingests the managed identity from the default set of resources created. So we can go in and change the main.bicep to read this from resources.outputs.MANAGED_IDENTITY_PRINCIPAL_ID.

Any change to the bicep in question requires an infra synth to stay in sync, and obviously wipes out this change. This is just 1 extra file, but there are of course other scenarios wherein there would be more each requiring changes in the future. With azd infra synth overwriting everything by default it's very difficult to keep changes outside of git history.

The only stable option at this point remains to keep bicep separate from the set of files from infra synth so that they can't be overridden which once again, goes against the "developer simplicity" nature of this tool.

Side note, Aspire specific you can workaround needing to edit the main.bicep by simply doing .WithParameter("principalId") which uses some azd magic to "know" that it wants the MANAGED_IDENTITY_PRINCIPAL_ID but of course this isn't documented anywhere. Only found this out by adding a KeyVault which generates its own Bicep file that magically fills in the principalId parameter. Turns out it also works with custom Bicep (see dotnet/docs-aspire#522).

[vhvb1989]

@rudiv , I'm curious on learning more about your use-case.

Ideally, when you use .netAspire, you will not call azd infra synth unless there is something you are not able to define for the infrastructure in the AppHost. So, I'm curious about your use-case.

Actually, there are two main scenarios when you want to use infra synth:

• Adjusting the infrastructure beyond what you can define in the appHost (as mentioned before)
• As the strategy to skip in-memory infrastructure generation. For example, for a very large AppHost projects, it might take some time for azd to produce the infrastructure (bicep files), and maybe you know that AppHost is not expected to be change anymore (you are only working on the service's code, w/o updating infrastructure). It might be a good idea to run infra synth and commit the infrastructure to your repo and keep it like some sort of cache.

There are some changes happening for .netAspire for preview5 which will hopefully eliminate the first scenario. You will be able to define everything for a resource, like you do with bicep, but using the AppHost. You can see some initial examples here: https://github.com/dotnet/aspire/blob/main/playground/cdk/CdkSample.AppHost/Program.cs#L44

After that, the scenario for running infra synth will no longer be for adjusting infrastructure, but for committing the infrastructure to your repo and probably share it with other tools/integrations (for example, someone not using azd might be expecting bicep).

Once you commit the infrastructure to your repo, your version-control, like git, should help you to easily see new changes if you ever need to update it by running infra synth again.

[rudiv]

Thanks @vhvb1989 @davidfowl.

I’ll wait patiently for P5. Don’t get me wrong, if getting away from the magic and the majority of handcrafted bicep is still possible within Aspire I (and I’m sure a lot of others) are going to be very happy, even if we have to fallback to writing bicep for some other stuff that isn’t in Aspire. I certainly don’t expect everything to be in there.

Again Aspire and azd have been great for local dev & test deployments it’s just getting the final bits in place that was a bit manual.

I’ll check out PublishAsAzureRedis.

[rudiv]

One further thing to add on this as I'm not sure I see it referenced anywhere.

I've been through #3292 as well as other issues, and a lot of them (including the P4) changes are "RunAs" and "PublishAs", but what about intermediate environments between staging and prod? Right now we handle it with Bicep that takes the resource group name (or now, a parameter) and determines which environment it's pushing to, but for instance with Azure Redis we may want a Basic C0 in dev, a Standard C0 in Staging and a Standard C1 in live. Another example would be redundancy requirements on Azure Storage.

That's 3 "publish configurations", is consideration being made for this scenario too when it comes to CDK / Aspire generated Bicep or is it a scenario wherein we would still have to fall down to manual edits?

[davidfowl]

I've been through #3292 as well as other issues, and a lot of them (including the P4) changes are "RunAs" and "PublishAs", but what about intermediate environments between staging and prod? Right now we handle it with Bicep that takes the resource group name (or now, a parameter) and determines which environment it's pushing to, but for instance with Azure Redis we may want a Basic C0 in dev, a Standard C0 in Staging and a Standard C1 in live. Another example would be redundancy requirements on Azure Storage.

You should be able to model this by setting an DOTNET_ENVIRONMENT in your azd environment (azd env set DOTNET_ENVIRONMENT value) (#3425). Then you can use conditional logical in your application:

builder.AddRedis("redis").PublishAsAzureRedis((_, _, cache) =>
{
    var (sku, family) = builder.Environment.EnvironmentName switch
    {
        "Staging" => ("Standard", "C0"),
        "Live" => ("Premium", "C1"),
        _ => ("Basic", "C0")
    };

    cache.AssignProperty(r => r.Sku.Name, sku);
    cache.AssignProperty(r => r.Sku.Family, family);
});

PS: azd still has work to do, to generate the infra in a folder related to the azd environment I believe (@ellismg?)

[rudiv]

PS: azd still has work to do, to generate the infra in a folder related to the azd environment I believe (@ellismg?)

I think this was more what I was getting at. Even if this is in the Aspire app, it would overwrite the Bicep files depending on the environment which isn't applicable.

With other tooling (*cough Terraform) we can use workspaces (or Terragrunt), have multiple versions of infra code, or whatever else. azd limits us to a single version of truth meaning that the Bicep itself has to be crafted to support multiple environments for each resource. Ideally, an infra.dev, infra.stage, infra.prod (or / to nest) would fit the azd model to support this, likely with an unspecified mode of infra.default if this multi-environment support wasn't a requirement.

It could be modelled using env variables or parameters as possible variations, but it's a lot of cognitive overhead to up-front decide what could change and model the bicep around that. Given that it's been generated by Aspire, being able to support these multiple environments natively and just configure the output from code would be brilliant, and once again means we shouldn't really have to drop down to bicep to get the full story in place from local dev -> prod.

Having something like this would make azd far more generally applicable. Even though it would differ from the way other tooling would work, it feels very ".NETy" (thinking of appsettings.{env}.json).

EDIT: Related #2373

[davidfowl]

This is something that we'll address in updates to azd and aspire post the GA release. We have a good base but there's a lot more work to do to make the product ready for more real deployment situations.