Add the foundations for an extensible cost optimization recommendations pipeline framework

[helderpinto]

📝 Scenario

As a FinOps Practitioner using FinOps Hubs, I need to have a centralized list of cloud provider-generated and custom cost optimization recommendations with an historical perspective in order to be able to prioritize rate and usage optimization tasks based on potential savings and historical relevance.

💎 Solution

The Workload optimization and Rate optimization reports will include pages dedicated to cost optimization recommendations, not only coming from Azure Advisor, but also from custom recommendations based on Azure Resource Graph KQL queries (similar to the ones available in the Cost Optimization Workbook). Those pages will provide not only the latest available recommendations but also an historical perspective of how long those recommendations have been made.

📋 Tasks

Required tasks

Preview Give feedback

1. Daily export of Azure Advisor and custom Azure Resource Graph-based recommendations into Azure Storage.
2. Extend Azure Data Explorer recommendations schema with the new properties required by this new recommendations type.
3. Implement Azure Storage and Azure Data Explorer-based Power BI pages for the Workload optimization and Rate Optimization reports.
4. Hubs deployment documented manual steps to assign the required permissions to the Azure Data Factory identity.
5. Update documentation with this new capability

Stretch goals

Preview Give feedback

1. Hubs deployment automation to assign the required permissions to the Azure Data Factory identity.
2. Enrich recommendations information presented in the reports by correlating with costs data.

ℹ️ Additional context

The solution must allow for easily adding new custom recommendations in future releases.

🙋‍♀️ Ask for the community

We could use your help:

1. Please vote this issue up (👍) to prioritize it.
2. Leave comments to help us solidify the vision.