Can't Datasource a Deployment by its name

[phergoualch]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

Terraform Version

1.9.8

Affected Resource(s)

• data.prefect_deployment

Terraform Configuration Files

data "data" "existing" {
  # id = "9e1eb232-4f15-404d-8a84-3727cc68f584"
  name = "mysql-backup/mysql-backup"
}

Expected Behavior

It is working fine with the deployment id but not the deployment name. I tested both mysql-backup and mysql-backup/mysql-backup as outputed by the prefect deployment ls.

$ prefect deployment ls

Name	ID	Work Pool
mysql-backup/mysql-backup	9e1eb232-4f15-404d-8a84-3727cc68f584	ecs-fargate-spot

[mitchnielsen]

Thanks for reaching out @phergoualch

I believe you just need to tweak your data block slightly based on the documentation:

- data "data" "existing" {
+ data "prefect_deployment" "existing"
  # id = "9e1eb232-4f15-404d-8a84-3727cc68f584"
-  name = "mysql-backup/mysql-backup"
+  name = "mysql-backup"
}

This does the following:

• Changes the first argument to "data" from "data" to "prefect_deployment" so it knows which data source logic to use
• Changes the name to the name of the deployment (I believe prefect deployment ls prints out <flow name>/<deployment name>)

[phergoualch]

Well, the data is a mistake from my copy past, I used the correct data_source name.
And as said in the issue, I tested both mysql-backup and mysql-backup/mysql-backup

[mitchnielsen]

Sure, just pointing out that the documentation shows you only need the deployment name, not flow name/deployment name.

Since I wasn't able to replicate the error, can you please:

1. Include your complete (relevant) Terraform configuration instead of just the data block
2. Include the relevant output you see from your side

This will hopefully help me replicate the issue and find the root cause.

[phergoualch]

The full code

data "prefect_deployment" "existing" {
  # id = "9e1eb232-4f15-404d-8a84-3727cc68f584"
  name = "mysql-backup"
}

output "existing_deployment" {
  value = data.prefect_deployment.existing
}

The error when trying to datasource with name

│ Error: Could not find Deployment
│ 
│   with data.prefect_deployment.existing,
│   on main.tf line 1, in data "prefect_deployment" "existing":
│    1: data "prefect_deployment" "existing" {
│ 
│ Could not find Deployment with name mysql-backup
╵

The deployment output when datasourcing with id

{
  "account_id" = tostring(null)
  "concurrency_limit" = tonumber(null)
  "concurrency_options" = null /* object */
  "created" = "2024-12-13T14:19:12Z"
  "description" = "Main flow for MySQL backup operations"
  "enforce_parameter_schema" = tobool(null)
  "entrypoint" = "flow.py:mysql_backup"
  "flow_id" = "79685e32-e1cd-46f5-b401-b4dfaa1b8427"
  "id" = "9e1eb232-4f15-404d-8a84-3727cc68f584"
  "job_variables" = "{\"image\":\"503532123506.dkr.ecr.eu-west-1.amazonaws.com/prefect-flows:mysql-backup\"}"
  "manifest_path" = tostring(null)
  "name" = "mysql-backup"
  "parameter_openapi_schema" = "{\"definitions\":{\"BackupConfig\":{\"description\":\"Backup configuration\",\"properties\":{\"backup_dir\":{\"default\":\"/tmp/sqlbackup\",\"description\":\"Backup directory path\",\"title\":\"Backup Dir\",\"type\":\"string\"},\"db_exclude\":{\"default\":[\"information_schema\",\"innodb\",\"sys\",\"tmp\",\"mysql\",\"performance_schema\",\"awsdms_control\"],\"description\":\"Databases to exclude from backup\",\"items\":{\"type\":\"string\"},\"title\":\"Db Exclude\",\"type\":\"array\"},\"s3_bucket\":{\"default\":\"rs-databases-backup\",\"description\":\"S3 bucket name\",\"title\":\"S3 Bucket\",\"type\":\"string\"}},\"title\":\"BackupConfig\",\"type\":\"object\"},\"DBCredentials\":{\"description\":\"Database connection credentials\",\"properties\":{\"host\":{\"description\":\"Database host\",\"title\":\"Host\",\"type\":\"string\"},\"password\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"null\"}],\"default\":null,\"description\":\"Database password\",\"title\":\"Password\"},\"port\":{\"default\":3306,\"description\":\"Database port\",\"title\":\"Port\",\"type\":\"integer\"},\"secret_arn\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"null\"}],\"default\":null,\"description\":\"AWS Secrets Manager ARN\",\"title\":\"Secret Arn\"},\"user\":{\"anyOf\":[{\"type\":\"string\"},{\"type\":\"null\"}],\"default\":null,\"description\":\"Database username\",\"title\":\"User\"}},\"required\":[\"host\"],\"title\":\"DBCredentials\",\"type\":\"object\"},\"MySQLOptions\":{\"description\":\"MySQL client options configuration\",\"properties\":{\"backup\":{\"default\":\"--opt --skip-lock-tables --single-transaction --max_allowed_packet=256M --routines\",\"description\":\"MySQL dump options\",\"title\":\"Backup\",\"type\":\"string\"}},\"title\":\"MySQLOptions\",\"type\":\"object\"}},\"properties\":{\"config\":{\"$ref\":\"#/definitions/BackupConfig\",\"position\":1,\"title\":\"config\"},\"credentials\":{\"$ref\":\"#/definitions/DBCredentials\",\"position\":0,\"title\":\"credentials\"},\"mysql_opts\":{\"anyOf\":[{\"$ref\":\"#/definitions/MySQLOptions\"},{\"type\":\"null\"}],\"default\":null,\"position\":2,\"title\":\"mysql_opts\"}},\"required\":[\"credentials\",\"config\"],\"title\":\"Parameters\",\"type\":\"object\"}"
  "parameters" = "{}"
  "path" = "."
  "paused" = false
  "pull_steps" = tolist([])
  "storage_document_id" = "00000000-0000-0000-0000-000000000000"
  "tags" = tolist(null) /* of string */
  "updated" = "2024-12-19T15:45:23Z"
  "version" = "1ff9fcc2f5f4c5a9f97fec0a6bde0e83"
  "work_pool_name" = "ecs-fargate-spot"
  "work_queue_name" = "default"
  "workspace_id" = tostring(null)
}

Note that the exisiting deployment was not created using Terraform but Python. Could this be the cause ?
I want to be able to create a "template" deployment with the flow.deploy() function and then be able to duplicate it using Terraform

[mitchnielsen]

Thank you, I think I found the problem - there's some missing logic in Deployment.List. Will follow up soon.

[mitchnielsen]

I've opened #343 which should help here. In the meantime, you should be able to datasource the deployment by id for now.

[mitchnielsen]

@phergoualch - give 2.13.4 a try when you have a chance, we pushed a fix for datasourcing Deployments.