How to have the terraform manager search bitbucket or git tags for a bitbucket git URL terraform module source? #28620

shaafekhan · 2024-04-24T13:36:51Z

shaafekhan
Apr 24, 2024

What would you like help with?

I would like help with my configuration

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

Bitbucket Cloud

Please tell us more about your question or problem

Hi there,

As an example, lets say I have the following module.tf file with the following module block:

module "terraform-module" {
  source = "[email protected]:bitbucketuser/terraform-module-repo.git?ref=1.3.3"
}

I'd like to have renovate check any new versions based on the ref=1.3.3 string in the source variable, these refs are referring to a bitbucket tag in the terraform-module-repo repository. If I've understood the documentation properly, the terraform module manager should automatically be able to detect bitbucket-tags docs, however it does not seem to be doing this. in the terraform-module-repo, newer versions of the module have been released using the semver versioning method. Please let me know how I could configure this, thank you.

Logs (if relevant)

Logs



DEBUG: Using RE2 regex engine
DEBUG: Parsing configs
DEBUG: Checking for config file in /opt/atlassian/pipelines/agent/build/renovate.js
DEBUG: File config
       "config": {
         "platform": "bitbucket",
         "password": "***********",
         "autodiscover": false,
         "repositories": [{"repository": "companydev/renovate-bot"}],
         "username": "team-bot",
         "gitAuthor": "team-bot <[email protected]>",
         "onboarding": true,
         "logFileLevel": "debug"
       }
DEBUG: CLI config
       "config": {}
DEBUG: Env config
       "config": {"hostRules": [], "password": "***********"}
DEBUG: Combined config
       "config": {
         "platform": "bitbucket",
         "password": "***********",
         "autodiscover": false,
         "repositories": [{"repository": "companydev/renovate-bot"}],
         "username": "team-bot",
         "gitAuthor": "team-bot <[email protected]>",
         "onboarding": true,
         "logFileLevel": "debug",
         "hostRules": []
       }
DEBUG: Enabling forkProcessing while in non-autodiscover mode
DEBUG: Found valid git version: 2.43.2
DEBUG: Setting global hostRules
DEBUG: hostRules: authentication already set for api.bitbucket.org
DEBUG: Using configured gitAuthor (team-bot <[email protected]>)
DEBUG: Adding password authentication for api.bitbucket.org (hostType=bitbucket) to hostRules
DEBUG: Using baseDir: /tmp/renovate
DEBUG: Using cacheDir: /tmp/renovate/cache
DEBUG: Using containerbaseDir: /tmp/renovate/cache/containerbase
DEBUG: Initializing Renovate internal cache into /tmp/renovate/cache/renovate/renovate-cache-v1
DEBUG: Commits limit = null
DEBUG: Setting global hostRules
DEBUG: Adding password authentication for api.bitbucket.org (hostType=bitbucket) to hostRules
DEBUG: validatePresets()
DEBUG: Reinitializing hostRules for repo
DEBUG: Clearing hostRules
DEBUG: Adding password authentication for api.bitbucket.org (hostType=bitbucket) to hostRules
 INFO: Repository started (repository=companydev/renovate-bot)
       "renovateVersion": "37.305.0"
DEBUG: Using localDir: /tmp/renovate/repos/bitbucket/companydev/renovate-bot (repository=companydev/renovate-bot)
DEBUG: PackageFiles.clear() - Package files deleted (repository=companydev/renovate-bot)
DEBUG: initRepo("companydev/renovate-bot") (repository=companydev/renovate-bot)
DEBUG: hostRules: applying Basic authentication for api.bitbucket.org (repository=companydev/renovate-bot)
DEBUG: companydev/renovate-bot owner = companydev (repository=companydev/renovate-bot)
DEBUG: Resetting npmrc (repository=companydev/renovate-bot)
DEBUG: Resetting npmrc (repository=companydev/renovate-bot)
DEBUG: checkOnboarding() (repository=companydev/renovate-bot)
DEBUG: isOnboarded() (repository=companydev/renovate-bot)
DEBUG: findPr(renovate/configure, Configure Renovate, !open) (repository=companydev/renovate-bot)
DEBUG: getPrList() (repository=companydev/renovate-bot)
DEBUG: Syncing PR list (repository=companydev/renovate-bot)
DEBUG: http cache: saving https://api.bitbucket.org/2.0/repositories/companydev/renovate-bot/pullrequests?state=OPEN&state=MERGED&state=DECLINED&state=SUPERSEDED&fields=values.id%2Cvalues.title%2Cvalues.state%2Cvalues.links.commits.href%2Cvalues.summary.raw%2Cvalues.source.branch.name%2Cvalues.destination.branch.name%2Cvalues.reviewers.display_name%2Cvalues.reviewers.uuid%2Cvalues.reviewers.nickname%2Cvalues.reviewers.account_status%2Cvalues.created_on%2Cvalues.updated_on&q=author.uuidredacted lastModified=undefined) (repository=companydev/renovate-bot)
DEBUG: findPr(renovate/configure, chore: Configure Renovate, !open) (repository=companydev/renovate-bot)
DEBUG: getPrList() (repository=companydev/renovate-bot)
DEBUG: findFile(renovate.json) (repository=companydev/renovate-bot)
DEBUG: Initializing git repository into /tmp/renovate/repos/bitbucket/companydev/renovate-bot (repository=companydev/renovate-bot)
DEBUG: Performing blobless clone (repository=companydev/renovate-bot)
DEBUG: git clone completed (repository=companydev/renovate-bot)
       "durationMs": 469
DEBUG: latest repository commit (repository=companydev/renovate-bot)
       "latestCommit": {
         "hash": "redacted",
         "date": "2024-04-24T15:27:28+02:00",
         "message": "add",
         "refs": "HEAD -> main, origin/main, origin/HEAD",
         "body": "",
         "author_name": "author",
         "author_email": "[email protected]"
       }
DEBUG: findFile(renovate.json5) (repository=companydev/renovate-bot)
DEBUG: findFile(.github/renovate.json) (repository=companydev/renovate-bot)
DEBUG: findFile(.github/renovate.json5) (repository=companydev/renovate-bot)
DEBUG: findFile(.gitlab/renovate.json) (repository=companydev/renovate-bot)
DEBUG: findFile(.gitlab/renovate.json5) (repository=companydev/renovate-bot)
DEBUG: findFile(.renovaterc) (repository=companydev/renovate-bot)
DEBUG: findFile(.renovaterc.json) (repository=companydev/renovate-bot)
DEBUG: findFile(.renovaterc.json5) (repository=companydev/renovate-bot)
DEBUG: config file not found (repository=companydev/renovate-bot)
DEBUG: Found no closed onboarding PR (repository=companydev/renovate-bot)
DEBUG: Repo is not onboarded (repository=companydev/renovate-bot)
DEBUG: getBranchPr(renovate/configure) (repository=companydev/renovate-bot)
DEBUG: findPr(renovate/configure, undefined, open) (repository=companydev/renovate-bot)
DEBUG: getPrList() (repository=companydev/renovate-bot)
DEBUG: Found PR #1 (repository=companydev/renovate-bot)
DEBUG: Onboarding PR already exists (repository=companydev/renovate-bot)
DEBUG: branch.isModified(): using git to calculate (repository=companydev/renovate-bot)
DEBUG: branch.isModified() = true (repository=companydev/renovate-bot)
       "branchName": "renovate/configure",
       "lastAuthor": "[email protected]",
       "gitAuthorEmail": "[email protected]"
DEBUG: setCachedModifiedResult(): Branch cache not present (repository=companydev/renovate-bot)
DEBUG: isBranchConflicted(main, renovate/configure) (repository=companydev/renovate-bot)
DEBUG: branch.isConflicted(): using git to calculate (repository=companydev/renovate-bot)
DEBUG: Setting git author name: team-bot (repository=companydev/renovate-bot)
DEBUG: Setting git author email: [email protected] (repository=companydev/renovate-bot)
DEBUG: setCachedConflictResult(): Branch cache not present (repository=companydev/renovate-bot)
DEBUG: branch.isConflicted(): false (repository=companydev/renovate-bot)
DEBUG: Merge onboarding branch in default branch (repository=companydev/renovate-bot)
DEBUG: Create Onboarding Cache (repository=companydev/renovate-bot)
       "onboardingCache": {
         "defaultBranchSha": "redacted",
         "onboardingBranchSha": "redacted",
         "isConflicted": false,
         "isModified": true
       }
DEBUG: Found renovate.json config file (repository=companydev/renovate-bot)
DEBUG: Repository config (repository=companydev/renovate-bot)
       "fileName": "renovate.json",
       "config": {
         "$schema": "https://docs.renovatebot.com/renovate-schema.json",
         "enabledManagers": ["terraform"],
         "terraform": {
           "commitMessageTopic": "Terraform {{depName}}",
           "fileMatch": ["\\.tf$"],
           "pinDigests": false,
           "depTypes": ["module"]
         }
       }
DEBUG: migrateAndValidate() (repository=companydev/renovate-bot)
DEBUG: Config migration necessary (repository=companydev/renovate-bot)
       "oldConfig": {
         "$schema": "https://docs.renovatebot.com/renovate-schema.json",
         "enabledManagers": ["terraform"],
         "terraform": {
           "commitMessageTopic": "Terraform {{depName}}",
           "fileMatch": ["\\.tf$"],
           "pinDigests": false,
           "depTypes": ["module"]
         }
       },
       "newConfig": {
         "$schema": "https://docs.renovatebot.com/renovate-schema.json",
         "enabledManagers": ["terraform"],
         "terraform": {
           "commitMessageTopic": "Terraform {{depName}}",
           "fileMatch": ["\\.tf$"],
           "pinDigests": false
         }
       }
DEBUG: Post-massage config (repository=companydev/renovate-bot)
       "config": {
         "$schema": "https://docs.renovatebot.com/renovate-schema.json",
         "enabledManagers": ["terraform"],
         "terraform": {
           "commitMessageTopic": "Terraform {{depName}}",
           "fileMatch": ["\\.tf$"],
           "pinDigests": false
         }
       }
DEBUG: Found repo ignorePaths (repository=companydev/renovate-bot)
       "ignorePaths": ["**/node_modules/**", "**/bower_components/**"]
DEBUG: No vulnerability alerts found (repository=companydev/renovate-bot)
DEBUG: No baseBranches (repository=companydev/renovate-bot)
DEBUG: extract() (repository=companydev/renovate-bot)
DEBUG: Setting current branch to main (repository=companydev/renovate-bot)
DEBUG: latest commit (repository=companydev/renovate-bot)
       "branchName": "main",
       "latestCommitDate": "2024-04-24T13:28:16+00:00"
DEBUG: Using file match: \.tf$ for manager terraform (repository=companydev/renovate-bot)
DEBUG: Using file match: \.tf$ for manager terraform (repository=companydev/renovate-bot)
DEBUG: Matched 1 file(s) for manager terraform: https://bitbucket.org/companydev/renovate-bot/src/redacted/test-file/module.tf (repository=companydev/renovate-bot)
DEBUG: manager extract durations (ms) (repository=companydev/renovate-bot)
       "managers": {"terraform": 8}
DEBUG: Found terraform package files (repository=companydev/renovate-bot)
DEBUG: Found 1 package file(s) (repository=companydev/renovate-bot)
 INFO: Dependency extraction complete (repository=companydev/renovate-bot, baseBranch=main)
       "stats": {
         "managers": {"terraform": {"fileCount": 1, "depCount": 1}},
         "total": {"fileCount": 1, "depCount": 1}
       }
DEBUG: PackageFiles.add() - Package file saved for base branch (repository=companydev/renovate-bot, baseBranch=main)
DEBUG: Package releases lookups complete (repository=companydev/renovate-bot, baseBranch=main)
DEBUG: branchifyUpgrades (repository=companydev/renovate-bot)
DEBUG: detectSemanticCommits() (repository=companydev/renovate-bot)
DEBUG: getCommitMessages (repository=companydev/renovate-bot)
DEBUG: semanticCommits: detected "unknown" (repository=companydev/renovate-bot)
DEBUG: semanticCommits: disabled (repository=companydev/renovate-bot)
DEBUG: 0 flattened updates found:  (repository=companydev/renovate-bot)
DEBUG: Returning 0 branch(es) (repository=companydev/renovate-bot)
DEBUG: config.repoIsOnboarded=false (repository=companydev/renovate-bot)
DEBUG: packageFiles with updates (repository=companydev/renovate-bot, baseBranch=main)
       "config": {
         "terraform": [
           {
             "deps": [
               {
                 "depName": "terraform-module-repo",
                 "depType": "module",
                 "updates": [],
                 "packageName": "terraform-module-repo"
               }
             ],
             "packageFile": "test-file/module.tf"
           }
         ]
       }
DEBUG: detectSemanticCommits() (repository=companydev/renovate-bot)
DEBUG: semanticCommits: returning "disabled" from cache (repository=companydev/renovate-bot)
DEBUG: ensureOnboardingPr() (repository=companydev/renovate-bot)
DEBUG: getBranchPr(renovate/configure) (repository=companydev/renovate-bot)
DEBUG: findPr(renovate/configure, undefined, open) (repository=companydev/renovate-bot)
DEBUG: getPrList() (repository=companydev/renovate-bot)
DEBUG: Found PR #1 (repository=companydev/renovate-bot)
DEBUG: Filling in onboarding PR template (repository=companydev/renovate-bot)
DEBUG: getConfigDesc() (repository=companydev/renovate-bot)
DEBUG: getDescriptionArray() (repository=companydev/renovate-bot)
DEBUG: getScheduleDesc() (repository=companydev/renovate-bot)
DEBUG: No schedule (repository=companydev/renovate-bot)
DEBUG: No config description found (repository=companydev/renovate-bot)
DEBUG: getPrList() (repository=companydev/renovate-bot)
DEBUG: Found open onboarding PR (repository=companydev/renovate-bot)
DEBUG: Pull Request #1 does not need updating (repository=companydev/renovate-bot)
DEBUG: processRepo() (repository=companydev/renovate-bot)
DEBUG: ensureDependencyDashboard() (repository=companydev/renovate-bot)
DEBUG: Closing Dependency Dashboard (repository=companydev/renovate-bot)
DEBUG: Issues are disabled - cannot ensureIssueClosing (repository=companydev/renovate-bot)
DEBUG: validateReconfigureBranch() (repository=companydev/renovate-bot)
DEBUG: No reconfigure branch found (repository=companydev/renovate-bot)
DEBUG: Removing any stale branches (repository=companydev/renovate-bot)
DEBUG: config.repoIsOnboarded=false (repository=companydev/renovate-bot)
DEBUG: Branch lists (repository=companydev/renovate-bot)
       "branchList": ["renovate/configure"],
       "renovateBranches": ["renovate/configure"]
DEBUG: remainingBranches= (repository=companydev/renovate-bot)
DEBUG: No branches to clean up (repository=companydev/renovate-bot)
DEBUG: Issues are disabled - cannot ensureIssueClosing (repository=companydev/renovate-bot)
DEBUG: Issues are disabled - cannot ensureIssueClosing (repository=companydev/renovate-bot)
DEBUG: PackageFiles.clear() - Package files deleted (repository=companydev/renovate-bot)
DEBUG: getPrList() (repository=companydev/renovate-bot)
DEBUG: Branch summary (repository=companydev/renovate-bot)
       "cacheModified": undefined,
       "baseBranches": [{"branchName": "main", "sha": "redacted"}],
       "branches": [],
       "defaultBranch": "main",
       "inactiveBranches": []
DEBUG: Renovate repository PR statistics (repository=companydev/renovate-bot)
       "stats": {"total": 0, "open": 0, "closed": 0, "merged": 0}
DEBUG: Repository result: undefined, status: onboarding, enabled: true, onboarded: false (repository=companydev/renovate-bot)
DEBUG: Repository timing splits (milliseconds) (repository=companydev/renovate-bot)
       "splits": {"init": 1756, "extract": 70, "lookup": 30, "onboarding": 64, "update": 0},
       "total": 1933
DEBUG: Package cache statistics (repository=companydev/renovate-bot)
       "get": {"count": 0, "avgMs": 0, "medianMs": 0, "maxMs": 0, "totalMs": 0},
       "set": {"count": 0, "avgMs": 0, "medianMs": 0, "maxMs": 0, "totalMs": 0}
DEBUG: HTTP statistics (repository=companydev/renovate-bot)
       "urls": {
         "https://api.bitbucket.org/2.0/repositories/companydev/renovate-bot": {
           "GET": {"200": 1}
         },
         "https://api.bitbucket.org/2.0/repositories/companydev/renovate-bot/pullrequests/1": {
           "GET": {"200": 1}
         },
         "https://api.bitbucket.org/2.0/repositories/companydev/renovate-bot/pullrequests": {
           "GET": {"200": 1}
         }
       },
       "hosts": {
         "api.bitbucket.org": {
           "count": 3,
           "reqAvgMs": 174,
           "reqMedianMs": 145,
           "reqMaxMs": 252,
           "queueAvgMs": 0,
           "queueMedianMs": 0,
           "queueMaxMs": 0
         }
       },
       "requests": 3
DEBUG: HTTP cache statistics (repository=companydev/renovate-bot)
       "https://api.bitbucket.org": {
         "/2.0/repositories/companydev/renovate-bot/pullrequests": {
           "hit": 0,
           "miss": 1
         }
       }
DEBUG: Lookup statistics (repository=companydev/renovate-bot)
DEBUG: dns cache (repository=companydev/renovate-bot)
       "hosts": []
 INFO: Repository finished (repository=companydev/renovate-bot)
       "cloned": true,
       "durationMs": 1933
DEBUG: Checking file package cache for expired items
DEBUG: Verifying and cleaning cache: /tmp/renovate/cache/renovate/renovate-cache-v1
DEBUG: Deleted 0 of 0 file cached entries in 15ms

Searching for test report files in directories named [test-results, failsafe-reports, test-reports, TestResults, surefire-reports] down to a depth of 4
Finished scanning for test reports. Found 0 test report files.
Merged test suites, total number tests is 0, with 0 failures and 0 errors.

Answered by shaafekhan

Apr 30, 2024

Ended up figuring it out myself. solution (in renovate.json):

 "customManagers": [
    {
      "customType": "regex",
      "fileMatch": ["\\.tf$"],
      "matchStrings": ["source.=.\\\"(?<depName>.*?)\\?ref=(?<currentValue>\\d.\\d.\\d)"],
      "datasourceTemplate": "git-tags"
    }
  ]

View full answer

shaafekhan · 2024-04-24T14:41:56Z

github-actions[bot]
bot Apr 24, 2024

Hi there,

This issue or discussion is missing some logs, making it difficult or impossible to help you. Depending on which situation applies follow one, some or all of these instructions.

No logs at all

If you haven't posted any log yet, we need you to find and copy/paste the log into the issue template.

Finding logs on hosted app

Select me to read instructions

If you use the Mend Renovate app (GitHub):

Log in to the Mend Developer Portal
Navigate to the correct organization and repository
Locate the appropriate log (it may not always be the latest one)
Copy/paste the log contents
Follow the steps in the formatting your logs section

Finding logs when self-hosting

Select me to read instructions

Read the Renovate docs, troubleshooting, self-hosted to learn how to find the logs.

Insufficient logs

Select me to read instructions

If you already gave us a log, and the Renovate team said it's not enough, then follow the instructions from the No logs at all section.

Formatting your logs

Select me to read instructions

Please put your logs in a <details> and <summary> element like this:

<details><summary>Select me to see logs</summary>

```
Copy/paste your logs here, between the starting and ending backticks
```

</details>

If you feel the logs are too large to paste here, please use a service like GitHub Gist and paste the link here.

Good luck,

The Renovate team

1 reply

shaafekhan Apr 24, 2024
Author

added full logs

shaafekhan · 2024-04-30T09:04:16Z

shaafekhan
Apr 30, 2024
Author

Ended up figuring it out myself. solution (in renovate.json):

 "customManagers": [
    {
      "customType": "regex",
      "fileMatch": ["\\.tf$"],
      "matchStrings": ["source.=.\\\"(?<depName>.*?)\\?ref=(?<currentValue>\\d.\\d.\\d)"],
      "datasourceTemplate": "git-tags"
    }
  ]

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to have the terraform manager search bitbucket or git tags for a bitbucket git URL terraform module source? #28620

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 2 comments 1 reply

{{title}}

{{title}}

{{title}}

Select a reply

How to have the terraform manager search bitbucket or git tags for a bitbucket git URL terraform module source? #28620

shaafekhan Apr 24, 2024

What would you like help with?

How are you running Renovate?

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

Please tell us more about your question or problem

Logs (if relevant)

Replies: 2 comments · 1 reply

github-actions[bot] bot Apr 24, 2024

No logs at all

Finding logs on hosted app

Finding logs when self-hosting

Insufficient logs

Formatting your logs

shaafekhan Apr 24, 2024 Author

shaafekhan Apr 30, 2024 Author

shaafekhan
Apr 24, 2024

Replies: 2 comments 1 reply

github-actions[bot]
bot Apr 24, 2024

shaafekhan Apr 24, 2024
Author

shaafekhan
Apr 30, 2024
Author