Renovate merging faulty branches despite having ignoreTests=false

[ivanalmendros]

What would you like help with?

I think I found a bug

How are you running Renovate?

None

If you're self-hosting Renovate, tell us which platform (GitHub, GitLab, etc) and which version of Renovate.

renovate:37-full

Please tell us more about your question or problem

We have a multibranch pipeline setup to allow renovate branches to run our stages before merging to our main branch to avoid any faulty merges from happening. This has been working pretty decently for some time now but recently we discovered some unwanted behaviour which we have not been able to find the reason to yet.

It seems like the renovate bot is merging pull requests even when the branch build of the changes it is about to merge is in a faulty status.

From our first analysis, we have seen that for the cases where this behaviour has occurred, for some reason the branch build was not listed among the build status section in Bitbucket for the PR that was merged, but if we navigated to the pipeline for that branch build we could see that it was indeed a failure.
However, in the cases where a PR was merged and the build was 'green', the branch build successful mark did show on the list of builds in the Bitbucket page for that PR.

We have done some testing since we noticed this behaviour and it seems like the bot will act as if all checks are green when there's a branch build running at the moment of the bot cronJob execution, even though that build might end up failing and hence causing a faulty merge to main. In other words, just to clarify: it seems like when a build is running; the check that is done in the build status feature of bitbucket does not have the multibranch build listed at all, so it goes on and merges it.

Any help or suggestions would be much appreciated. Any other thoughts or questions about our setup, just let me know I'll be happy to provide that info.

Thanks

Logs (if relevant)

Logs

Replace this text with your logs, between the starting and ending triple backticks

[github-actions[bot]]

Hi there,

This issue or discussion is missing some logs, making it difficult or impossible to help you. Depending on which situation applies follow one, some or all of these instructions.

No logs at all

If you haven't posted any log yet, we need you to find and copy/paste the log into the issue template.

Finding logs on hosted app

Select me to read instructions

If you use the Mend Renovate app (GitHub):

1. Log in to the Mend Developer Portal
2. Navigate to the correct organization and repository
3. Locate the appropriate log (it may not always be the latest one)
4. Copy/paste the log contents
5. Follow the steps in the formatting your logs section

Finding logs when self-hosting

Select me to read instructions

Read the Renovate docs, troubleshooting, self-hosted to learn how to find the logs.

Insufficient logs

Select me to read instructions

If you already gave us a log, and the Renovate team said it's not enough, then follow the instructions from the No logs at all section.

Formatting your logs

Select me to read instructions

Please put your logs in a <details> and <summary> element like this:

<details><summary>Select me to see logs</summary>

```
Copy/paste your logs here, between the starting and ending backticks
```

</details>

If you feel the logs are too large to paste here, please use a service like GitHub Gist and paste the link here.

Good luck,

The Renovate team

[rarkins]

Renovate should not merge anything if any checks are pending or failed. Please locate the debug logs at the time of automerge to see what the logs say.

[slowtarget]

I'll see what I can do to get those.

We are using bitbucket cloud and jenkins - my current understanding is that when a jenkins build is in progress against a renovate managed rebased pr and a new renovate job is run, it will merge that pr - understanding that the status of the job is not a failure (as it is in progress).