[BUG] Status checks are not reported in a recently configured project — ansible/awx

[webknjaz]

Describe the bug

$sbj, basically. I've configured yet another project the same way as many before. The config file diff is the same except for the token and folder names. The GitHub App is installed and has access to the repo.
However, no status checks are being posted to GitHub. Neither through the Checks API (or Statuses API), nor thought comments in PRs.

The reports are being uploaded and show up in the Codecov dashboard, though.

Environment (please complete the following information):

• Browser [e.g. chrome, safari] (if applicable)
• System (e.g., local machine, CI, etc) (if applicable)

To Reproduce
Steps to reproduce the behavior:

1. Go to https://github.com/ansible/awx
2. See no status information posted by the Codecov bot

Expected behavior

The statuses should appear in PR and regular commit-related checks pages.

Screenshots

N/A

Additional context

I thought this might be related to codecov/codecov-cli#500 but it seems it's not.

cc @rohan-at-sentry is this the right place for a bug report?

[drazisil-codecov]

Hi @webknjaz

/repos/ansible/awx/statuses/c9ae36804a1cac937307f0f9df4269c7985c1957
{"message":"Not Found","documentation_url":"https://docs.github.com/rest/commits/statuses#create-a-commit-status","status":"404"}

It looks like Ansible has Codecov configured to ignore the GitHub App, and uae the creds of carsongee instead. Does that user have write access to this repo?

[webknjaz]

Woah! Thanks for looking @drazisil-codecov! I don't even know who that is 🤯 Are you saying that it's this specific repository that's blocked? I thought it could be https://docs.github.com/en/apps/maintaining-github-apps/suspending-a-github-app-installation.
I don't have enough privileges in the repo, but I'm looking into it with somebody who has. The app is not suspended, and ansible/awx is listed among the repos the app has access to. Also, we don't think that @carsongee ever worked for Ansible, so we're quite puzzled as to how their account is being pulled in.

[drazisil-codecov]

The more I look, the more puzzled I an as well. Ansible has been with Codecov almost as long as Codecov has existed (relatively)

I'm wondering if this is a VERY OLD setting that we are just now looking at due to a recent change. Looking...

[drazisil-codecov]

@webknjaz Can you double check the app settings? I'm not showing awx as one of the repos it's allowed to access.

[carsongee]

Hi! I've been summoned from a very long time ago. I believe it's been about 9-10 years since I was a core maintainer for that part of Ansible. I must have configured it ages ago. Let me know if there's anything I can do to help unstick things

[webknjaz]

@drazisil-codecov

I'm wondering if this is a VERY OLD setting that we are just now looking at due to a recent change. Looking...

When you say setting, you mean how the GitHub App is installed, right? (If not — please clarify what you mean, please.) GitHub Apps became a thing in 2018. What are you looking into exactly? Does the Codecov GH App only record what it can access on webhooks or does it also query the API to get this information? GitHub's webhooks aren't guaranteed to be delivered. Sometimes they get lost in the network and GitHub never auto-attempts re-deliveries.
If you only rely on the webhooks, that would explain the inconsistency in your DB, I suppose.

@webknjaz Can you double check the app settings? I'm not showing awx as one of the repos it's allowed to access.

We need to ask @thedoubl3j or @TheRealHaoLiu to do that — they should have enough access to the GH org settings page.

FWIW, I requested installing the app into ansible/awx via GH UI about a week ago and then, one of the admins approved it so it's listed among the repositories that the GH App can access.

Hi! I've been summoned from a very long time ago. I believe it's been about 9-10 years since I was a core maintainer for that part of Ansible. I must have configured it ages ago. Let me know if there's anything I can do to help unstick things

@carsongee good to know. Thanks for answering the call! I'm not sure what you can do currently since in the Codecov dashboard you're not listed as an admin. Perhaps log into codecov.io and see if that would re-sync the privilege info it has...

[carsongee]

I tried logging in, but it's been so long not even a password recovery will work. I logged in with my GitHub creds, but AWX doesn't show as a repo (or any other Ansible repos as far as I can tell). Sorry, I couldn't be of much more help!

[webknjaz]

@carsongee no problem, thanks for trying! Please, feel free to unsubscribe so we don't spam you :)

[thedoubl3j]

@webknjaz @drazisil-codecov I am seeing it listed

[drazisil-codecov]

Ok, thanks everyone! @webknjaz , I've cleared the bot, since we now know how it was set in the first place. Can you please try with a fresh commit and share the SHA if you still see issues?

[webknjaz]

Thanks! I've rebased a bunch of PRs and will report back once I start seeing CI jobs finishing their runs.

[webknjaz]

@drazisil-codecov I restarted all the PR CIs in this list https://github.com/ansible/awx/pulls?q=sort%3Aupdated-desc+is%3Apr+is%3Aopen+author%3Awebknjaz and none of them reported a status check.

[drazisil-codecov]

@webknjaz did you rebase to commits that Codecov has already seen? If so, you need to create fresh ones, sorry.

[webknjaz]

You're right. I worked once some merges rolled into devel. Thanks!