Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve handling of Gist URLs #2653

Merged
merged 3 commits into from
May 24, 2024

Conversation

rgmz
Copy link
Contributor

@rgmz rgmz commented Apr 2, 2024

Description:

This is a follow-up to #2625. It fixes #2640 (at least based on my testing) and should allow scanning of gists with GHES, which was highlighted in #2640 (comment).

Checklist:

  • Tests passing (make test-community)?
  • Lint passing (make lint this requires golangci-lint)?

@rgmz rgmz requested a review from a team as a code owner April 2, 2024 13:58
@rgmz rgmz force-pushed the fix/github-enterprise-gist branch 3 times, most recently from 730ec6d to 75a23ef Compare April 2, 2024 15:59
case 3:
// Do nothing
case 4:
if !(!strings.EqualFold(urlParts[0], "github.com") && strings.EqualFold(urlParts[1], "gist")) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there's an extra ! in there and it should be:

  !(strings.EqualFold(urlParts[0], "github.com") && strings.EqualFold(urlParts[1], "gist"))
// ^^ removed this one

A gist segment is only expected for the github.com host, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is difficult to grok in its current state, I'll need to change it — or at least add a comment. 😵

I think there's an extra ! in there and it should be:

  !(strings.EqualFold(urlParts[0], "github.com") && strings.EqualFold(urlParts[1], "gist"))
// ^^ removed this one

A gist segment is only expected for the github.com host, right?

It's actually the opposite. GitHub uses the subdomain gist.github.com for gists, and although github.com/gist redirects -> gist.github.com, it isn't a valid gist URL (e.g., https://github.com/gist/280192b0523099d0614a95f579d99aa9 and https://github.com/gist/rgmz/280192b0523099d0614a95f579d99aa9). However, /gist/<id> can be valid for instances of GitHub Enterprise Server (e.g., https://github.company.org/gist/nat/5fdbb7f945d121f197fb074578e53948.git).

The logic is meant to accept a gist segment for any domain except github.com.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if @joeleonjr has anything to add about Gist URLs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is difficult to grok in its current state, I'll need to change it — or at least add a comment. 😵

I did both. Let me know if that looks better.

@rgmz rgmz force-pushed the fix/github-enterprise-gist branch 2 times, most recently from 1c3ddff to cbe808f Compare April 14, 2024 14:21
@rgmz rgmz force-pushed the fix/github-enterprise-gist branch 2 times, most recently from 5ff26ff to a4dca22 Compare May 2, 2024 12:49
@rgmz rgmz force-pushed the fix/github-enterprise-gist branch from a4dca22 to d009666 Compare May 22, 2024 20:01
Copy link
Collaborator

@ahrav ahrav left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 non-blocking question, otherwise LGTM! Thanks for updating this, the switch make this a lot easier to grok.

pkg/sources/github/github.go Outdated Show resolved Hide resolved
pkg/sources/github/github.go Outdated Show resolved Hide resolved
@rgmz rgmz force-pushed the fix/github-enterprise-gist branch from d009666 to 7ef3f2e Compare May 24, 2024 13:48
@ahrav ahrav merged commit e53f5bd into trufflesecurity:main May 24, 2024
11 of 12 checks passed
@rgmz rgmz deleted the fix/github-enterprise-gist branch May 24, 2024 15:37
haraldh referenced this pull request in matter-labs/vault-auth-tee Jun 10, 2024
[![Mend
Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Type | Update | Change |
|---|---|---|---|
|
[trufflesecurity/trufflehog](https://togithub.com/trufflesecurity/trufflehog)
| action | minor | `v3.76.3` -> `v3.78.0` |

---

### Release Notes

<details>
<summary>trufflesecurity/trufflehog
(trufflesecurity/trufflehog)</summary>

###
[`v3.78.0`](https://togithub.com/trufflesecurity/trufflehog/releases/tag/v3.78.0)

[Compare
Source](https://togithub.com/trufflesecurity/trufflehog/compare/v3.77.0...v3.78.0)

#### What's Changed

- Add postman to tui by [@&#8203;hxnyk](https://togithub.com/hxnyk) in
[https://github.com/trufflesecurity/trufflehog/pull/2895](https://togithub.com/trufflesecurity/trufflehog/pull/2895)
- chore(deps): update alpine docker tag to v3.20 by
[@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2874](https://togithub.com/trufflesecurity/trufflehog/pull/2874)
- fix(deps): update golang.org/x/exp digest to
[`fd00a4e`](https://togithub.com/trufflesecurity/trufflehog/commit/fd00a4e)
by [@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2899](https://togithub.com/trufflesecurity/trufflehog/pull/2899)
- Update metadata for DataDog for API + APPKey by
[@&#8203;abmussani](https://togithub.com/abmussani) in
[https://github.com/trufflesecurity/trufflehog/pull/2879](https://togithub.com/trufflesecurity/trufflehog/pull/2879)
- Consistent docker image of MSSQL for integration testing. by
[@&#8203;abmussani](https://togithub.com/abmussani) in
[https://github.com/trufflesecurity/trufflehog/pull/2898](https://togithub.com/trufflesecurity/trufflehog/pull/2898)
- fix(deps): update module github.com/charmbracelet/bubbletea to v0.26.4
by [@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2885](https://togithub.com/trufflesecurity/trufflehog/pull/2885)
- Remove 'www' from `DefaultFalsePositives` by
[@&#8203;rgmz](https://togithub.com/rgmz) in
[https://github.com/trufflesecurity/trufflehog/pull/2896](https://togithub.com/trufflesecurity/trufflehog/pull/2896)
- Redis integration test by
[@&#8203;abmussani](https://togithub.com/abmussani) in
[https://github.com/trufflesecurity/trufflehog/pull/2901](https://togithub.com/trufflesecurity/trufflehog/pull/2901)
- Fix Github `enumerateWithToken` test failure by
[@&#8203;rgmz](https://togithub.com/rgmz) in
[https://github.com/trufflesecurity/trufflehog/pull/2880](https://togithub.com/trufflesecurity/trufflehog/pull/2880)
- fix(deps): update module github.com/aws/aws-sdk-go to v1.53.14 by
[@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2900](https://togithub.com/trufflesecurity/trufflehog/pull/2900)
- fix(deps): update module github.com/hashicorp/go-retryablehttp to
v0.7.7 by [@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2904](https://togithub.com/trufflesecurity/trufflehog/pull/2904)
- integration testing for mongodb. by
[@&#8203;abmussani](https://togithub.com/abmussani) in
[https://github.com/trufflesecurity/trufflehog/pull/2907](https://togithub.com/trufflesecurity/trufflehog/pull/2907)
- fix(deps): update module
github.com/azure/go-autorest/autorest/azure/auth to v0.5.13 by
[@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2902](https://togithub.com/trufflesecurity/trufflehog/pull/2902)
- chore: fix some comments by
[@&#8203;jinjiadu](https://togithub.com/jinjiadu) in
[https://github.com/trufflesecurity/trufflehog/pull/2903](https://togithub.com/trufflesecurity/trufflehog/pull/2903)
- \[chore] Always log git repositories being scanned by
[@&#8203;mcastorina](https://togithub.com/mcastorina) in
[https://github.com/trufflesecurity/trufflehog/pull/2909](https://togithub.com/trufflesecurity/trufflehog/pull/2909)
- Add Jenkins scanning by
[@&#8203;dustin-decker](https://togithub.com/dustin-decker) in
[https://github.com/trufflesecurity/trufflehog/pull/2892](https://togithub.com/trufflesecurity/trufflehog/pull/2892)

#### New Contributors

- [@&#8203;jinjiadu](https://togithub.com/jinjiadu) made their first
contribution in
[https://github.com/trufflesecurity/trufflehog/pull/2903](https://togithub.com/trufflesecurity/trufflehog/pull/2903)

**Full Changelog**:
trufflesecurity/trufflehog@v3.77.0...v3.78.0

###
[`v3.77.0`](https://togithub.com/trufflesecurity/trufflehog/releases/tag/v3.77.0)

[Compare
Source](https://togithub.com/trufflesecurity/trufflehog/compare/v3.76.3...v3.77.0)

#### What's Changed

- Remove "finished verificationOverlap chunks" log line by
[@&#8203;rgmz](https://togithub.com/rgmz) in
[https://github.com/trufflesecurity/trufflehog/pull/2860](https://togithub.com/trufflesecurity/trufflehog/pull/2860)
- fix(deps): update module github.com/wasilibs/go-re2 to v1.5.3 by
[@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2861](https://togithub.com/trufflesecurity/trufflehog/pull/2861)
- fix(deps): update module google.golang.org/api to v0.181.0 by
[@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2857](https://togithub.com/trufflesecurity/trufflehog/pull/2857)
- fix(deps): update module github.com/aws/aws-sdk-go to v1.53.5 by
[@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2859](https://togithub.com/trufflesecurity/trufflehog/pull/2859)
- Update azure storage extra data by
[@&#8203;abmussani](https://togithub.com/abmussani) in
[https://github.com/trufflesecurity/trufflehog/pull/2808](https://togithub.com/trufflesecurity/trufflehog/pull/2808)
- Update regex for Organization in Azure Devops detector by
[@&#8203;abmussani](https://togithub.com/abmussani) in
[https://github.com/trufflesecurity/trufflehog/pull/2866](https://togithub.com/trufflesecurity/trufflehog/pull/2866)
- fix(deps): update module github.com/aws/aws-sdk-go to v1.53.6 by
[@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2867](https://togithub.com/trufflesecurity/trufflehog/pull/2867)
- \[chore] - Use http.NewRequestWithContext by
[@&#8203;ahrav](https://togithub.com/ahrav) in
[https://github.com/trufflesecurity/trufflehog/pull/2870](https://togithub.com/trufflesecurity/trufflehog/pull/2870)
- adding Groq detector by [@&#8203;0x1](https://togithub.com/0x1) in
[https://github.com/trufflesecurity/trufflehog/pull/2873](https://togithub.com/trufflesecurity/trufflehog/pull/2873)
- Log reasons for GitLab repo exclusion by
[@&#8203;rosecodym](https://togithub.com/rosecodym) in
[https://github.com/trufflesecurity/trufflehog/pull/2875](https://togithub.com/trufflesecurity/trufflehog/pull/2875)
- \[github] Scan user repositories by
[@&#8203;rgmz](https://togithub.com/rgmz) in
[https://github.com/trufflesecurity/trufflehog/pull/2814](https://togithub.com/trufflesecurity/trufflehog/pull/2814)
- Elastic adapter by [@&#8203;camgunz](https://togithub.com/camgunz) in
[https://github.com/trufflesecurity/trufflehog/pull/2727](https://togithub.com/trufflesecurity/trufflehog/pull/2727)
- Improve handling of Gist URLs by
[@&#8203;rgmz](https://togithub.com/rgmz) in
[https://github.com/trufflesecurity/trufflehog/pull/2653](https://togithub.com/trufflesecurity/trufflehog/pull/2653)
- Fix some GitHub source test issues by
[@&#8203;rgmz](https://togithub.com/rgmz) in
[https://github.com/trufflesecurity/trufflehog/pull/2774](https://togithub.com/trufflesecurity/trufflehog/pull/2774)
- fix(deps): update module github.com/aws/aws-sdk-go to v1.53.10 by
[@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2871](https://togithub.com/trufflesecurity/trufflehog/pull/2871)
- fix(deps): update module github.com/go-logr/logr to v1.4.2 by
[@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2869](https://togithub.com/trufflesecurity/trufflehog/pull/2869)
- fix(deps): update module cloud.google.com/go/secretmanager to v1.13.1
by [@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2884](https://togithub.com/trufflesecurity/trufflehog/pull/2884)
- fix(deps): update golang.org/x/exp digest to
[`4c93da0`](https://togithub.com/trufflesecurity/trufflehog/commit/4c93da0)
by [@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2883](https://togithub.com/trufflesecurity/trufflehog/pull/2883)
- fix(deps): update module github.com/elastic/go-elasticsearch/v8 to
v8.13.1 by [@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2886](https://togithub.com/trufflesecurity/trufflehog/pull/2886)
- fix(deps): update module github.com/gabriel-vasile/mimetype to v1.4.4
by [@&#8203;renovate](https://togithub.com/renovate) in
[https://github.com/trufflesecurity/trufflehog/pull/2890](https://togithub.com/trufflesecurity/trufflehog/pull/2890)
- Added extra data for LaunchDarkly by
[@&#8203;abmussani](https://togithub.com/abmussani) in
[https://github.com/trufflesecurity/trufflehog/pull/2836](https://togithub.com/trufflesecurity/trufflehog/pull/2836)
- feat: support docker image history scanning by
[@&#8203;jamestelfer](https://togithub.com/jamestelfer) in
[https://github.com/trufflesecurity/trufflehog/pull/2882](https://togithub.com/trufflesecurity/trufflehog/pull/2882)

#### New Contributors

- [@&#8203;camgunz](https://togithub.com/camgunz) made their first
contribution in
[https://github.com/trufflesecurity/trufflehog/pull/2727](https://togithub.com/trufflesecurity/trufflehog/pull/2727)
- [@&#8203;jamestelfer](https://togithub.com/jamestelfer) made their
first contribution in
[https://github.com/trufflesecurity/trufflehog/pull/2882](https://togithub.com/trufflesecurity/trufflehog/pull/2882)

**Full Changelog**:
trufflesecurity/trufflehog@v3.76.3...v3.77.0

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined),
Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you
are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the
rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update
again.

---

- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check
this box

---

This PR has been generated by [Mend
Renovate](https://www.mend.io/free-developer-tools/renovate/). View
repository job log
[here](https://developer.mend.io/github/matter-labs/vault-auth-tee).

<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zNzcuOCIsInVwZGF0ZWRJblZlciI6IjM3LjM4OC4xIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

Gist Scanning Errors
3 participants