New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
chore: Improve networking code pt 1 #10219
Conversation
pretty soon we will want to upstream our proxy changes or at least fork cohttp. Either way we need to use the new version
Before we were assuming that cohttp doesn't blow up over random stuff. But it does. So lets improve our http helpers api to wrap these errors
PR checklist:
If you're unsure about any of this, please see: |
It looks like the new version of cohttp relies on opam 2.1, which windows doesn't have :/ so not sure if the cohttp bump part of this PR is possible. I may just bump cohttp back down a version |
semgrep-compare-github-qsndg resultsRan benchmark on 38 repositories The number of findings differs for 3 repos Whole benchmark is 1.3% faster (a bit of noise is expected) Relative speed improvement is 1.01 on average Relative memory improvement is 0.99 on average |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Overall the API updates look good. Some various questions/other minor improvements suggested.
A couple spots I would want to see more docs before merging but other stuff is mostly informative/nits.
Co-authored-by: Cooper Pierce <[email protected]>
Co-authored-by: Cooper Pierce <[email protected]>
Co-authored-by: Cooper Pierce <[email protected]>
Co-authored-by: Cooper Pierce <[email protected]>
semgrep-compare-github-e40oh resultsRan benchmark on 38 repositories The number of findings differs for 4 repos Whole benchmark is 4.0% faster (a bit of noise is expected) Relative speed improvement is 1.01 on average
Relative memory improvement is 0.99 on average |
semgrep-compare-github-p9179 resultsRan benchmark on 38 repositories The number of files checked differs for 1 repos The number of findings differs for 3 repos Whole benchmark is 0.3% slower (a bit of noise is expected) Relative speed improvement is 1.01 on average Relative memory improvement is 0.99 on average |
semgrep-compare-github-vxc5f resultsRan benchmark on 38 repositories The number of findings differs for 2 repos Whole benchmark is 2.9% slower (a bit of noise is expected) Relative speed improvement is 0.98 on average
Relative memory improvement is 0.99 on average |
semgrep-compare-github-4y5o0 resultsRan benchmark on 38 repositories The number of findings differs for 3 repos Whole benchmark is 1.3% faster (a bit of noise is expected) Relative speed improvement is 1.01 on average
Relative memory improvement is 0.99 on average |
semgrep-compare-github-7zrkd resultsRan benchmark on 38 repositories The number of findings differs for 2 repos Whole benchmark is 0.6% slower (a bit of noise is expected) Relative speed improvement is 0.98 on average
Relative memory improvement is 0.99 on average |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Glad we removed a functor! I hate functors!
Http_helpers.post ~body ~headers caps#network url | ||
match%lwt Http_helpers.post ~body ~headers caps#network url with | ||
| Ok { body = Ok body; _ } -> Lwt.return_ok body | ||
| Ok { body = Error msg; code; _ } -> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the error management seems a bit heavier now ... it's not possible to factorize
a bit so we just handle Ok body = Ok body, and then whatever else as a general error?
This PR is part 1 of 2 of improvements I plan to make to the networking library. The next PR will introduce a custom client that will (optionally) use proxies when establishing connections directly, instead of the hacky way we're doing rn that abuses Cohttp.
Changes:
Bump cohttp to 6.0.0 beta 2@akuhlens has a PR that will come out in the release version of the library, and there's been a bunch of improvements anyways, so let's just use itThe proxy changes I'm going to make I want to upstream, so might as well do them in the version they might be released in instead of writing 2 different versionsSemgrep Pro PR
https://github.com/semgrep/semgrep-proprietary/pull/1488
Test plan
Tests pass