fix: exclude context canccel error from forward error counting #98
+66
−2
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
atenjin
approved these changes
Dec 3, 2024
jelias2
reviewed
Dec 11, 2024
Comment on lines
+589
to
+591
| if errors.Is(err, context.Canceled) { | ||
| return nil, err | ||
| } |
There was a problem hiding this comment.
Would it be possible to add a metric for canceled contexts here?
jelias2
reviewed
Dec 11, 2024
| @@ -20,3 +26,34 @@ func TestStripXFF(t *testing.T) { | |||
| assert.Equal(t, test.out, actual) | |||
| } | |||
| } | |||
|
|
|||
| func TestForwardContextCanceled(t *testing.T) { | |||
There was a problem hiding this comment.
Looks good, would it be possible to add a few more tests and cases such as incrementing the intermittentErrorsSlidingWindow first and then checking that they are not incremented again.
Maybe asset that the debug log exists too in the case of context canceled.
Also test the multicall routing strategy
jelias2
reviewed
Dec 11, 2024
| @@ -583,6 +585,10 @@ func (b *Backend) doForward(ctx context.Context, rpcReqs []*RPCReq, isBatch bool | |||
| start := time.Now() | |||
| httpRes, err := b.client.DoLimited(httpReq) | |||
| if err != nil { | |||
| // if it's canceld, we don't want to count it as an error | |||
|
Left a few comments, overall looks good! Thanks for the contribution @Revolution1! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Proxyd bans backends based on its error rate.
But it even counts context canceled error as a forwarding error,
while the cancellation is actually triggered by incomplete HTTP requests from the client.
This means hackers can make DoS attacks by just sendinga bunch of requests and closing without waiting for a response.
Tests
We have an environment that client sends a lot this kind of requests to test connection or some thing.
Before this patch:
the proxy will have no backends to serve in a very short time.
After:
works like a charm.
BTW:
wrapErr()will strip the error's type.It's recommended to use this package to do error wrapping and assertion.