-
Notifications
You must be signed in to change notification settings - Fork 113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Kubernetes client rate-limiting #2942
Comments
Here's what happened in my case: the provider was sent a Two possible follow-ups:
The low-level throttling code is here: |
Is there another alternative where we generously bump the QPS ceiling if running under debug? A quick workaround like that might be prudent if this is impacting the debug loop but not end-users. Related #1748 |
What happened?
I was running the kubernetes provider in a debugger, and attaching to it using
PULUMI_DEBUG_PROVIDERS
. I used the same process for numerous deployments, and eventually the provider transitioned to a failure state, apparently due to client-side rate limiting. Once I restarted the provider process, the problem was fixed.I decided to file an issue because, though my specific case is exotic, there might be a deeper scalability problem in the provider related to rate-limiting in the kube client.
See kubernetes/kubernetes#111880 for more background.
Here's the update made just prior to the first rate-limit error. I'd deliberately used an invalid image
nginxfoo
.Example
N/A
Output of
pulumi about
Additional context
No response
Contributing
Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).
The text was updated successfully, but these errors were encountered: