Speed up consumer group creation when running tests #6530
Jacqopb
started this conversation in
Show and tell
Replies: 3 comments 2 replies
-
|
cc: @Feediver1 |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
I think we can dump this into mode=dev-container, as well. cc @r-vasquez |
Beta Was this translation helpful? Give feedback.
2 replies
-
|
good thinkin @twmb - if we do i think we need to update the docs, so the dockerfile shows how to use mode=dev-container |
Beta Was this translation helpful? Give feedback.
-
|
cc: @micheleRP |
Beta Was this translation helpful? Give feedback.
-
|
I'm also trying to speed up my unit test. In my case initialise a new tropic and a new consumer group for every test: to fully isolate. The code can be found here: twmb/franz-go#208 I've tried the configuration as described and successfully removed the "3s delay" here but unfortunately I'm now running into another "suspiciously consistent" delay of 2s. The "create_topic_timeout_ms" seems to be the obvious candidate but changing that doesn't solve it. The weird thing is that when I start the Redpanda Docker container initially my tests always runs fast (on the first run). Afterwards it runs with the 2-second delay. The only hint I have is the following log line that only shows on the "slow" test: But I'm only running a single node. In a local container. I've tried the dial down all kinds of timeouts, to no avail. This is my current config: The logs of my unit test look like this on a fast run (the first one): And like this on a slow run: As you can tell from the logs I'm using the |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
When I started using Redpanda in my test-suites, I noticed there was a 3s delay when creating new consumer groups. After some digging (and asking in the community Slack channel) I learned there's a
group_initial_rebalance_delayconfiguration which defaults to 3000ms (the documentation @ https://docs.redpanda.com/docs/reference/tunable-properties/ says 300ms, but that is incorrect).I assume 3s is a reasonable value to use in production, but when running locally for test/dev on a single node system, there's no reason why you can't use a lower value.
I'm running Redpanda in docker using docker-compose and my solution was the following.
./redpanda/Dockerfileto build the serviceBeta Was this translation helpful? Give feedback.
All reactions