No error on consumer timeout

[Tomczik76]

I'm getting these errors in my logs when subscribing to a 40 partitions.

2021-12-07 14:27:36,129INFO [Consumer clientId=consumer-test-stream-service-2-1, groupId=test-stream-service-2] Error sending fetch request (sessionId=INVALID, epoch=INITIAL) to node 1:
org.apache.kafka.common.errors.DisconnectException: null

The streams seem to just hang and no error was raised. When I reduced the number of partitions I'm subscribing to the errors go away and everything seems to work fine. I also resolved this issue by adding these configs to my consumer.

.withPollTimeout(Duration.fromScala(1.minutes))
.withProperty("request.timeout.ms", "60000")

I would expect that an error is raised in this case.

[ChallenHB]

Hi all, my team is looking to implement a workaround for this ticket.
Before going down that path, we wanted to know if this issue is set to be addressed in a future release, and if so when that might be.
Thanks!

[svroonland]

@ChallenHB There is no concrete plan for a next release, contributions to fix this issue are welcome :)

[erikvanoosten]

With some tuning zio-kafka can handle thousands of partitions. I still want to write a tuning doc. I'll get to it one day. I promise!
For now, I am going to close this very old issue.