-
Notifications
You must be signed in to change notification settings - Fork 222
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unexpected error during batch delivery #1004
Comments
After a couple of days trying to get to the bottom of this, we have resolved the issue by changing the order of sending the offsets and sending the records within the transaction, like so: async with producer.transaction():
await producer.send_offsets_to_transaction({topic_partition: commit_offsets}, self.__group_id)
for record in records_to_write:
await producer.send(record.topic, value=record.value, key=record.key) Given that this has solved the problem, it means that there is some sort of race condition within the library that is only triggered when using Kafka 3.7. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug
We are getting an "Unexpected error during batch delivery" following an upgrade to Kafka 3.7.
We are using Aiven's platform for Kafka and we upgraded to Kafka version 3.7. However, as soon as the upgrade finished, we noticed that the Transactional Producers are issuing the following errors:
The Producer seemingly still keeps going and any further attempts to send messages on the transaction end up with:
We are using a Transactional Producer with
acks
set toall
and with a settransactional.id
.Environment (please complete the following information):
Reproducible example
I have reproduced below the code we are using to issue transactional messages using the producer.
The text was updated successfully, but these errors were encountered: