Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature Request: Optional delete #23

Open
taylorsmithgg opened this issue May 9, 2022 · 2 comments
Open

Feature Request: Optional delete #23

taylorsmithgg opened this issue May 9, 2022 · 2 comments

Comments

@taylorsmithgg
Copy link

Our use case is fairly unique, but I feel like this could also be helpful for other edge cases.

Currently, the https://github.com/Nordstrom/kafka-connect-sqs/blob/master/src/main/java/com/nordstrom/kafka/connect/sqs/SqsSourceConnectorTask.java#L115 class is set to always delete, which can make both testing and monitoring solutions difficult.

For our case specifically, we would like to pseduo-replicate our SQS messages into kafka for analysis to gain visibility with SQL where SQS leaves us fairly blind and forces us to rely on logging or tracing.

If this were to delete the record on the consumer, this would cause production issues for us.

A simple feature flag or environment variable would be great.

Happy to provide a PR

@dylanmei
Copy link
Contributor

A configuration option to control the behavior is welcome!

@taylorsmithgg
Copy link
Author

Implemented here:
govyrl@674d71b

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants