Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new feature about CDC Handler for 1 => N(multi row) data handle #116

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

gjchoi
Copy link

@gjchoi gjchoi commented Aug 22, 2022

I'd like to develop Custom CDC Handler for N data insert with 1 sinkData.
But I can't find any base kafka connect feature about this ( kafka connect options, SMT )
And CDCHandler, CDCOperation to make custom CDC Handle below

CdcOperation

WriteModel<BsonDocument> perform(SinkDocument doc);

CdcHandler

  public abstract Optional<WriteModel<BsonDocument>> handle(SinkDocument doc);

only support 1 => 1 data handle.

so I make a PR about this new feature

  • sink connector option added ( change.data.capture.multi.row.handler )
  • CdcMultiRowOperation, CdcMultiRowHanlder added
  • modified sink task logic that return list and append to bulkWrite

Can you review this PR for new feature add?

or

Is there any other way for this I don't know?

- sink connector option added ( change.data.capture.multi.row.handler )
- CdcMultiRowOperation, CdcMultiRowHanlder added
- modified sink task logic that return list<WriteModel> and append to bulkWrite
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants