Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add basic SASL and TLS support for Kafka cloud events #5814

Merged
merged 2 commits into from
Oct 31, 2024

Conversation

Sovietaced
Copy link
Contributor

@Sovietaced Sovietaced commented Oct 6, 2024

Tracking issue

Closes #3220

Why are the changes needed?

These changes are needed to use cloud events with kafka brokers with password or certificate auth enabled.

What changes were proposed in this pull request?

This change expands the KafkaConfig struct to include some SASL fields for password based auth as well as a TLSConfig struct for certificate based auth. It then provides a helper function to load these into the Sarama config. Lastly it adds helm support for loading the cloud events config from a secret since password based auth will contain sensitive fields.

How was this patch tested?

This was tested on our production Flyte clusters.

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Copy link

codecov bot commented Oct 6, 2024

Codecov Report

Attention: Patch coverage is 11.11111% with 32 lines in your changes missing coverage. Please review.

Project coverage is 36.83%. Comparing base (553a702) to head (6d7a27f).
Report is 8 commits behind head on master.

Files with missing lines Patch % Lines
...kg/runtime/interfaces/application_configuration.go 8.57% 29 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5814      +/-   ##
==========================================
- Coverage   36.84%   36.83%   -0.02%     
==========================================
  Files        1309     1309              
  Lines      130967   130998      +31     
==========================================
- Hits        48259    48255       -4     
- Misses      78524    78557      +33     
- Partials     4184     4186       +2     
Flag Coverage Δ
unittests-datacatalog 51.58% <ø> (ø)
unittests-flyteadmin 54.06% <11.11%> (-0.09%) ⬇️
unittests-flytecopilot 11.73% <ø> (ø)
unittests-flytectl 62.40% <ø> (ø)
unittests-flyteidl 6.92% <ø> (ø)
unittests-flyteplugins 53.64% <ø> (ø)
unittests-flytepropeller 43.00% <ø> (ø)
unittests-flytestdlib 55.41% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Sovietaced Sovietaced marked this pull request as ready for review October 6, 2024 01:33
@Sovietaced Sovietaced requested a review from pingsutw October 26, 2024 06:48
Signed-off-by: Jason Parraga <[email protected]>
@Sovietaced Sovietaced merged commit 84bcc26 into flyteorg:master Oct 31, 2024
54 of 56 checks passed
@Sovietaced Sovietaced deleted the kafka-certs branch October 31, 2024 16:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Core feature] Kafka Authentication
2 participants