Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

stripping metadata from dataclass types (#626) #6177

Merged

Conversation

hamersaw
Copy link
Contributor

@hamersaw hamersaw commented Jan 17, 2025

Tracking issue

N/A

Why are the changes needed?

Metadata is stripped from all other places in the interface(s). It is unnecessary for type checking and only attributes to bloating the CRD.

What changes were proposed in this pull request?

Stripping metadata from dataclass types.

How was this patch tested?

Tested under a variety of workflows and

Labels

  • changed: For changes in existing functionality.

Setup process

N/A

Screenshots

N/A

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

N/A

Docs link

N/A

Summary by Bito

Implementation of metadata stripping functionality for dataclass types in Flyte compiler, modifying StripTypeMetadata function to handle dataclass types and replacing direct struct field access with getter methods. The changes focus on removing unnecessary metadata and reducing CRD bloat while maintaining type safety, with updates to test coverage to verify the functionality.

Unit tests added: True

Estimated effort to review (1-5, lower is better): 1

* stripping metadata from dataclass types

Signed-off-by: Daniel Rammer <[email protected]>

* actually setting on Structure

Signed-off-by: Daniel Rammer <[email protected]>

* added unit test

Signed-off-by: Daniel Rammer <[email protected]>

---------

Signed-off-by: Daniel Rammer <[email protected]>
@hamersaw hamersaw added the changed For changes in existing functionality label Jan 17, 2025
@flyte-bot
Copy link
Collaborator

flyte-bot commented Jan 17, 2025

Code Review Agent Run #59e151

Actionable Suggestions - 1
  • flytepropeller/pkg/compiler/transformers/k8s/utils_test.go - 1
    • Consider simplifying nested structure initialization · Line 262-271
Review Details
  • Files reviewed - 2 · Commit Range: 0f1e654..0f1e654
    • flytepropeller/pkg/compiler/transformers/k8s/utils.go
    • flytepropeller/pkg/compiler/transformers/k8s/utils_test.go
  • Files skipped - 0
  • Tools
    • Golangci-lint (Linter) - ✖︎ Failed
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

AI Code Review powered by Bito Logo

Copy link

codecov bot commented Jan 17, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 37.06%. Comparing base (729e71a) to head (bcaa86f).
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #6177   +/-   ##
=======================================
  Coverage   37.05%   37.06%           
=======================================
  Files        1318     1318           
  Lines      132638   132644    +6     
=======================================
+ Hits        49151    49163   +12     
+ Misses      79237    79231    -6     
  Partials     4250     4250           
Flag Coverage Δ
unittests-datacatalog 51.58% <ø> (ø)
unittests-flyteadmin 54.33% <ø> (+0.02%) ⬆️
unittests-flytecopilot 30.99% <ø> (ø)
unittests-flytectl 62.29% <ø> (ø)
unittests-flyteidl 7.23% <ø> (ø)
unittests-flyteplugins 53.85% <ø> (ø)
unittests-flytepropeller 42.70% <100.00%> (+0.01%) ⬆️
unittests-flytestdlib 55.27% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@flyte-bot
Copy link
Collaborator

flyte-bot commented Jan 17, 2025

Changelist by Bito

This pull request implements the following key changes.

Key Change Files Impacted
Feature Improvement - Enhanced Type Metadata Handling

utils.go - Added metadata stripping functionality for dataclass types

utils_test.go - Updated test cases to verify dataclass type metadata stripping

Comment on lines +262 to +271
Structure: &core.TypeStructure{
DataclassType: map[string]*core.LiteralType{
"foo": {
Type: &core.LiteralType_Simple{
Simple: core.SimpleType_STRING,
},
},
},
Tag: "str",
},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider simplifying nested structure initialization

Consider simplifying the nested structure by using a more concise initialization pattern for TypeStructure. The current nested structure with multiple levels of indentation could be made more readable. A similar issue was also found in flytepropeller/pkg/compiler/transformers/k8s/utils_test.go (line 227-245).

Code suggestion
Check the AI-generated fix before applying
 // Add helper function
 func createTypeStructure(key string, simpleType core.SimpleType, tag string) *core.TypeStructure {
     return &core.TypeStructure{
         DataclassType: map[string]*core.LiteralType{
             key: {
                 Type: &core.LiteralType_Simple{Simple: simpleType},
             },
         },
         Tag: tag,
     }
 }

 // Use helper function in test
 Structure: createTypeStructure("foo", core.SimpleType_STRING, "str"),

Code Review Run #59e151


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

Signed-off-by: Daniel Rammer <[email protected]>
@flyte-bot
Copy link
Collaborator

flyte-bot commented Jan 17, 2025

Code Review Agent Run #03ee88

Actionable Suggestions - 0
Additional Suggestions - 1
  • flytepropeller/pkg/compiler/transformers/k8s/utils.go - 1
Review Details
  • Files reviewed - 1 · Commit Range: 0f1e654..bcaa86f
    • flytepropeller/pkg/compiler/transformers/k8s/utils.go
  • Files skipped - 0
  • Tools
    • Golangci-lint (Linter) - ✖︎ Failed
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

AI Code Review powered by Bito Logo

@Future-Outlier Future-Outlier merged commit ab463c3 into master Jan 17, 2025
53 checks passed
@Future-Outlier Future-Outlier deleted the union/upstream-fb9dce3257974eea378328f2ab78a26c644adeaf branch January 17, 2025 08:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changed For changes in existing functionality
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants