[FlyteCopilot] Binary IDL Attribute Access Primitive Input

[Future-Outlier]

Tracking issue

#5318

Why are the changes needed?

We need to support attribute access for dataclass when it is a msgpack binary IDL object to a raw container task.

Example:

@workflow
def wf(dc: DC) -> Tuple[int, bool, float, str]:
    return python_return_same_values(a=dc.a, b=dc.b, c=dc.c, d=dc.d)

python_return_same_values = ContainerTask(
    name="python_return_same_values",
    inputs=kwtypes(a=int, b=bool, c=float, d=str),
    outputs=kwtypes(a=int, b=bool, c=float, d=str),
    image="futureoutlier/rawcontainer:0320",
    command=[
        "python", "return_same_value.py",
        "{{.inputs.a}}", "{{.inputs.b}}", "{{.inputs.c}}", "{{.inputs.d}}",
        "/var/outputs",
    ],
)

What does this PR change?

It adds logic to deserialize the msgpack binary IDL object and convert it into a string for the container task input, similar to how we handle primitive types.

Related Code: template.go#L175-L192

How was this tested?

• Unit tests
• Remote execution

Example:

import logging
from dataclasses import dataclass
from typing import Tuple
from flytekit import ContainerTask, kwtypes, workflow
logger = logging.getLogger(__file__)

@dataclass
class DC:
    a: int = 1
    b: bool = True
    c: float = 1.1
    d: str = "string"

@workflow
def wf(dc: DC) -> Tuple[int, bool, float, str]:
    return python_return_same_values(a=dc.a, b=dc.b, c=dc.c, d=dc.d)

python_return_same_values = ContainerTask(
    name="python_return_same_values",
    input_data_dir="/var/inputs",
    output_data_dir="/var/outputs",
    inputs=kwtypes(a=int, b=bool, c=float, d=str),
    outputs=kwtypes(a=int, b=bool, c=float, d=str),
    image="futureoutlier/rawcontainer:0320",
    command=[
        "python",
        "return_same_value.py",
        "{{.inputs.a}}",
        "{{.inputs.b}}",
        "{{.inputs.c}}",
        "{{.inputs.d}}",
        "/var/outputs",
    ],
)

if __name__ == "__main__":
    from flytekit.clis.sdk_in_container import pyflyte
    from click.testing import CliRunner
    import os

    runner = CliRunner()
    path = os.path.realpath(__file__)
    input_val = '{"a": 1, "b": true, "c": 1.0, "d": "string"}'

    result = runner.invoke(pyflyte.main,
                           ["run", path, "wf", "--dc", input_val])
    print("Local Execution: ", result.output)

    result = runner.invoke(pyflyte.main,
                           ["run", "--remote", path, "wf", "--dc", input_val])
    print("Remote Execution: ", result.output)

Setup for remote execution:

1. Build the sandbox image:

   cd docker/sandbox-bundled
   make build
   flytectl demo start --image flyte-sandbox:latest --force

2. Run the workflow example

Note: See related PR for image futureoutlier/rawcontainer:0320: flyteorg/flytekit#2258

Checklist

• Documentation updated
• All tests passed
• Commits signed-off

Screenshots

Check all the applicable boxes

• I updated the documentation accordingly.
• All new and existing tests passed.
• All commits are signed-off.

[codecov]

Codecov Report

Attention: Patch coverage is 77.77778% with 2 lines in your changes missing coverage. Please review.

Project coverage is 36.36%. Comparing base (f2a0619) to head (c3a1d9c).
Report is 154 commits behind head on master.

Files with missing lines	Patch %	Lines
...go/tasks/pluginmachinery/core/template/template.go	77.77%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5850      +/-   ##
==========================================
+ Coverage   36.33%   36.36%   +0.02%     
==========================================
  Files        1304     1304              
  Lines      110137   110154      +17     
==========================================
+ Hits        40023    40054      +31     
+ Misses      65946    65929      -17     
- Partials     4168     4171       +3     

Flag	Coverage Δ
unittests-datacatalog	51.37% <ø> (ø)
unittests-flyteadmin	55.57% <ø> (ø)
unittests-flytecopilot	12.17% <ø> (ø)
unittests-flytectl	62.21% <ø> (ø)
unittests-flyteidl	7.14% <ø> (ø)
unittests-flyteplugins	53.46% <77.77%> (+0.11%)	⬆️
unittests-flytepropeller	42.04% <ø> (+0.02%)	⬆️
unittests-flytestdlib	55.31% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Future-Outlier]

In this PR, datetime, timedelta, flyte directory, and flyte file are not supported, and these are not supported before, so let's ignore them now.

The hard part of this is that we have only scalar value, but not expected literal type or expected python type as our hint, that's why I can't support these in this PR.

cc @pingsutw @wild-endeavor @eapolinario

[eapolinario]

This is pretty cool.

Can you make sure to create a gh issue to track the TODO?

[Future-Outlier]

This is pretty cool.

Can you make sure to create a gh issue to track the TODO?

Yes, will do it tmr