[flytekit/small feature] Allow pyflyte run to execute reference entities

[wild-endeavor]

Issue

If you have a file like so

from flytekit import task, workflow

@task
def base_list_adder(x: list[int], y: list[int]) -> list[int]:
    return [a + b for a, b in zip(x, y)]

@workflow
def base_lists_wf(x: list[int], y: list[int]):
    base_list_adder(x=x, y=y)

and another file that invokes it as a reference workflow

from flytekit import reference_workflow

@reference_workflow(project="flytesnacks", domain="development", name="yt_dbg.scratchpad.ref_base.base_lists_wf", version="8jE3QH2_lkxcA6YIsRgQrg")
def base_lists_wf(x: list[int], y: list[int]):
    ...

and the first file is registered with the ID fields in the second, running on cli currently fails

pyflyte -vv -c ~/.flyte/config-sandbox.yaml run --remote using_references.py base_lists_wf --x "[1,2,3]" --y "[1,2,3]"

To fix:

First step is to just get the script to return the flyte entity
https://github.com/flyteorg/flytekit/compare/run-ref-entities?expand=1

(need to handle tasks/launch plans as well ofc).

Need to think about the usage of default inputs. This example shows a reference workflow, not a reference launch plan.
When FlyteRemote tries to execute a workflow, it will first fetch the default launch plan, and then run that. What if the default launch plan was registered with inputs different than the ones specified in the signature of the reference workflow? I think the reference workflow one should win (does it? need to check). Is it worth it to print a log line?

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[machichima]

#take