Friendly Name & Renaming for Flyte Assets

[zeryx]

RFC: User Defined Names for Flyte Constructs (Workflows, Tasks, Launch Plans)

Motivation

The current naming convention in Flyte, which is based on the local filesystem path, often leads to workflow and task names that are not intuitive or descriptive. This issue hampers effective workflow identification and management. There is a pressing need to enhance this naming system to improve user experience and workflow management. Specifically, it's important to enable users to define articulate and independent names for their workflows or tasks that are not tied to the project directory structure. Moreover, ensuring global uniqueness of projects and packages is crucial to prevent accidental overwrites of workflows, especially in cases where directory structures might be identical.

Proposed Implementation

The proposal is to introduce an additional field for tasks, workflows, and launch plans in Flyte. This field would allow users to assign more descriptive and user-friendly names or nicknames at the time of registration. These names should be easily searchable and integrated within Flyte's various interfaces like uctl, flytectl, and flytekit remote. The implementation would involve:

• Adding a new field, "name" or "nickname," for tasks, workflows, and launch plans.
• Ensuring this nickname is seamlessly integrated into Flyte's tools and web dashboard for better accessibility and management.
• Maintaining the original technical name field for backend processes to ensure continuity and stability.

This approach addresses concerns similar to those raised in Flyte issue #4376, especially regarding runtime override capabilities.

Limitations

Alternative solutions considered include the manual maintenance of a separate mapping of descriptive names to the existing names, which would be cumbersome and prone to errors. Another option considered was to modify the existing naming convention to make it more descriptive. However, this could potentially disrupt existing workflows and integrations. The proposed implementation aims to strike a balance between the need for descriptive, user-defined names and the stability of existing systems.

[bstadlbauer]

Hey @zeryx!

A few thoughts, also based on what has been discussed in last week's contributors sync:

1. I think this is generally a good idea 👍
2. As @EngHabu suggested, it would probably be easiest to add this as metadata, similar to how we add a sescription (e.g. here).
3. Would we really need uniqueness of those friendly names when they are only aliases for the real ids? I would imagine that any UI would be able to show both, so in case there is a collision, the user could still look at task/workflow id.

I know there have also been talks around re-writing/structuring the submitted code tarball, which is something I'd be strongly against doing as that would get us into all kinds of awkward situations. I would see this more as additional metadata attached to a registered entity. This view would also allow for different friendly names per workflow version, which is something that I have not fully thought through.

Also, as suggested in the contributors sync, a small frontend change which could already easy some of the problems could be to just show the non-common parts of the workflow/task identifiers in the UI. E.g. instead of mypackage.foo.bar.baz.my_long_workflow_name this could be shown as my_long_workflow_name in the UI. As Python function names are somewhat arbitrary (within the constraints of the language), this should already make it easier to read those. If two workflows in different packages/modules would be name the same, e.g. mypackage.foo.bar.module1.my_long_workflow_name and mypackage.foo.bar.module2.my_long_workflow_name, the name shown would include everything that's different. In this case module1.my_workflow_name, module2.my_workflow_name.

Overall, as there seemed to be agreement that this could be a valuable addition, should we convert this into a short RFC?

[kumare3]

I agree that this should just be metadata and no uniqueness guaranteed.

[fg91]

+1

[davidmirror-ops]

related work in progress: #5965