Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Boolean values within pydantic base model being casted to scalar value #5409

Open
2 tasks done
jiwidi opened this issue May 22, 2024 · 3 comments
Open
2 tasks done
Labels
bug Something isn't working flytekit FlyteKit Python related issue

Comments

@jiwidi
Copy link

jiwidi commented May 22, 2024

Describe the bug

Describe the bug

Trying to pass a pydantic model between task but its converting bool variables to int. Is there a way to make flyte work with pydantic models?

Expected behavior

For bool types to stay as bool types

Additional context to reproduce

I have this main.py file containing my workflow:

from flytekit import task, workflow
from pydantic import BaseModel

class Config(BaseModel):
    bool_var: bool

@task
def create_config() -> Config:
    return Config(bool_var=True)

@task
def check_bool(x: bool) -> bool:
    return x

@workflow
def workflow() -> bool:
    config = create_config()
    return check_bool(x=config['bool_var'])

When I run this with the command

pyflyte run issue.py workflow

I get the error:

TypeTransformerFailedError: Failed to convert inputs of task 'issue.check_bool':
  Error converting input 'x' at position 0:
  Cannot convert literal <FlyteLiteral(Literal) scalar { blob { metadata { type { format: "PythonPickle" } } uri:
"/var/folders/zr/72m6cw8x66b4g3q_s8dmy5c00000gn/T/flyte-bgc4g0zt/raw/d8a7be6139519884010995bc2aa7ddd9/cd165a40ea1442ca1acb6fbd4b3700f4" } }> to <class 'bool'>

The bool property in the Config object is converted to a scalar.

Screenshots

No response

Are you sure this issue hasn't been raised already?

  • Yes

Have you read the Code of Conduct?

  • Yes
@jiwidi jiwidi added bug Something isn't working untriaged This issues has not yet been looked at by the Maintainers labels May 22, 2024
Copy link

runllm bot commented May 22, 2024

Flyte currently has issues with handling Pydantic models, particularly with type conversions. Boolean values within Pydantic models may be cast to scalar values due to serialization issues. This is a known problem and has been discussed in the community.

Would you like more detail on potential workarounds or ongoing efforts to address this issue?

Sources:


This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @RunLLM to ask follow-up questions. See runllm.com for more!

@eapolinario eapolinario added flytekit FlyteKit Python related issue and removed untriaged This issues has not yet been looked at by the Maintainers labels May 23, 2024
@novahow
Copy link
Contributor

novahow commented Jul 2, 2024

Hi, @jiwidi

I guess this might relate to how flyte handles complex data types. Currently as a workaround, you can try the following:

from flytekit import task, workflow
from pydantic import BaseModel

class Config(BaseModel):
    bool_var: bool


@task
def create_config() -> Config:
    return Config(bool_var=True)

@task
def check_bool(x: bool) -> bool:
    return x

@task
def bool_parser(x: Config) -> bool:
    return x.bool_var


@workflow
def wf() -> bool:
    config = create_config()
    return check_bool(x=bool_parser(x=config))

@kumare3
Copy link
Contributor

kumare3 commented Sep 11, 2024

fixed by - #5318

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working flytekit FlyteKit Python related issue
Projects
None yet
Development

No branches or pull requests

4 participants