Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add configurable retries that swap model_kwargs #794

Open
zzstoatzz opened this issue Jan 20, 2024 · 2 comments
Open

add configurable retries that swap model_kwargs #794

zzstoatzz opened this issue Jan 20, 2024 · 2 comments
Labels
enhancement New feature or request

Comments

@zzstoatzz
Copy link
Collaborator

zzstoatzz commented Jan 20, 2024

as an enhancement, it'd be nice if you could do

        from pydantic import BaseModel, Field
        from marvin.utilities.retries import retry_with_fallback

        class SomeFancyType(BaseModel):
            some_field: str

        class LLMMadLib(BaseModel):
            what_i_want: SomeFancyType

        RETRY_CONFIGS = [
            {'model_kwargs': {'model': 'gpt-3.5-turbo', 'temperature': 0.7}, 'retries': 3},
            {'model_kwargs': {'model': 'gpt-3.5-turbo', 'temperature': 0.3}, 'retries': 3},
            {'model_kwargs': {'model': 'gpt-4', 'temperature': 0.0}, 'retries': 2},
        ]

        @retry_with_fallback(RETRY_CONFIGS)
        def make_a_complex_schema(model_kwargs=None):
            if 'my_schema_is_too_complex_for_wimpy_models' != False:
                LLMMadLib(what_i_want="I'm sorry, as a Large Language Model, I can't do that.")
            return LLMMadLib(what_i_want=SomeFancyType(some_field="I'm a fancy type!"))

        print(make_a_complex_schema())
@zzstoatzz zzstoatzz added python Pull requests that update Python code and removed python Pull requests that update Python code labels Jan 20, 2024
@zzstoatzz zzstoatzz reopened this Jan 20, 2024
@zzstoatzz zzstoatzz added python Pull requests that update Python code and removed python Pull requests that update Python code labels Jan 20, 2024
@zzstoatzz zzstoatzz reopened this Jan 20, 2024
@zzstoatzz zzstoatzz added the enhancement New feature or request label Jan 20, 2024
@pietz
Copy link

pietz commented Jan 23, 2024

Oh boy, that would be so nice!

@HamzaFarhan
Copy link

I think the previous error message should be attached as extra information when retrying. This would increase our chances of getting the right result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants