Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add request signal configuration #50

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

WillianAgostini
Copy link
Contributor

Fixes #48

Adding new signal options

  • Tests pass
  • Appropriate changes to documentation are included in the PR

Copy link

changeset-bot bot commented Feb 2, 2024

🦋 Changeset detected

Latest commit: 904795e

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@google/generative-ai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

Copy link

@tmkx tmkx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you!

packages/main/src/requests/request.ts Outdated Show resolved Hide resolved
@gaodeng
Copy link

gaodeng commented Mar 17, 2024

Could you please take a look at this PR, @hsubox76? Thanks in advance!

@hsubox76
Copy link
Collaborator

Sorry I haven't gotten back to this. One question: what's the expected behavior if both timeout and signal are specified? It seems like it might be unexpected behavior as is. We should probably log a warning?

Also heads-up, there is a plan to change timeout to timeoutMillis or timeoutMilliseconds so we might end up doing that all in one PR.

@tmkx
Copy link

tmkx commented Mar 30, 2024

There is a real-world library(axios) that supports both options simultaneously:

import axios from 'axios';

axios
  .get('https://httpbin.org/delay/2', {
    // catch AxiosError: timeout of 1000ms exceeded
    timeout: 1000,
    // catch CanceledError: canceled
    signal: AbortSignal.timeout(1000),
  })
  .then((response) => console.log('then', response.data))
  .catch((reason) => console.error('catch', reason));

it behaves like Promise.race

@hsubox76
Copy link
Collaborator

hsubox76 commented Apr 1, 2024

I thought that was actually going to be pretty complex but it looks like we can just collect all the signals the user wants to use (up to 2 in this case) and wrap them with AbortSignal.any(): https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal#aborting_a_fetch_with_timeout_or_explicit_abort

@hsubox76
Copy link
Collaborator

hsubox76 commented May 2, 2024

An additional wrinkle is that we don't want an abort signal to be set at the model level, you want to set it on a per-request level, otherwise it cancels all requests. We're working on another PR to do this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature Request: signal option
4 participants