Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AGPT-698] Update cli commands and add groq integration #229

Merged

Conversation

Swiftyos
Copy link
Contributor

@Swiftyos Swiftyos commented Apr 22, 2024

  • Allows configuration of codex-url so we can use resume on gcp envs
  • Adds a quick an easy way to switch to groq and override chat model and max tokens
  • Added rate limiting based on requests per minute (10_000) and number of concurrent requests (1_000) defaulted to our openai limits

Copy link

linear bot commented Apr 22, 2024

AGPT-698

@Torantulino
Copy link
Member

/review auto_approve

Copy link

Auto-approval option for PR-Agent is disabled. You can enable it via a configuration file

@Torantulino
Copy link
Member

/reflect_and_review

Copy link

Questions to better understand the PR:

  1. Was the removal of port configuration in CLI commands replaced entirely by the base URL configuration, or is there another mechanism to specify the port now?
  2. Does the GROQ integration default to using the "llama3-70b-8192" model only when no other model is specified, or is it a fixed setting?
  3. How are the new rate limits configured in the OpenAIChatClient class intended to interact with the existing system's throughput and performance?

Please respond to the questions above in the following format:

/answer

  1. ...
  2. ...
    ...

@Swiftyos
Copy link
Contributor Author

/review

@Swiftyos
Copy link
Contributor Author

/answer

  1. Yes we now use url only as that includes the port
  2. We use llama 3 when no model is specified for groq
  3. Its meant to limit the existing system, though defaults match our limits with openai

@ntindle
Copy link
Member

ntindle commented Apr 24, 2024

We are throwing errors on their side using pro

@Swiftyos
Copy link
Contributor Author

/review

@Swiftyos Swiftyos merged commit 705aa21 into main Apr 24, 2024
3 checks passed
@Swiftyos Swiftyos deleted the swiftyos/agpt-698-update-cli-commands-and-add-groq-integration branch April 24, 2024 13:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants