Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] 增加自定义模型服务商 #5833

Open
LiMinghan1220 opened this issue Nov 16, 2024 · 5 comments
Open

[Feature Request] 增加自定义模型服务商 #5833

LiMinghan1220 opened this issue Nov 16, 2024 · 5 comments
Labels
enhancement New feature or request

Comments

@LiMinghan1220
Copy link

🥰 需求描述

模型兼容自定义

🧐 解决方案

希望在模型服务商可使用户自行可多开几个兼容OpenAI的以使用未兼容模型(阶跃星辰等)。

📝 补充信息

No response

@LiMinghan1220 LiMinghan1220 added the enhancement New feature or request label Nov 16, 2024
@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Title: [Feature Request] Add custom model service provider

🥰 Description of requirements

Model compatible with customization

🧐 Solution

It is hoped that the model service provider can enable users to open more OpenAI-compatible models to use incompatible models (step stars, etc.).

📝 Supplementary information

No response

@nealhan
Copy link

nealhan commented Nov 22, 2024

是的,这个功能十分重要,可以添加第三方的接口和API Key,更加灵活。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


Yes, this function is very important. You can add third-party interfaces and API Keys to make it more flexible.

@lloydzhou
Copy link
Member

这个其实在 #5001 处理自定义模型的功能的时候,就有想过。

大致的想法是:

  1. providertypename
  2. 针对相同的type可以使用不同的name定义多个服务商(例如多个中转服务商的情况)
  3. model@providerName指定不同服务商(使用不同的api_key和api_base)
  4. 具体聊天过程中处理messages(不管是生成还是解析)会按不同的providerType找到相应的处理逻辑。

@Issues-translate-bot
Copy link

Bot detected the issue body's language is not English, translate it automatically.


I actually thought about this when #5001 was dealing with the function of custom models.

The general idea is:

  1. provider has type and name
  2. For the same type, you can use different name to define multiple service providers (such as the case of multiple transit service providers)
  3. model@providerName specifies different service providers (using different api_key and api_base)
  4. When processing messages (whether generated or parsed) during the specific chat process, the corresponding processing logic will be found according to different providerType.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants