We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
llamafactory
Put your message here.
No response
The text was updated successfully, but these errors were encountered:
服务器使用API_PORT=8000 CUDA_VISIBLE_DEVICES=0 llamafactory-cli api examples/inference/llama3.yaml,正常启动,无报错;
Sorry, something went wrong.
终端调用代码: from openai import OpenAI client = OpenAI(api_key="0",base_url="http://服务器IP:8000/v1") messages = [{"role": "user", "content": "Who are you?"}] result = client.chat.completions.create(messages=messages, model="服务器上模型地址") print(result.choices[0].message)
问题描述:服务器正常打印info,显示message,但终端无响应。 尝试排错:1、在服务器上直接调用API,修改ip为0.0.0.0,仍无响应无报错; 2、直接使用transformers.pipeline调用预训练模型,模型正常推理,预训练文件正常
No branches or pull requests
Reminder
System Info
llamafactory
version: 0.9.2.dev0Reproduction
Others
No response
The text was updated successfully, but these errors were encountered: