The visual perception speed of GPT-4o has slowed down. #2697

wupengfei0904 · 2024-05-14T02:07:24Z

wupengfei0904
May 14, 2024
Sponsor

I updated to GPT-4o today. It processes text very quickly, but when I send images to it, the processing speed reverts to that of GPT-4 Turbo. Moreover, subsequent conversations also become slower. I'm not sure if this is due to the lack of integration in the visual perception aspect, which might be causing this issue.

danny-avila · 2024-05-14T02:17:41Z

danny-avila
May 14, 2024
Maintainer

LibreChat prioritizes the current model selected as the "vision" model. if it's not vision-capable, it will use the first vision-capable model available in the model list, by order of the list.

Being able to select the vision model outright would improve this current functionality, but this is definitely the case for gpt-4o if you have correctly updated to the latest commit of LibreChat.

If you are noticing response degradation, it's likely a result of including images to gpt-4o. It will then become "slower" for subsequent messages because by default, LibreChat will resend all previous images with each new message.

You can disable this behavior with the following conversation/preset setting. Doing so would make it only send images when you attach them.

confirmation

TL;DR, if you select gpt-4o, it will use gpt-4o for vision.

1 reply

wupengfei0904 May 14, 2024
Author Sponsor

Thank you for your reply. I checked the log files and saw that I selected the GPT-4o model at the OpenAI endpoint. When I sent an image, I noticed that the URL was sent to GPT-4 Turbo. Here are my log files:

2024-05-14T02:29:48.321Z debug: [Balance.check]
{
user: "661609792897da1922873f81",
model: "gpt-4o",
endpoint: "openAI",
valueKey: undefined,
tokenType: "prompt",
amount: 8,
balance: 986381558,
multiplier: 30,
endpointTokenConfig: false,
}
2024-05-14T02:29:48.321Z debug: [Balance.check]
{
tokenCost: 240,
}
2024-05-14T02:29:48.321Z debug: [OpenAIClient] chatCompletion
{
baseURL: "https://api.openai.com/v1",
modelOptions.model: "gpt-4o",
modelOptions.temperature: 1,
modelOptions.top_p: 1,
modelOptions.presence_penalty: 0,
modelOptions.frequency_penalty: 0,
modelOptions.stop: undefined,
modelOptions.max_tokens: undefined,
modelOptions.user: "661609792897da1922873f81",
modelOptions.stream: true,
// 1 message(s)
modelOptions.messages: [{"role":"user","content":"hello"}],
}
2024-05-14T02:29:49.481Z debug: [OpenAIClient] chatCompletion response
{
object: "chat.completion",
id: "chatcmpl-9OcDEPyCVLGdAO0FQjv9hkmduqF1D",
// 1 choice(s)
choices: [{"message":{"role":"assistant","content":"Hello! How can I assist you today?"},"finish_reason":"stop... [truncated]],
created: 1715653788,
model: "gpt-4o-2024-05-13",
system_fingerprint: "fp_729ea513f7",
}
2024-05-14T02:29:49.481Z debug: [spendTokens] conversationId: e9866d86-19b5-47d3-8450-f076edaffcbc | Context: message | Token usage:
{
promptTokens: 8,
completionTokens: 9,
}
2024-05-14T02:29:49.488Z debug: [spendTokens] Transaction data record against balance:
{
user: "661609792897da1922873f81",
prompt: -240,
promptRate: 30,
completion: -540,
completionRate: 60,
balance: 986380778,
}
2024-05-14T02:29:49.496Z debug: [AskController] Request closed
2024-05-14T02:29:49.503Z debug: [createStartHandler] handleChatModelStart: title
{
model: "gpt-3.5-turbo",
function_call.name: "output_formatter",
}
2024-05-14T02:29:49.503Z debug: [createStartHandler]
{
prelimPromptTokens: 85,
tokenBuffer: 150,
}
2024-05-14T02:29:49.505Z debug: [Balance.check]
{
user: "661609792897da1922873f81",
model: "gpt-3.5-turbo",
endpoint: "openAI",
valueKey: undefined,
tokenType: "prompt",
amount: 235,
balance: 986380778,
multiplier: 1.5,
endpointTokenConfig: false,
}
2024-05-14T02:29:49.505Z debug: [Balance.check]
{
tokenCost: 352.5,
}
2024-05-14T02:29:50.865Z debug: [RunManager] handleLLMEnd: {"context":"title","tokenBuffer":150,"conversationId":"e9866d86-19b5-47d3-8450-f076edaffcbc"}
{
runId: "1d566a9e-1234-4f20-bb4b-ba2724b5e4b8",
_parentRunId: undefined,
tokenUsage.completionTokens: 5,
tokenUsage.promptTokens: 88,
tokenUsage.totalTokens: 93,
}
2024-05-14T02:29:50.865Z debug: [spendTokens] conversationId: e9866d86-19b5-47d3-8450-f076edaffcbc | Context: title | Token usage:
{
promptTokens: 88,
completionTokens: 5,
}
2024-05-14T02:29:50.870Z debug: [spendTokens] Transaction data record against balance:
{
user: "661609792897da1922873f81",
prompt: -132,
promptRate: 1.5,
completion: -10,
completionRate: 2,
balance: 986380636,
}
2024-05-14T02:29:50.871Z debug: [createStartHandler] handleChatModelStart: title
{
model: "gpt-3.5-turbo",
function_call.name: "output_formatter",
}
2024-05-14T02:29:50.872Z debug: [createStartHandler]
{
prelimPromptTokens: 137,
tokenBuffer: 150,
}
2024-05-14T02:29:50.872Z debug: [Balance.check]
{
user: "661609792897da1922873f81",
model: "gpt-3.5-turbo",
endpoint: "openAI",
valueKey: undefined,
tokenType: "prompt",
amount: 287,
balance: 986380636,
multiplier: 1.5,
endpointTokenConfig: false,
}
2024-05-14T02:29:50.873Z debug: [Balance.check]
{
tokenCost: 430.5,
}
2024-05-14T02:29:51.837Z debug: [RunManager] handleLLMEnd: {"context":"title","tokenBuffer":150,"conversationId":"e9866d86-19b5-47d3-8450-f076edaffcbc"}
{
runId: "bdb74d9c-32ec-4bbd-adfd-974208098c7c",
_parentRunId: undefined,
tokenUsage.completionTokens: 11,
tokenUsage.promptTokens: 140,
tokenUsage.totalTokens: 151,
}
2024-05-14T02:29:51.838Z debug: [spendTokens] conversationId: e9866d86-19b5-47d3-8450-f076edaffcbc | Context: title | Token usage:
{
promptTokens: 140,
completionTokens: 11,
}
2024-05-14T02:29:51.843Z debug: [spendTokens] Transaction data record against balance:
{
user: "661609792897da1922873f81",
prompt: -210,
promptRate: 1.5,
completion: -22,
completionRate: 2,
balance: 986380404,
}
2024-05-14T02:29:51.844Z debug: [OpenAIClient] Convo Title: Hello How Can I Assist You Today
2024-05-14T02:30:04.298Z debug: [AskController]
{
text: "what is？",
conversationId: "e9866d86-19b5-47d3-8450-f076edaffcbc",
endpoint: "openAI",
chatGptLabel: null,
promptPrefix: null,
resendFiles: true,
imageDetail: "auto",
iconURL: undefined,
greeting: undefined,
spec: undefined,
maxContextTokens: undefined,
modelOptions.model: "gpt-4o",
modelOptions.temperature: 1,
modelOptions.top_p: 1,
modelOptions.presence_penalty: 0,
modelOptions.frequency_penalty: 0,
modelOptions.stop: undefined,
modelOptions.max_tokens: undefined,
// 2 openAI(s)
modelsConfig.openAI: ["gpt-4o","gpt-4-turbo"],
// 12 google(s)
modelsConfig.google: ["gemini-pro","gemini-pro-vision","chat-bison","chat-bison-32k","codechat-bison","codechat-bison-32k","text-bison","text-bison-32k","text-unicorn","code-gecko","code-bison","code-bison-32k"],
// 10 anthropic(s)
modelsConfig.anthropic: ["claude-3-opus-20240229","claude-3-sonnet-20240229","claude-3-haiku-20240307","claude-2.1","claude-2","claude-1.2","claude-1","claude-1-100k","claude-instant-1","claude-instant-1-100k"],
// 20 gptPlugin(s)
modelsConfig.gptPlugins: ["gpt-4o-2024-05-13","gpt-4o","gpt-3.5-turbo-16k","gpt-4","gpt-4-0613","gpt-4-turbo-2024-04-09","gpt-3.5-turbo-1106","gpt-4-turbo","gpt-3.5-turbo","gpt-3.5-turbo-0301","gpt-4-1106-preview","gpt-4-0125-preview","gpt-3.5-turbo-0125","gpt-4-turbo-preview","gpt-3.5-turbo-0613","gpt-4-1106-vision-preview","gpt-4-vision-preview","gpt-3.5-turbo-16k-0613","gpt-3.5-turbo-instruct-0914","gpt-3.5-turbo-instruct"],
// 19 azureOpenAI(s)
modelsConfig.azureOpenAI: ["gpt-3.5-turbo-0125","gpt-4-turbo","gpt-4-turbo-2024-04-09","gpt-3.5-turbo-16k-0613","gpt-3.5-turbo-16k","gpt-4-turbo-preview","gpt-4-0125-preview","gpt-4-1106-preview","gpt-3.5-turbo","gpt-3.5-turbo-1106","gpt-4-vision-preview","gpt-4","gpt-3.5-turbo-instruct-0914","gpt-3.5-turbo-0613","gpt-3.5-turbo-0301","gpt-3.5-turbo-instruct","gpt-4-0613","text-davinci-003","gpt-4-0314"],
// 2 bingAI(s)
modelsConfig.bingAI: ["BingAI","Sydney"],
// 2 chatGPTBrowser(s)
modelsConfig.chatGPTBrowser: ["text-davinci-002-render-sha","gpt-4"],
// 2 assistant(s)
modelsConfig.assistants: ["gpt-4o","gpt-4-turbo"],
// 1 通义千问(s)
modelsConfig.通义千问: ["qwen-turbo"],
attachments: [object Promise],
}
2024-05-14T02:30:04.299Z debug: [BaseClient] Loading history:
{
conversationId: "e9866d86-19b5-47d3-8450-f076edaffcbc",
parentMessageId: "5b2d3851-d584-439d-8a12-286390b34ce2",
}
2024-05-14T02:30:04.304Z debug: [BaseClient] Context Count (1/2)
{
remainingContextTokens: 7734,
maxContextTokens: 8187,
}
2024-05-14T02:30:04.304Z debug: [BaseClient] Context Count (2/2)
{
remainingContextTokens: 7734,
maxContextTokens: 8187,
}
2024-05-14T02:30:04.304Z debug: [BaseClient] tokenCountMap:
{
cf17c994-6387-4cc9-9f52-76eb82223e61: 5,
5b2d3851-d584-439d-8a12-286390b34ce2: 13,
12b1ba56-6cd8-463a-9675-11b5db445903: 432,
}
2024-05-14T02:30:04.304Z debug: [BaseClient]
{
promptTokens: 453,
remainingContextTokens: 7734,
payloadSize: 3,
maxContextTokens: 8187,
}
2024-05-14T02:30:04.304Z debug: [BaseClient] tokenCountMap
{
cf17c994-6387-4cc9-9f52-76eb82223e61: 5,
5b2d3851-d584-439d-8a12-286390b34ce2: 13,
12b1ba56-6cd8-463a-9675-11b5db445903: 432,
instructions: undefined,
}
2024-05-14T02:30:04.304Z debug: [BaseClient] userMessage
{
messageId: "12b1ba56-6cd8-463a-9675-11b5db445903",
parentMessageId: "5b2d3851-d584-439d-8a12-286390b34ce2",
conversationId: "e9866d86-19b5-47d3-8450-f076edaffcbc",
sender: "User",
text: "what is？",
isCreatedByUser: true,
// 1 image_url(s)
image_urls: [{"type":"image_url","image_url":{"url":"data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAA1sAAAFmCAYAAA... [truncated]],
tokenCount: 432,
}
2024-05-14T02:30:04.304Z debug: [BaseClient] Skipping cf17c994-6387-4cc9-9f52-76eb82223e61: already had a token count.
2024-05-14T02:30:04.304Z debug: [BaseClient] Skipping 5b2d3851-d584-439d-8a12-286390b34ce2: already had a token count.
2024-05-14T02:30:04.316Z debug: [Balance.check]
{
user: "661609792897da1922873f81",
model: "gpt-4-turbo",
endpoint: "openAI",
valueKey: undefined,
tokenType: "prompt",
amount: 453,
balance: 986380404,
multiplier: 10,
endpointTokenConfig: false,
}
2024-05-14T02:30:04.316Z debug: [Balance.check]
{
tokenCost: 4530,
}
2024-05-14T02:30:04.316Z debug: [OpenAIClient] chatCompletion
{
baseURL: "https://api.openai.com/v1",
modelOptions.model: "gpt-4-turbo",
modelOptions.temperature: 1,
modelOptions.top_p: 1,
modelOptions.presence_penalty: 0,
modelOptions.frequency_penalty: 0,
modelOptions.max_tokens: undefined,
modelOptions.user: "661609792897da1922873f81",
modelOptions.stream: true,
// 3 message(s)
modelOptions.messages: [{"role":"user","content":"hello"},{"role":"assistant","content":"Hello! How can I assist you today?"},{"role":"user","content":[{"type":"text","text":"what is？"},{"type":"image_url","image_url":{"url":"... [truncated]],
}
2024-05-14T02:30:16.018Z debug: [OpenAIClient] chatCompletion response
{
object: "chat.completion",
id: "chatcmpl-9OcDWa4oDuEturUgS9Zes8Bz3oEpk",
// 1 choice(s)
choices: [{"message":{"role":"assistant","content":"The image shows a conversation between a user and a GPT-4 ... [truncated]],
created: 1715653806,
model: "gpt-4-turbo-2024-04-09",
system_fingerprint: "fp_0737e0dfd9",
}
2024-05-14T02:30:16.077Z debug: [spendTokens] conversationId: e9866d86-19b5-47d3-8450-f076edaffcbc | Context: message | Token usage:
{
promptTokens: 453,
completionTokens: 67,
}
2024-05-14T02:30:16.083Z debug: [spendTokens] Transaction data record against balance:
{
user: "661609792897da1922873f81",
prompt: -4530,
promptRate: 10,
completion: -2010,
completionRate: 30,
balance: 986373864,
}
2024-05-14T02:30:16.092Z debug: [AskController] Request closed

wupengfei0904 · 2024-05-14T02:33:11Z

wupengfei0904
May 14, 2024
Author Sponsor

0 replies

danny-avila · 2024-05-14T02:47:49Z

danny-avila
May 14, 2024
Maintainer

The logs that actually matter immediately follow [OpenAIClient] chatCompletion

Make sure you are updated to the latest commit with git log. You should see this at the top:

commit 638ac5bba61a524cc4ae99711a91f19572c4f2a0 (origin/main, origin/HEAD, main)
Author: Danny Avila <[email protected]>
Date:   Mon May 13 14:25:02 2024 -0400

    🚀 feat: gpt-4o (#2692)
    
    * 🚀 feat: gpt-4o
    
    * update readme.md
    
    * feat: Add new test case for getMultiplier function
    
    * feat: Refactor getMultiplier function to use valueKey variable

Make sure you use docker compose pull if you are using docker, since this will pull the latest images.

I can't reproduce the model switching to gpt-4-turbo except deliberately

0 replies

wupengfei0904 · 2024-05-14T03:50:14Z

wupengfei0904
May 14, 2024
Author Sponsor

Thank you again. I have updated it, and it is now back to normal.

0 replies

wupengfei0904 · 2024-05-14T04:25:40Z

wupengfei0904
May 14, 2024
Author Sponsor

Additionally, I just tried the assistant model, and gpt-4o is currently not supported in assistant V1.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

The visual perception speed of GPT-4o has slowed down. #2697

{{title}}

Replies: 5 comments 1 reply

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

The visual perception speed of GPT-4o has slowed down. #2697

wupengfei0904 May 14, 2024 Sponsor

Replies: 5 comments · 1 reply

danny-avila May 14, 2024 Maintainer

confirmation

wupengfei0904 May 14, 2024 Author Sponsor

wupengfei0904 May 14, 2024 Author Sponsor

danny-avila May 14, 2024 Maintainer

wupengfei0904 May 14, 2024 Author Sponsor

wupengfei0904 May 14, 2024 Author Sponsor

wupengfei0904
May 14, 2024
Sponsor

Replies: 5 comments 1 reply

danny-avila
May 14, 2024
Maintainer

wupengfei0904 May 14, 2024
Author Sponsor

wupengfei0904
May 14, 2024
Author Sponsor

danny-avila
May 14, 2024
Maintainer

wupengfei0904
May 14, 2024
Author Sponsor

wupengfei0904
May 14, 2024
Author Sponsor