BUG: Stop button don't stop generation ollama

[SN4K3D]

Bug Report

When i click on stop button on the web-ui the generation ollama backend don't stop work , the cpu / gpu is always solicited

Description

Bug Summary:
[Provide a brief but clear summary of the bug]

Steps to Reproduce:
create one request on prompt when the first word appeared click on the stop button . the text stop to appeared but the cpu continu to work for generate the response

Environment

model mistral is used with your helm deploy

• Operating System: [Ubuntu 22.04]
• Browser (if applicable): [operaGX]

Reproduction Details

Confirmation:

• [x ] I have read and followed all the instructions provided in the README.md.
• [x ] I am on the latest version of both Open WebUI and Ollama.
• I have included the browser console logs.
• I have included the Docker container logs.

Installation Method

deploy with helm charts describe in your doc

[tnertia]

I am also encountering this issue with Docker (Open WebUI 0.1.118 and Ollama 0.1.31). Have not investigated much, but the problem might stem from a null requestId being passed to cancelOllamaRequest().

Container logs:

INFO:apps.ollama.main:url: http://ollama:11434
INFO:     172.21.0.1:60056 - "POST /ollama/v1/chat/completions HTTP/1.1" 200 OK
INFO:     172.21.0.1:60056 - "GET /api/v1/chats/ HTTP/1.1" 200 OK
INFO:apps.ollama.main:url: http://ollama:11434
INFO:     172.21.0.1:55602 - "POST /ollama/api/chat HTTP/1.1" 200 OK
INFO:     172.21.0.1:53478 - "GET /ollama/cancel/null HTTP/1.1" 200 OK
INFO:     172.21.0.1:53478 - "GET /api/v1/chats/ HTTP/1.1" 200 OK

[jukofyork]

It seems to be working fine for llama.cpp server now. I don't think the prompt evaluation can be stopped instantly (but I think that is the case for everything using llama.cpp server; for large GPU batches anyway), but the generation seems to stop fine now.

I've completely ditched Ollama and just moved over to the llama.cpp server and just want to say thanks as it's working really smoothly with Open-WebUI! 👍

[jukofyork]

One possible (low-priority) improvement would be to set the stream_options --> include_usage = true setting in the API call. It then (like OpenAI) returns the 3 values: completion_tokens, prompt_tokens and total_tokens (but no timings like native llama.cpp and Ollama - I guess OpenAI didn't want anybody to be able to measure lag-spikes lol!).

These are quite useful to see how close you are to using up the context and better pre-empt the need to summarise your story so far, etc.

[0x7CFE]

I am still experiencing the issue on ollama.

[Arche151]

Same

[SN4K3D]

Sorry for the delay , i have just try the stop button and now it's okay , the stop button stop all thread "generating ollama response" and the cpu are not used. Thanks a lot for this patch !