windows ollama 0.1.34 can not use GPU，with nvidia RTX 4060

[zhafree]

What is the issue?

C:\Users\zh_af>nvidia-smi
Thu May  9 08:53:43 2024
+---------------------------------------------------------------------------------------+
| NVIDIA-SMI 537.70                 Driver Version: 537.70       CUDA Version: 12.2     |
|-----------------------------------------+----------------------+----------------------+
| GPU  Name                     TCC/WDDM  | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
|                                         |                      |               MIG M. |
|=========================================+======================+======================|
|   0  NVIDIA GeForce RTX 4060      WDDM  | 00000000:01:00.0  On |                  N/A |
| 31%   30C    P8              N/A / 115W |   7776MiB /  8188MiB |      2%      Default |
|                                         |                      |                  N/A |
+-----------------------------------------+----------------------+----------------------+

+---------------------------------------------------------------------------------------+
| Processes:                                                                            |
|  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
|        ID   ID                                                             Usage      |
|=======================================================================================|
|    0   N/A  N/A      2824    C+G   ...on\124.0.2478.80\msedgewebview2.exe    N/A      |
|    0   N/A  N/A      4304    C+G   ...__8wekyb3d8bbwe\Notepad\Notepad.exe    N/A      |
|    0   N/A  N/A      8132    C+G   ...nt.CBS_cw5n1h2txyewy\SearchHost.exe    N/A      |
|    0   N/A  N/A      9796    C+G   ...5\extracted\runtime\WeChatAppEx.exe    N/A      |
|    0   N/A  N/A     10424    C+G   ...wekyb3d8bbwe\XboxGameBarWidgets.exe    N/A      |
|    0   N/A  N/A     11688    C+G   C:\Windows\System32\NahimicSvc64.exe      N/A      |
|    0   N/A  N/A     12648    C+G   ...5n1h2txyewy\ShellExperienceHost.exe    N/A      |
|    0   N/A  N/A     19188    C+G   ...cal\Microsoft\OneDrive\OneDrive.exe    N/A      |
|    0   N/A  N/A     20240    C+G   ...oogle\Chrome\Application\chrome.exe    N/A      |
|    0   N/A  N/A     20948    C+G   C:\Windows\explorer.exe                   N/A      |
|    0   N/A  N/A     21820    C+G   ...x64__8wekyb3d8bbwe\WinStore.App.exe    N/A      |
|    0   N/A  N/A     24376    C+G   ...ekyb3d8bbwe\PhoneExperienceHost.exe    N/A      |
|    0   N/A  N/A     25396      C   ...\cuda_v11.3\ollama_llama_server.exe    N/A      |
|    0   N/A  N/A     26708    C+G   ...8bbwe\SnippingTool\SnippingTool.exe    N/A      |
|    0   N/A  N/A     27804    C+G   ...\Docker\frontend\Docker Desktop.exe    N/A      |
|    0   N/A  N/A     30560    C+G   C:\Windows\explorer.exe                   N/A      |
|    0   N/A  N/A     30980    C+G   ...siveControlPanel\SystemSettings.exe    N/A      |
|    0   N/A  N/A     31312    C+G   ...n\SunloginClient\SunloginClient.exe    N/A      |
|    0   N/A  N/A     35540    C+G   ...crosoft\Edge\Application\msedge.exe    N/A      |
|    0   N/A  N/A     38564    C+G   ...__8wekyb3d8bbwe\WindowsTerminal.exe    N/A      |
|    0   N/A  N/A     39748    C+G   ...CBS_cw5n1h2txyewy\TextInputHost.exe    N/A      |
|    0   N/A  N/A     41288    C+G   ...2txyewy\StartMenuExperienceHost.exe    N/A      |
+---------------------------------------------------------------------------------------+

OS

Windows

GPU

Nvidia

CPU

Intel

Ollama version

0.1.34

[zhafree]

I run llama3:8B use ollama run llama3；0.1.33 version is OK,when update GPU can not used.

[owenzhao]

Did you update your Nvidia driver recently? I found that after your update your Nvidia's driver, Ollama will use the CPU instead of GPU. You can reboot your windows the Ollama will use GPU again.

[dhiltgen]

In addition to the suggestion to reboot if you updated the driver recently, if that doesn't resolve the problem, can you upgrade to the latest ollama version and if you still see CPU only, please share your server log?