You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Three weeks ago I discovered whisper.cpp and tried it on my 6 year old desktop, CPU: AMD Ryzen 5 1600 Six-Core Processor, GPU: Radeon RX 460/560D, Video memory: 2048MB, gfx803, running Arch Linux based Manjaro. I have tested with a 120.0 sec German spoken file with the “medium” Whisper model and derivatives which provide just sufficient quality for German.
CPU: ./main -m models/ggml-medium.bin -f -l de 252 s
Using 8 threads resulted in a bit shorter time. Then I discovered the WHISPER_HIPBLAS option in the Makefile and installed the official Arch Linux packages for AMD’s ROCm 5.6.1. After make clean and WHISPER_HIPBLAS=1 make -j:
GPU: ./main -m models/ggml-medium.bin -f -l de 71 s
After Sun Nov 5, “cuda : fix HIPBLAS build” commit:
3a. GPU: ./main -m models/ggml-medium.bin -f -l de 66 s
3b. GPU: ./main -m models/ggml-medium-q8_0.bin -f -l de 58 s
After Sun Nov 12, “whisper : add full CUDA and Metal offloading (#1472)” commit
4a. GPU: ./main -m models/ggml-medium.bin -f -l de
CUDA error 2 at ggml-cuda.cu:8202: out of memory
current device: 0
4b. GPU: ./main -m models/ggml-medium-q8_0.bin -f -l de 36 s
As to 4a., my 2 GB graphics card is really low end. Otherwise I experienced a 7-fold improvement using HIPBLAS. Thank you very much for the fantastic work and making whisper.cpp so wide open.
How to set it up Arch Linux based: First make sure that the AMD GPU or APU to be used is supported by the ROCm stack.
sudo pacman -Sy rocm-opencl-runtime rocminfo
opt/rocm/opencl/bin/clinfo
This should show your graphics device. If clinfo does not find anything or even terminates with an assert error there is little chance that whisper.cpp will do better. If successful install ROCm HIP.
sudo pacman -Sy rocm-hip-sdk
That is quite a big download but contains everything to continue in the whisper.cpp directory.
make clean
WHISPER_HIPBLAS=1 make -j
I think that whisper.cpp’s HIPBLAS is mature enough to be advertised and tested on ROCm-supported AMD graphics.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Three weeks ago I discovered whisper.cpp and tried it on my 6 year old desktop, CPU: AMD Ryzen 5 1600 Six-Core Processor, GPU: Radeon RX 460/560D, Video memory: 2048MB, gfx803, running Arch Linux based Manjaro. I have tested with a 120.0 sec German spoken file with the “medium” Whisper model and derivatives which provide just sufficient quality for German.
Using 8 threads resulted in a bit shorter time. Then I discovered the WHISPER_HIPBLAS option in the Makefile and installed the official Arch Linux packages for AMD’s ROCm 5.6.1. After make clean and WHISPER_HIPBLAS=1 make -j:
After Sun Nov 5, “cuda : fix HIPBLAS build” commit:
3a. GPU: ./main -m models/ggml-medium.bin -f -l de 66 s
3b. GPU: ./main -m models/ggml-medium-q8_0.bin -f -l de 58 s
After Sun Nov 12, “whisper : add full CUDA and Metal offloading (#1472)” commit
4a. GPU: ./main -m models/ggml-medium.bin -f -l de
CUDA error 2 at ggml-cuda.cu:8202: out of memory
current device: 0
4b. GPU: ./main -m models/ggml-medium-q8_0.bin -f -l de 36 s
As to 4a., my 2 GB graphics card is really low end. Otherwise I experienced a 7-fold improvement using HIPBLAS. Thank you very much for the fantastic work and making whisper.cpp so wide open.
How to set it up Arch Linux based: First make sure that the AMD GPU or APU to be used is supported by the ROCm stack.
sudo pacman -Sy rocm-opencl-runtime rocminfo
opt/rocm/opencl/bin/clinfo
This should show your graphics device. If clinfo does not find anything or even terminates with an assert error there is little chance that whisper.cpp will do better. If successful install ROCm HIP.
sudo pacman -Sy rocm-hip-sdk
That is quite a big download but contains everything to continue in the whisper.cpp directory.
make clean
WHISPER_HIPBLAS=1 make -j
I think that whisper.cpp’s HIPBLAS is mature enough to be advertised and tested on ROCm-supported AMD graphics.
Beta Was this translation helpful? Give feedback.
All reactions