Please downloaded openchat_3.5.Q4_K_M.gguf
and tokenizer.json
from Huggingface:
download TheBloke/openchat_3.5-GGUF openchat_3.5.Q4_K_M.gguf
download openchat/openchat_3.5 tokenizer.json
Copy them to the folder ./models/openchat-3.5-gguf/
and then
cargo run --release