Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Fix hot model reload with `drama_llama` backend not working #7 * Loading is now entirely in the worker thread. * During load, the settings are locked but the brief blocking before is gone. * Several new `Request` and `Response` messages were added. The issue with the panic for unsupported models (on Metal at least) is still there. That requires changes in `llama.cpp` itself or duplication of code.
- Loading branch information