Greatly improved LLaMA sampling defaults. (#2)

By default `drama_llama` was using greedy sampling with no repetition penalty. This was a mistake in the implementation of `Default` for various settings structs. The default has now been changed to locally typical sampling with a minor repetition penalty. Quality of generation should be greatly improved. Additionally, `llama.cpp` has been updated. This means any models will need to be updated since the tokenizer code has changed. The user will be warned in the terminal if that is the case.
mdegans · May 24, 2024 · 6b36ea2 · 6b36ea2
1 parent 0246432
commit 6b36ea2
Show file tree

Hide file tree

Showing 2 changed files with 42 additions and 43 deletions.
diff --git a/Cargo.lock b/Cargo.lock
diff --git a/Cargo.toml b/Cargo.toml
@@ -1,6 +1,6 @@
 [package]
 name = "weave-writer"
-version = "0.0.0"
+version = "0.0.1"
 edition = "2021"
 description = "A tool for collaborative generative writing."
 license-file = "LICENSE.md"