Use accelerate
for spacy-llm
to offload to CPU via device_map
#13696
Unanswered
sus-domesticus
asked this question in
Help: Coding & Implementations
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
I want to integrate an LLM into my spacy pipeline but my GPU doesn't have enough space.
I read here that
accelerate
could be used with the optiondevice_map=auto
to offload to the CPU but I don't know how to adapt a basic example to do that.For example how would you do this for the following example (taken from here):
Beta Was this translation helpful? Give feedback.
All reactions