We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
--------------------------------------------------------------------------- IndexError Traceback (most recent call last) Cell In[1], line 40 37 rng = torch.Generator(device="cuda") 38 rng.manual_seed(789001) ---> 40 character = generator("Give me a character description", rng=rng) 42 print(repr(character)) 43 # Character(name='Anderson', age=28, armor=<Armor.chainmail: 'chainmail'>, weapon=<Weapon.sword: 'sword'>, strength=8) File ~/<redacted>/lib/python3.10/site-packages/outlines/generate/api.py:207, in SequenceGenerator.__call__(self, prompts, max_tokens, stop_at, rng) 205 while True: 206 try: --> 207 last_state = next(states) 208 if max_tokens or stop_sequences: 209 token_ids = last_state.token_ids File ~/<redacted>/lib/python3.10/site-packages/outlines/generate/generator.py:81, in sequence_generator(model, sampler, fsms, token_ids, sequence_weights, attention_masks, fsm_states, rng) 76 raise ContextLengthExceededError( 77 "The input length exceeds the context length of the model." 78 ) 80 allowed_tokens = get_allowed_tokens(fsms, fsm_states) ---> 81 biased_logits = bias_logits(logits, allowed_tokens) 82 next_token_ids, ancestors, sequence_weights = sampler( 83 biased_logits, sequence_weights, rng 84 ) ... 304 for i, ids in enumerate(allowed_token_ids): --> 305 biased_logits[i, ids] = logits[i, ids] 306 return biased_logits IndexError: index 54732 is out of bounds for dimension 0 with size 32000
Copy this example from the readme into a notebook.
no crash
see above
Python 3.10.14
Library versions (let me know if you need more, there are many others that are non-ML that just add noise)
outlines 0.0.40 torch 2.3.0+cu118 transformers 4.40.1
No response
The text was updated successfully, but these errors were encountered:
Unfortunately I cannot reproduce. The only difference is I'm running torch==2.3.0 with torch.version.cuda == 12.1
torch==2.3.0
torch.version.cuda == 12.1
My output is
Character(name='Alex Chen', age=28, armor=<Armor.leather: 'leather'>, weapon=<Weapon.bow: 'bow'>, strength=65) Character(name='Boo', age=27, armor=<Armor.chainmail: 'chainmail'>, weapon=<Weapon.bow: 'bow'>, strength=70)
Could you please try the following:
~/.cache/huggingface/hub
import outlines.caching; outlines.caching.clear_cache()
pip install --upgrade outlines
If that doesn't work, I'd like to know your full pip freeze, nvcc --version, nvidia-smi, and the model of your device please.
pip freeze
nvcc --version
nvidia-smi
Sorry, something went wrong.
Closing for now, as it is not reproducible. Feel free to re-open if the error persists.
No branches or pull requests
Describe the issue as clearly as possible:
Steps/code to reproduce the bug:
Copy this example from the readme into a notebook.
Expected result:
Error message:
Outlines/Python version information:
Python 3.10.14
Library versions (let me know if you need more, there are many others that are non-ML that just add noise)
Context for the issue:
No response
The text was updated successfully, but these errors were encountered: