ValueError: Unknown quantization method: bitsandbytes. Must be one of ['aqlm', 'awq', 'fp8', 'gptq', 'squeezellm', 'gptq_marlin', 'marlin'].

[chjoo7]

I met this error during the call LLM with the Mixtral8x7B fine-tuned LoRA Merged Model. It worked well in torch 2.1.2 and vlllm 0.3.3 but torch 2,3 and vlllm 0.4.2 met this valueError. What is missing here? Should I use the different quantization method like awq or gptq than bitsandbytes?

[cyp-jlu-ai]

we meet the same promblem

[jctux]

same here

[sajadn]

same here!

[fersebas]

Same any news? :)