[REQUEST] EXAONE 3.5 Support

[necrogay]

Problem

LG AI Research has released new EXAONE 3.5 models in sizes 2.4B, 7.8B, and 32B:

https://huggingface.co/LGAI-EXAONE/EXAONE-3.5-2.4B-Instruct
https://huggingface.co/LGAI-EXAONE/EXAONE-3.5-7.8B-Instruct
https://huggingface.co/LGAI-EXAONE/EXAONE-3.5-32B-Instruct

Unfortunately, exllamav2 does not currently support these models:

!! Warning, unknown architecture: ExaoneForCausalLM

Solution

Could you please consider adding support for these models?

Alternatives

No response

Explanation

Examples

No response

Additional context

No response

Acknowledgements

• I have looked for similar requests before submitting this one.
• I understand that the developers have lives and my issue will be answered when possible.
• I understand the developers of this program are human, and I will make my requests politely.

[Nrgte]

I can confirm, I tried to play around with it and can't quantize it:

raise ValueError(f" ## Could not find {prefix}.* in model")

ValueError: ## Could not find model.norm.* in model

[turboderp]

It's one of those models that implements a Llama architecture but renames everything so it isn't compatible with existing frameworks. There are a number of "llamafied" versions on HF that you can use here including EXL2 quants.

I will add support for the naming scheme they use at some point, but I'm not yet sure if they're sticking with the scheme going forward.