Support Falcon2-11B

[reneleonhardt]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• I carefully followed the README.md.
• I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• I reviewed the Discussions, and have a new bug or useful enhancement to share.

Feature Description

Add support for the new Falcon2-11B model if there are differences compared to the current Falcon-7B support.
https://falconllm.tii.ae/falcon-models.html
https://huggingface.co/tiiuae/falcon-11B
https://huggingface.co/DevQuasar/falcon2-11B-GGUF

Marketing speak:

The Next-Gen Falcon 2 series launches an AI model that is Open-Source, multilingual, and multimodal – and is the only AI model with Vision-to-Language capabilities.

The new Falcon 2 11B outperforms Meta’s Llama 3 8B, and performs on par with the leading Google Gemma 7B Model, as independently verified by the Hugging Face Leaderboard.

Motivation

Users should be able to choose from another high-quality open model.

Possible Implementation

The converter and server should fully support the new model with correct tokenization.

[bartowski1182]

These load and generate without issue:

https://huggingface.co/bartowski/falcon-11B-GGUF

[reneleonhardt]

Thank you, then I just can't get the prompt template right it seems, it doesn't stop generating hallucinated user prompts 😅