Should We Add Model Merging to the Course?

[duydl]

Hi!

I wanted to start a conversation about adding a section on Model Merging to the course, either as a new module or as part of an existing one (like Parameter-efficient Fine-tuning). Tools like MergeKit could make it convenient to combine pre-trained models, creating something more versatile or specialized.

Why This Could Be Useful

• It’s Practical: Model merging lets you blend the strengths of different models, which can be handy for domain adaptation or multitasking.
• Fits the Course: The course is about aligning models to specific needs, and merging is another tool to do that.
• Works with LoRA: We could merge models first and then fine-tune them efficiently with LoRA for a specific task.

How It Could Fit In

• Option 1: Add a dedicated Model Merging module that covers methods like weight averaging, layer merging, and ensemble distillation.
• Option 2: Expand the Parameter-efficient Fine-tuning module to include a section on model merging as a complementary approach.

What do you think?

[burtenshaw]

This is a cool idea. I could definitely see model merging in the course.

However, I feel like most of the topics on the current roadmap should come before model merging. What do you think?