[Suggestion] Instruction Fine-Tuning - SFT Module

[ArlindKadra]

Thanks for taking the time into developing this interesting course. I wanted to suggest that regarding the SFT module in Chapter 1, the bigcode/the-stack-smol dataset seems to break the flow a bit. Since it is not an instruction-tuning dataset, but more a domain-specific dataset with no instruction following. As such, it does not have the question/answering pairs.

Based on that, if you train on the dataset, the response to the prompt is the same as before. Maybe switching it up to the openai/gsm8k dataset, or something similar? That way one would still have to prepare the dataset before feeding it to the SFTTrainer.

[burtenshaw]

Thanks. That's a nice suggestion. Are you interested in opening a PR?

[ArlindKadra]

I can give it a try, however, I am not able to do it in a timely manner. I will be available starting from the 18th.

[burtenshaw]

No worries @ArlindKadra . Come back when you're ready and check if it still needs doing.

Thanks for the issue.

[asvskartheek]

Wanted to add few of my queries here, instead of creating a new issue. (I hope this is ok)

1. What is the task the model is being trained on?

• My guess is Casual LM with cross entropy loss.

2. If it is Casual LM then are we in anyway also ignoring special characters and phrases like <|im_start|>?
   2b. How are we forcing the end of generation after just the assistant turn?

3. The everyday conversations dataset has several columns, where are we specifying to train on the "chatml" templated messages column of the dataset?

Tried to get answers for these queries in SFT couldn't get answers. Maybe adding bit more detail in "The Finetuning Process" sub-section of the Supervised Fine tuning page would be helpful.