Supervised Fine-tuning smolLM

[usametov]

I run SFT notebook, it's great. However, I wasn't impressed with the haiku. I want to build my own dataset to fine-tune smolLM. Are there any guidelines to evaluate the dataset quality? Also, how long would it take to fine-tune smolLM?

[burtenshaw]

That's a really nice observation!

I'd suggest you stick with course because we're going to learn how to evaluate models and build datasets, so you'll definitely be able to improve the Haiku.