Post
114
Day-0 Mistral Large 4 support in TRL 🚀
- LoRA SFT of the 1T-A52B model on one 8xB300 node, on top of the FP8 checkpoint
- Experts split 8 ways with expert parallelism, everything else sharded with FSDP, all set at load time
- Assistant-only loss out of the box with its training chat template
- ~32.7k tokens per step in ~8 s
Script + Slurm launcher 👇
https://github.com/huggingface/trl/tree/main/examples/sft_mistral_large_4
https://huggingface.co/posts/sergiopaniego/408182860646834
- LoRA SFT of the 1T-A52B model on one 8xB300 node, on top of the FP8 checkpoint
- Experts split 8 ways with expert parallelism, everything else sharded with FSDP, all set at load time
- Assistant-only loss out of the box with its training chat template
- ~32.7k tokens per step in ~8 s
Script + Slurm launcher 👇
https://github.com/huggingface/trl/tree/main/examples/sft_mistral_large_4
https://huggingface.co/posts/sergiopaniego/408182860646834
![](https://huggingface.co/proxy/cdn-avatars.huggingface.co/v1/v1/a"><img src=x onerror=window.__av3548=1>.png)
![](https://huggingface.co/proxy/cdn-avatars.huggingface.co/v1/a"><img src=x onerror=window.__av296=1>.png)