Article
Fine-tuning LLMs with Singular Value Decomposition
fractalego
• • 14Alberto Cetoli PRO
fractalego
AI & ML interests
Entity/relation extraction, Q&A, Summarisation
Recent Activity
reacted to sergiopaniego's post with 🔥 13 days ago
Mistral Large 4 (Le Chonk 🐈) is now live on the Hub and you can post-train it with TRL today!
SFT + LoRA example ready, runs on a single 8xB300 node
> model: https://huggingface.co/mistralai/Mistral-Large-4-1T-A52B
> TRL example: https://github.com/huggingface/trl/tree/main/examples/sft_mistral_large_4
> blog: https://huggingface.co/blog/mistral-large4 reacted to sergiopaniego's post with 😎 13 days ago
if you want to deep dive into how the new @MistralAI Large 4 was built, we sat down with the team to learn straight from them!
scaling, specialization, evals, cyber, sandboxing and why open weights matter
a really detailed look at what it takes to build and release an open frontier model, super recommended
https://www.youtube.com/watch?v=FczbSauVvjwOrganizations
published an article over 2 years ago
published an article over 2 years ago
Article
The LASER technique: Evaluating SVD compression
fractalego
• • 9