← Attention & Transformers
Pro topic
Transformer Training Craft
Stability tricks used in modern LLM pretraining recipes.
Transformer Training Craft
Stability toolkit
- Pre-norm vs post-norm
- Warmup + cosine decay
- Gradient clipping
- Mixed precision with loss scaling
Data quality > clever tricks
For language models, corpus curation and deduplication often beat exotic architectural tweaks.
Continue this theory with Pro
You've reached the free preview. Pro unlocks the full write-up, videos, quiz scoring, and cheatsheet.