HIGH-YIELD REVISION GUIDE & CHEAT SHEET
Transformers & Large Language Mod-
els (LLMs) Complete Semester Guide
High-Yield Comprehensive Guide, Attention Mathematics, RoPE Derivations,
KV-Cache VRAM Calculations, LoRA Proofs & Quick-Cram Appendix
Artificial Intelligence & NLP Engineering | Course Code: University Of California - Berkeley -
CS 288 | 2024-2025 Comprehensive Edition
📘 WHAT’S INSIDE THIS GUIDE:
✔ Master Scaled Dot-Product Attention: Variance proof of 1/sqrt(d_k) scaling factor and causal
autoregressive masks
✔ Positional Encodings: Complex-plane 2D rotation derivations for Rotary Position Embeddings
(RoPE)
✔ Autoregressive Generation & Memory: Detailed hand-calculated KV-Cache VRAM formulas
across MHA, MQA, and GQA
✔ Parameter-Efficient Fine-Tuning (PEFT): Step-by-step mathematical trace of LoRA rank-r matri-
ces, parameter savings, and inference merge
✔ Training Stability: Mathematical analysis of Pre-LN vs. Post-LN gradient norms, SwiGLU acti-
vations, and RMSNorm
✔ Quantization Foundations: Post-Training Quantization (PTQ) vs. QLoRA 4-bit NormalFloat (NF4)
and double quantization
✔ 10 Deadly LLM Exam Traps (Attention FLOPs vs parameter scaling, causal mask -infinity,
temperature sampling)
✔ APPENDIX: 1-Hour Emergency Pre-Exam Quick-Cram Cheat Sheet with master architecture
table and 25 active recall flashcards
, Artificial Intelligence & NLP Engineering · University OfTransformers & Large Language Models (LLMs) Complete
California - Berkeley - CS 288 Semester Guide
Disclaimer: This is an independently created academic study aid. It is not affiliated with, endorsed by, or sponsored by any specific
university, textbook author, or publisher.
Independent Study Aid. Not affiliated with or endorsed by any specific university or Page 2 of 8
publisher.
Transformers & Large Language Mod-
els (LLMs) Complete Semester Guide
High-Yield Comprehensive Guide, Attention Mathematics, RoPE Derivations,
KV-Cache VRAM Calculations, LoRA Proofs & Quick-Cram Appendix
Artificial Intelligence & NLP Engineering | Course Code: University Of California - Berkeley -
CS 288 | 2024-2025 Comprehensive Edition
📘 WHAT’S INSIDE THIS GUIDE:
✔ Master Scaled Dot-Product Attention: Variance proof of 1/sqrt(d_k) scaling factor and causal
autoregressive masks
✔ Positional Encodings: Complex-plane 2D rotation derivations for Rotary Position Embeddings
(RoPE)
✔ Autoregressive Generation & Memory: Detailed hand-calculated KV-Cache VRAM formulas
across MHA, MQA, and GQA
✔ Parameter-Efficient Fine-Tuning (PEFT): Step-by-step mathematical trace of LoRA rank-r matri-
ces, parameter savings, and inference merge
✔ Training Stability: Mathematical analysis of Pre-LN vs. Post-LN gradient norms, SwiGLU acti-
vations, and RMSNorm
✔ Quantization Foundations: Post-Training Quantization (PTQ) vs. QLoRA 4-bit NormalFloat (NF4)
and double quantization
✔ 10 Deadly LLM Exam Traps (Attention FLOPs vs parameter scaling, causal mask -infinity,
temperature sampling)
✔ APPENDIX: 1-Hour Emergency Pre-Exam Quick-Cram Cheat Sheet with master architecture
table and 25 active recall flashcards
, Artificial Intelligence & NLP Engineering · University OfTransformers & Large Language Models (LLMs) Complete
California - Berkeley - CS 288 Semester Guide
Disclaimer: This is an independently created academic study aid. It is not affiliated with, endorsed by, or sponsored by any specific
university, textbook author, or publisher.
Independent Study Aid. Not affiliated with or endorsed by any specific university or Page 2 of 8
publisher.
