Generative AI & Prompt Engineering
- Study Guide
How LLMs work, prompting techniques, RAG, agents and responsible AI use
Complete Study Guide | Key Concepts | Examples | Practice Questions
, 1. How Large Language Models Work
LLMs predict the next token from context, trained on massive text. Understanding this explains both their power
and their failure modes.
- Transformer architecture: attention lets the model weigh relevant context.
- Training: pre-training (predict next token) then fine-tuning (instructions, RLHF).
- Tokens are word pieces; context window limits how much text the model sees at once.
- Temperature controls randomness: low = focused/deterministic, high = creative/varied.
- Hallucination: fluent, confident, wrong - always verify facts, code and citations.
TIP: Mental model: the LLM is a brilliant intern with no memory of your project - brief it well every time.
Page 1
- Study Guide
How LLMs work, prompting techniques, RAG, agents and responsible AI use
Complete Study Guide | Key Concepts | Examples | Practice Questions
, 1. How Large Language Models Work
LLMs predict the next token from context, trained on massive text. Understanding this explains both their power
and their failure modes.
- Transformer architecture: attention lets the model weigh relevant context.
- Training: pre-training (predict next token) then fine-tuning (instructions, RLHF).
- Tokens are word pieces; context window limits how much text the model sees at once.
- Temperature controls randomness: low = focused/deterministic, high = creative/varied.
- Hallucination: fluent, confident, wrong - always verify facts, code and citations.
TIP: Mental model: the LLM is a brilliant intern with no memory of your project - brief it well every time.
Page 1
