UNIVERSITY OF OXFORD | MSC AI
FOR BUSINESS
MODULE: GENERATIVE AI & LARGE LANGUAGE MODELS (LLMs)
TERM 1 LECTURE NOTES: TRANSFORMERS, RAG, AND AGENTIC SYSTEMS
1. THE GENERATIVE REVOLUTION
In 2026, Generative AI (GenAI) is defined as a class of AI that can create new
content (text, images, code, video) by learning the underlying statistical patterns of
human-generated data.
1.1 The Shift to "Foundation Models"
The Oxford curriculum focuses on Foundation Models—massive models trained on
broad data that can be adapted to a wide range of downstream business tasks.
• Key Characteristic: Emergent properties (abilities like reasoning or coding
that weren't explicitly programmed but "emerged" during scale).
2. CORE ARCHITECTURE: THE TRANSFORMER
Everything in modern LLMs starts with the Transformer architecture (introduced by
Google in the "Attention is All You Need" paper).
• The Attention Mechanism: Allows the model to weigh the importance of
different words in a sentence, regardless of their distance.
• Tokenization: LLMs do not read "words"; they read Tokens (numerical
representations of word fragments).
• Positional Encoding: Since Transformers process all tokens simultaneously
(parallelization), they use positional encodings to remember the order of
the words.
FOR BUSINESS
MODULE: GENERATIVE AI & LARGE LANGUAGE MODELS (LLMs)
TERM 1 LECTURE NOTES: TRANSFORMERS, RAG, AND AGENTIC SYSTEMS
1. THE GENERATIVE REVOLUTION
In 2026, Generative AI (GenAI) is defined as a class of AI that can create new
content (text, images, code, video) by learning the underlying statistical patterns of
human-generated data.
1.1 The Shift to "Foundation Models"
The Oxford curriculum focuses on Foundation Models—massive models trained on
broad data that can be adapted to a wide range of downstream business tasks.
• Key Characteristic: Emergent properties (abilities like reasoning or coding
that weren't explicitly programmed but "emerged" during scale).
2. CORE ARCHITECTURE: THE TRANSFORMER
Everything in modern LLMs starts with the Transformer architecture (introduced by
Google in the "Attention is All You Need" paper).
• The Attention Mechanism: Allows the model to weigh the importance of
different words in a sentence, regardless of their distance.
• Tokenization: LLMs do not read "words"; they read Tokens (numerical
representations of word fragments).
• Positional Encoding: Since Transformers process all tokens simultaneously
(parallelization), they use positional encodings to remember the order of
the words.
