Lecture 1 - Intro
1. Introduction & Motivating Examples
Definition: NLG is the process of automatically generating human-readable
text from structured or unstructured data.
Two Key Scenarios:
1. Data-to-text: Converting structured database triples (e.g., company info)
into coherent sentences.
2. Open-ended text generation: Answering broad knowledge-based
questions (e.g., "Who is Slash?") using Large Language Models (LLMs).
Key Questions in NLG:
What is the model’s output based on?
What knowledge is it grounded in?
Can the same architecture handle both structured and unstructured
generation?
How should outputs be evaluated?
2. Types of NLG
Text-to-Text: Summarization, translation.
Data-to-Text: Weather reports, sports recaps, medical records.
Media-to-Text: Image/video captioning.
Open-ended Generation: Story generation, chatbot dialogues.
Modern NLG relies on Transformer-based neural networks, which unify
approaches across these tasks.
3. Content Planning vs. Realization
Strategic Choices (What to Say?):
Lecture 1 - Intro 1
, Content selection, organization.
Influenced by audience, language, and communicative intent.
Tactical Choices (How to Say It?):
Linguistic realization: word choice, phrasing, grammar.
Different languages and cultures may describe the same scene differently.
Classic NLG Pipeline
1. Content Selection – Deciding relevant information.
2. Rhetorical Structuring – Organizing content logically.
3. Lexicalization – Choosing words.
4. Sentence Aggregation – Combining sentences effectively.
5. Referring Expression Generation – Naming entities appropriately.
6. Surface Realization – Constructing grammatically correct sentences.
Modern neural approaches implicitly handle these steps in an end-to-end model.
4. Evolution of NLG: From Rule-Based to Neural
Networks
Early NLG Systems
Rule-Based (1990s-2000s):
Example: FoG (Forecast Generator) – Generated weather reports using
pre-defined templates.
Strength: Ensures grammatical correctness.
Weakness: Limited flexibility and adaptability.
Template-Based (2010s):
Example: Automated journalism – Sports, finance, and weather reports
generated by filling in templates.
Strength: Reliable factual accuracy.
Lecture 1 - Intro 2
, Weakness: Lacks linguistic diversity.
Transition to Machine Learning & Neural Networks
Statistical & Hybrid Methods (2010s):
Used corpora of text to learn patterns.
Early NLG still followed pipeline structures but with statistical learning.
Deep Learning & Transformers (2017-Present):
Transformer-based models (e.g., GPT-3, T5) allow end-to-end
generation.
Advantages:
Can handle multiple NLG tasks in a single model.
Generates more fluent and diverse text.
Challenges:
Hallucinations (false information).
Lack of explicit control over structure.
5. Applications of NLG
Automated Journalism (e.g., AP News uses NLG for financial reports).
Weather Forecasts (e.g., Meteo France uses NLG for marine forecasts).
Healthcare (e.g., personalized patient letters).
Gaming & Storytelling (e.g., AI Dungeon for interactive fiction).
Finance, E-commerce, Chatbots, and Report Generation.
6. Ethical & Practical Concerns
Key Evaluation Dimensions
World Dimension (Truth & Accuracy):
Lecture 1 - Intro 3
, Hallucination: Model generates false information.
Faithfulness: Model should stay consistent with input data.
Interpersonal Dimension (Impact on Users):
Avoiding biased or harmful text.
Ethical concerns in AI-generated misinformation.
Language Dimension (Fluency & Coherence):
Ensuring grammatically correct and contextually appropriate text.
Balancing creativity vs factual accuracy.
Mitigating Risks
Retrieval-Augmented Generation (RAG):
Combines LLMs with external knowledge retrieval (e.g., Wikipedia) to
ensure accuracy.
Human Review & Transparency:
AI-generated text should be reviewed by humans in critical applications.
Labeling AI-generated content for public awareness.
8. Key Takeaways
1. NLG converts structured/unstructured data into natural language.
2. Traditional pipeline models separate content selection from realization.
3. Modern deep learning models unify NLG tasks but introduce new
challenges.
4. NLG is widely used in journalism, healthcare, gaming, and finance.
5. Evaluation of NLG is complex, involving factuality, fluency, and ethical
concerns.
6. Retrieval-Augmented Generation (RAG) is a promising method to improve
factual accuracy.
Lecture 1 - Intro 4
1. Introduction & Motivating Examples
Definition: NLG is the process of automatically generating human-readable
text from structured or unstructured data.
Two Key Scenarios:
1. Data-to-text: Converting structured database triples (e.g., company info)
into coherent sentences.
2. Open-ended text generation: Answering broad knowledge-based
questions (e.g., "Who is Slash?") using Large Language Models (LLMs).
Key Questions in NLG:
What is the model’s output based on?
What knowledge is it grounded in?
Can the same architecture handle both structured and unstructured
generation?
How should outputs be evaluated?
2. Types of NLG
Text-to-Text: Summarization, translation.
Data-to-Text: Weather reports, sports recaps, medical records.
Media-to-Text: Image/video captioning.
Open-ended Generation: Story generation, chatbot dialogues.
Modern NLG relies on Transformer-based neural networks, which unify
approaches across these tasks.
3. Content Planning vs. Realization
Strategic Choices (What to Say?):
Lecture 1 - Intro 1
, Content selection, organization.
Influenced by audience, language, and communicative intent.
Tactical Choices (How to Say It?):
Linguistic realization: word choice, phrasing, grammar.
Different languages and cultures may describe the same scene differently.
Classic NLG Pipeline
1. Content Selection – Deciding relevant information.
2. Rhetorical Structuring – Organizing content logically.
3. Lexicalization – Choosing words.
4. Sentence Aggregation – Combining sentences effectively.
5. Referring Expression Generation – Naming entities appropriately.
6. Surface Realization – Constructing grammatically correct sentences.
Modern neural approaches implicitly handle these steps in an end-to-end model.
4. Evolution of NLG: From Rule-Based to Neural
Networks
Early NLG Systems
Rule-Based (1990s-2000s):
Example: FoG (Forecast Generator) – Generated weather reports using
pre-defined templates.
Strength: Ensures grammatical correctness.
Weakness: Limited flexibility and adaptability.
Template-Based (2010s):
Example: Automated journalism – Sports, finance, and weather reports
generated by filling in templates.
Strength: Reliable factual accuracy.
Lecture 1 - Intro 2
, Weakness: Lacks linguistic diversity.
Transition to Machine Learning & Neural Networks
Statistical & Hybrid Methods (2010s):
Used corpora of text to learn patterns.
Early NLG still followed pipeline structures but with statistical learning.
Deep Learning & Transformers (2017-Present):
Transformer-based models (e.g., GPT-3, T5) allow end-to-end
generation.
Advantages:
Can handle multiple NLG tasks in a single model.
Generates more fluent and diverse text.
Challenges:
Hallucinations (false information).
Lack of explicit control over structure.
5. Applications of NLG
Automated Journalism (e.g., AP News uses NLG for financial reports).
Weather Forecasts (e.g., Meteo France uses NLG for marine forecasts).
Healthcare (e.g., personalized patient letters).
Gaming & Storytelling (e.g., AI Dungeon for interactive fiction).
Finance, E-commerce, Chatbots, and Report Generation.
6. Ethical & Practical Concerns
Key Evaluation Dimensions
World Dimension (Truth & Accuracy):
Lecture 1 - Intro 3
, Hallucination: Model generates false information.
Faithfulness: Model should stay consistent with input data.
Interpersonal Dimension (Impact on Users):
Avoiding biased or harmful text.
Ethical concerns in AI-generated misinformation.
Language Dimension (Fluency & Coherence):
Ensuring grammatically correct and contextually appropriate text.
Balancing creativity vs factual accuracy.
Mitigating Risks
Retrieval-Augmented Generation (RAG):
Combines LLMs with external knowledge retrieval (e.g., Wikipedia) to
ensure accuracy.
Human Review & Transparency:
AI-generated text should be reviewed by humans in critical applications.
Labeling AI-generated content for public awareness.
8. Key Takeaways
1. NLG converts structured/unstructured data into natural language.
2. Traditional pipeline models separate content selection from realization.
3. Modern deep learning models unify NLG tasks but introduce new
challenges.
4. NLG is widely used in journalism, healthcare, gaming, and finance.
5. Evaluation of NLG is complex, involving factuality, fluency, and ethical
concerns.
6. Retrieval-Augmented Generation (RAG) is a promising method to improve
factual accuracy.
Lecture 1 - Intro 4
