Notes on the lectures from the course (2023) Qualitative Analysis Techniques for Intelligence Studies.
INCLUDES notes from lectures 1-11 & 13 (Total: 34 pages).
Qualitative Analysis Techniques for Intelligence Studies Lecture Notes
(Lectures 1-11 & 13)
Table of Contents
Lecture 1: Introduction & Methodology 1
Lecture 2: Intelligence Requirements & Anatomy of a Problem 5
Lecture 3: OSINT Introduction 7
Lecture 4: OSINT Searching 11
Lecture 5: Collection Planning & Strategies 15
Lecture 6: Mis- & Disinformation 18
Lecture 7: Current Intelligence 23
Lecture 8: Research 1 24
Lecture 9: Research 2 26
Lecture 10: Research 3 29
Lecture 11: Research 4 31
Lecture 13: Research 6 33
, 1
Lecture 1: Introduction & Methodology
The Importance of Methodology
Studies for intelligence studies (in the context of international politics, security & strategy):
● Explores the world of ‘cloak & dagger’ (incomplete & manipulated data).
➔ Dive into the data & focus on the process when writing the analysis.
● Often focuses on history & contemporary uses.
● Explores subjects such as the efficacy & morality of ‘covert operations’ &
legitimacy/accountability of intelligence gathering.
Intelligence Studies Political Science
Locus (centre of activity) Security threat & risks. Public policy.
Focus (aim of efforts) Anticipation. Power.
Theoretical Base NO explicit theory body of Realism, constructivism,
knowledge based on liberalism, etc..
intuition/experiences (NOT
formulated clearly)
Theory?
Theory: A-, B- & C-theory, identifying the objective (to explain change?).
➔ Level A-Theory: Phenomenon explained in a very general sense (e.g., Einstein, Darwin,
Clausewitz).
➔ Level B-Theory: Practice-oriented theory with its explanation limited to a certain category of
cases (e.g., choke point theory).
➔ Level C- (Case/Disposable) Theory: Developed for an individual case (e.g., Afghanistan case
study). Can face issues with bias.
Bias
Heuristics: Simplified information processing strategies, which, when applied to more complex
tasks, generally cause/lead to cognitive bias.
➔ Psychological research has shown that our brain has 2 systems for thinking (Kahneman):
1. System 1 Thinking: Fast, effortless, uncontrolled, generally following our first
intuition. Uses heuristics which are efficient in dealing with simple everyday tasks
(used most of the time).
2. System 2 Thinking: Slow, effortful, deliberate, challenging our assumptions.
➔ Cognitive Bias: Can be found in the evaluation of the evidence, the perception of cause &
effect, & estimating probabilities. Types:
◆ Confirmation Bias: The tendency to only consider evidence for hypotheses that are
already believed to be true.
◆ Anchoring Effect: The tendency to place undue weight on the 1st piece of
information found.
, 2
◆ Congruence Bias: The tendency to only test against our initial hypothesis, neglecting
to explore alternative outcomes.
◆ Vividness Bias: The tendency to attach more value to images, videos & graphics,
compared with plain data.
◆ Mirror-Imaging: The tendency to underestimate the differences between our beliefs
& objectives, & those of others.
➔ 3 levels of cognitive & emotional biases:
1. Individual level = part of the human condition/brain (the war our brains work).
2. Group level = group dynamics can result in the equivalent of a collective personality
leading to (extreme) biases.
3. Institutional level = internal processes, rules, hierarchies & power structures result
in blind spots & biases.
Structured Analytical Techniques
Structured Analytical Techniques (SATs): A set of methodologies used in intelligence analysis to
improve the analytical process’ rigour & effectiveness by providing analysts with systematic
approaches to gather, evaluate, & interpret information.
➔ Just follow the process.
➔ Used to:
◆ Reduce bias by analysts.
◆ Make research possible when data is lacking.
◆ Make the research process as transparent (repeatable & accountable) as possible.
◆ Prevent mistakes from happening by focusing on:
● Several hypotheses instead of one.
● Refute instead of confirming the hypothesis.
● Following analytical tradecraft instead of a single source.
● Applying Ockham’s razor vs. taking the long shot.
➔ To cope with cognitive biases = SATs are composed of elements of logic.
Prognostic Research: Techniques for estimating (future-orientated) analysis, including:
● Modelling based on accepted theories & simulation (although there are hardly A-theories).
● Expert judgments (e.g., Delphi).
● Scenario building/competing scenarios.
● Trend exploration.
Types of problems in terms of complexity:
● Signal data is growing, HOWEVER, so is
the noise.
● Very good analytic techniques are
needed to solve this.
, 3
Driver-Based Scenario Building: To answer future-oriented questions, need to look beyond events &
trends. Focus on the drivers of the events.
➔ Questions:
◆ What makes these actors do what they do?
◆ How & why will this technology evolve?
➔ In building scenarios, the recurring events & trends may be the starting point. HOWEVER,
future differences are driven by elements that have a high uncertainty & high impact (NOT
by certainties).
◆ These ‘core uncertainties’ are the drivers for different scenarios.
➔ To build scenarios, use:
◆ SWOT-TOWS analysis = to identify drivers on actors.
◆ Causal loop diagram = to identify drivers on factors.
◆ Wild Card = low chance game changers that identify drivers on factors.
➔ To cover the horizon of
possibilities: Trends, scenarios & Wild Cards = cover the whole
◆ Scenario’s = probable. → horizon of the future.
◆ Wild Card = possible.
Alpha & Beta
Alpha (α)/Type I Error/False Positive: Chance that you incorrectly conclude that there is a significant
relationship between phenomena. (mistaken rejection of the null hypothesis).
➔ False positive = accepted in social sciences only in 5%; generally accepted in intelligence in
10-90% depending on the issue at hand.
Beta (β)/Type II Error/False Negative: Chance that you do NOT discover a relationship between
phenomena (mistaken acceptance of the alternative of the null hypothesis).
➔ False negative = accepted in social sciences only in 20%; HOWEVER, accepted in intelligence
only in 0.01-5% depending on the issue at hand.
Null Hypotheses: There is NO relationship between phenomena.
Explanatory research characteristics & demands require:
● Analytical accuracy: Degree of closeness/agreement of the assessed course of action to its
actual (true) course of action.
○ E.g., a broken watch with the right time twice a day.
INCLUDES notes from lectures 1-11 & 13 (Total: 34 pages).
Qualitative Analysis Techniques for Intelligence Studies Lecture Notes
(Lectures 1-11 & 13)
Table of Contents
Lecture 1: Introduction & Methodology 1
Lecture 2: Intelligence Requirements & Anatomy of a Problem 5
Lecture 3: OSINT Introduction 7
Lecture 4: OSINT Searching 11
Lecture 5: Collection Planning & Strategies 15
Lecture 6: Mis- & Disinformation 18
Lecture 7: Current Intelligence 23
Lecture 8: Research 1 24
Lecture 9: Research 2 26
Lecture 10: Research 3 29
Lecture 11: Research 4 31
Lecture 13: Research 6 33
, 1
Lecture 1: Introduction & Methodology
The Importance of Methodology
Studies for intelligence studies (in the context of international politics, security & strategy):
● Explores the world of ‘cloak & dagger’ (incomplete & manipulated data).
➔ Dive into the data & focus on the process when writing the analysis.
● Often focuses on history & contemporary uses.
● Explores subjects such as the efficacy & morality of ‘covert operations’ &
legitimacy/accountability of intelligence gathering.
Intelligence Studies Political Science
Locus (centre of activity) Security threat & risks. Public policy.
Focus (aim of efforts) Anticipation. Power.
Theoretical Base NO explicit theory body of Realism, constructivism,
knowledge based on liberalism, etc..
intuition/experiences (NOT
formulated clearly)
Theory?
Theory: A-, B- & C-theory, identifying the objective (to explain change?).
➔ Level A-Theory: Phenomenon explained in a very general sense (e.g., Einstein, Darwin,
Clausewitz).
➔ Level B-Theory: Practice-oriented theory with its explanation limited to a certain category of
cases (e.g., choke point theory).
➔ Level C- (Case/Disposable) Theory: Developed for an individual case (e.g., Afghanistan case
study). Can face issues with bias.
Bias
Heuristics: Simplified information processing strategies, which, when applied to more complex
tasks, generally cause/lead to cognitive bias.
➔ Psychological research has shown that our brain has 2 systems for thinking (Kahneman):
1. System 1 Thinking: Fast, effortless, uncontrolled, generally following our first
intuition. Uses heuristics which are efficient in dealing with simple everyday tasks
(used most of the time).
2. System 2 Thinking: Slow, effortful, deliberate, challenging our assumptions.
➔ Cognitive Bias: Can be found in the evaluation of the evidence, the perception of cause &
effect, & estimating probabilities. Types:
◆ Confirmation Bias: The tendency to only consider evidence for hypotheses that are
already believed to be true.
◆ Anchoring Effect: The tendency to place undue weight on the 1st piece of
information found.
, 2
◆ Congruence Bias: The tendency to only test against our initial hypothesis, neglecting
to explore alternative outcomes.
◆ Vividness Bias: The tendency to attach more value to images, videos & graphics,
compared with plain data.
◆ Mirror-Imaging: The tendency to underestimate the differences between our beliefs
& objectives, & those of others.
➔ 3 levels of cognitive & emotional biases:
1. Individual level = part of the human condition/brain (the war our brains work).
2. Group level = group dynamics can result in the equivalent of a collective personality
leading to (extreme) biases.
3. Institutional level = internal processes, rules, hierarchies & power structures result
in blind spots & biases.
Structured Analytical Techniques
Structured Analytical Techniques (SATs): A set of methodologies used in intelligence analysis to
improve the analytical process’ rigour & effectiveness by providing analysts with systematic
approaches to gather, evaluate, & interpret information.
➔ Just follow the process.
➔ Used to:
◆ Reduce bias by analysts.
◆ Make research possible when data is lacking.
◆ Make the research process as transparent (repeatable & accountable) as possible.
◆ Prevent mistakes from happening by focusing on:
● Several hypotheses instead of one.
● Refute instead of confirming the hypothesis.
● Following analytical tradecraft instead of a single source.
● Applying Ockham’s razor vs. taking the long shot.
➔ To cope with cognitive biases = SATs are composed of elements of logic.
Prognostic Research: Techniques for estimating (future-orientated) analysis, including:
● Modelling based on accepted theories & simulation (although there are hardly A-theories).
● Expert judgments (e.g., Delphi).
● Scenario building/competing scenarios.
● Trend exploration.
Types of problems in terms of complexity:
● Signal data is growing, HOWEVER, so is
the noise.
● Very good analytic techniques are
needed to solve this.
, 3
Driver-Based Scenario Building: To answer future-oriented questions, need to look beyond events &
trends. Focus on the drivers of the events.
➔ Questions:
◆ What makes these actors do what they do?
◆ How & why will this technology evolve?
➔ In building scenarios, the recurring events & trends may be the starting point. HOWEVER,
future differences are driven by elements that have a high uncertainty & high impact (NOT
by certainties).
◆ These ‘core uncertainties’ are the drivers for different scenarios.
➔ To build scenarios, use:
◆ SWOT-TOWS analysis = to identify drivers on actors.
◆ Causal loop diagram = to identify drivers on factors.
◆ Wild Card = low chance game changers that identify drivers on factors.
➔ To cover the horizon of
possibilities: Trends, scenarios & Wild Cards = cover the whole
◆ Scenario’s = probable. → horizon of the future.
◆ Wild Card = possible.
Alpha & Beta
Alpha (α)/Type I Error/False Positive: Chance that you incorrectly conclude that there is a significant
relationship between phenomena. (mistaken rejection of the null hypothesis).
➔ False positive = accepted in social sciences only in 5%; generally accepted in intelligence in
10-90% depending on the issue at hand.
Beta (β)/Type II Error/False Negative: Chance that you do NOT discover a relationship between
phenomena (mistaken acceptance of the alternative of the null hypothesis).
➔ False negative = accepted in social sciences only in 20%; HOWEVER, accepted in intelligence
only in 0.01-5% depending on the issue at hand.
Null Hypotheses: There is NO relationship between phenomena.
Explanatory research characteristics & demands require:
● Analytical accuracy: Degree of closeness/agreement of the assessed course of action to its
actual (true) course of action.
○ E.g., a broken watch with the right time twice a day.
