W1.1
ARTICLE BY BUYALSKAYA, GALLO & CAMERER (2020) – THE GOLDEN AGE
OF SOCIAL SCIENCE
Big Data: Data that contains greater variety, arriving in increasing volumes and with more
velocity – The explosive growth of available data and computational power
↓
Interdisciplinarity: Active collaboration among scientists with different training, as opposed to
one researcher passively borrowing ideas from other fields
- Lingua franca: A common trade language across disciplines – Most useful language is
one where all disciplines adopt the best language from whichever discipline has
described an idea most effectively – Will enable diverse teams to tackle
multidimensional problems and create innovations for better well-being
Challenges in interdisciplinary research:
1. The question of where and how information is accumulated – Solution: Journals
should seriously consider and publish high-quality interdisciplinary research, even
when it falls outside their traditional sphere of work
2. Academics are often encouraged to remain focused on contributing to their respective
subject areas – Solution: In training and hiring new PhDs, departments and
organisations should consider ways to expose trainees to more breadth in social
science and develop better ways to evaluate interdisciplinary research
3. Different social science disciplines often have different tools and norms – Solution:
Researchers should be encouraged to define best practices for how the relevant sharing
of data and code should be done
4. The creation of unifying frameworks to explain behaviours across disciplines –
Solution: Trade-minded scholars should be humble and open to learning from other
social scientists who have long histories of concepts and methods to share
ARTICLE BY BORSBOOM, VAN DER MAAS, DALEGE, KIEVIET & HAIG (2021) –
THEORY CONSTRUCTION METHODOLOGY: A PRACTICAL FRAMEWORK
FOR BUILDING THEORIES IN PSYCHOLOGY
Reproducibility crisis → Theory crisis: The field of psychology lacks an overarching theory-
construction program as exists in other disciplines
↓
Toothbrush problem: In psychology, theories are typically products of (small groups of)
individuals – Psychology lacks a coordinated program of theory construction (Mischel)
Lack of methodology in psychology:
1. Lack a collective, coordinated research program focussed on theory formation
2. Skills that are conducive to constructing theories are seldom taught in psychology
3. There is a strong focus on the hypothetico-deductive method (= the idea that science
progresses through repeated empirical tests of hypotheses entailed by theories)
,Theory construction methodology (TCM): A method for explanatory theory formation that is
designed to assist researchers in the development of theories – Based on Haig’s abductive
theory of method, in which scientific inquiry is a two-phase process in which empirical
phenomena are detected and then explained by theories that are built to understand them
- Theories have both predictive and explanatory virtues
The lack of explanatory theories in psychology hinders progress:
1. It creates the danger of inventing the wheel over and over again because we do not
have a good grasp on how different phenomena relate to each other
2. Without strong theories we cannot identify the most effective interventions for
changing a system in the desired way
3. Without theories we often do not know where to look when designing new studies
Theory construction methodology:
1. Step 1: Identify a set of phenomena that need explanation
2. Step 2: Come up with a proto-theory (= a set of principles that putatively explains the
phenomena)
- Analogical abduction: Drawing on existing stock of knowledge and framing
unfamiliar situations as if they were similar to familiar situations – Explanatory
inference
3. Step 3: Formalise the proto-theory and phenomena – Mathematical equation
4. Step 4: Evaluate how well the resulting formal theory actually explains the phenomena
5. Step 5: Overall evaluation of the theory
- Inference to the best explanation: Theories that are prized for their ability to
explain phenomena should be evaluated with respect to their explanatory
virtues
Distinction:
- Data: Relatively direct observations or
reports thereof – Pliable
- Phenomena: Stable and general features of
the world that scientists seek to explain – Empirical generalisations – Robust
- Theories: Help to explain the empirical phenomena that they are devised to explain
ARTICLE BY WAGENMAKERS, VAN DER MAAS & GRASMAN (2007) – AN EZ-
DIFFUSION MODEL FOR RESPONSE TIME AND ACCURACY
Cognitive psychometrics: The field of research that uses cognitive models for measurement
The speed-accuracy tradeoff: The complex relationship between an individual’s willingness to
respond slowly and make relatively fewer errors compared to their willingness to respond
quickly and make relatively more errors (E.g.: the Lexical Decision Task, in which the
participant’s task is to decide, as quickly and as accurately as possible, whether the word is a
real word in his language)
- Standard analysis: Just taking response time into account – Does not account for the
ubiquitous trade-off between reaction time and accuracy
, - Process model: Accounts for how people generate the observed data – Decomposition
of underlying processes – Help us to understand human cognition, decompose and
measure underlying processes and predict decisions
- Ratcliff’s Diffusion Model → EZ-Diffusion Model: Assumes that binary
decisions are based on a continuous process that fluctuates between two
possible outcomes – As soon as the process reaches a critical value, a decision
is made, and the corresponding response is executed – Different components of
the decision process are represented by different parameters of the model – A
continuous-time, continuous-state random-walk sequential sampling model
- Drift rate: The systematic influences that ‘drive’ the process
continuously in one direction (= v) –
Quantifies subject ability or task
difficulty
- Boundary separation: If speed is
Only the end result is
emphasised, the boundaries are close;
visible! The process is
if accuracy is emphasised, the
hypothesised
boundaries are wide (= a) –
Quantifies response caution
- Starting point: A priori
biases/expectations (= z)
- Non-decision time: The time that is needed to encode the stimulus and
to execute the response (= TER)
Much overlap = Task is difficult; lower drift rate
Little overlap = Task is easy; higher drift rate
Conclusion of the Ratcliff diffusion model: Older adults
are just as efficient in activating lexical content, but are
slower because they are more cautious (i.e. their boundary
separation is wider)
Why the diffusion model is not standardly applied as a
psychometric analysis tool:
1. It may take very many trials to obtain an accurate estimate of the error RT distribution
2. The parameter-fitting procedure is complex, and many experimental psychologists
will find the amount of effort prohibitive
LECTURE W1.1
Behavioural data science: A multidisciplinary field that aims to facilitate understanding,
prediction and change of human behaviour through the analysis of behaviourally defined
variables as they arise in large datasets, typically gathered using modern digital technology
and analysed with techniques for detecting patterns from high-dimensional data
- Multidisciplinary: Operates on the cross-roads of psychology, technology, statistics
and methodology
Skinner: ‘Human behaviour is possibly the most difficult subject ever submitted to statistical
analysis’
ARTICLE BY BUYALSKAYA, GALLO & CAMERER (2020) – THE GOLDEN AGE
OF SOCIAL SCIENCE
Big Data: Data that contains greater variety, arriving in increasing volumes and with more
velocity – The explosive growth of available data and computational power
↓
Interdisciplinarity: Active collaboration among scientists with different training, as opposed to
one researcher passively borrowing ideas from other fields
- Lingua franca: A common trade language across disciplines – Most useful language is
one where all disciplines adopt the best language from whichever discipline has
described an idea most effectively – Will enable diverse teams to tackle
multidimensional problems and create innovations for better well-being
Challenges in interdisciplinary research:
1. The question of where and how information is accumulated – Solution: Journals
should seriously consider and publish high-quality interdisciplinary research, even
when it falls outside their traditional sphere of work
2. Academics are often encouraged to remain focused on contributing to their respective
subject areas – Solution: In training and hiring new PhDs, departments and
organisations should consider ways to expose trainees to more breadth in social
science and develop better ways to evaluate interdisciplinary research
3. Different social science disciplines often have different tools and norms – Solution:
Researchers should be encouraged to define best practices for how the relevant sharing
of data and code should be done
4. The creation of unifying frameworks to explain behaviours across disciplines –
Solution: Trade-minded scholars should be humble and open to learning from other
social scientists who have long histories of concepts and methods to share
ARTICLE BY BORSBOOM, VAN DER MAAS, DALEGE, KIEVIET & HAIG (2021) –
THEORY CONSTRUCTION METHODOLOGY: A PRACTICAL FRAMEWORK
FOR BUILDING THEORIES IN PSYCHOLOGY
Reproducibility crisis → Theory crisis: The field of psychology lacks an overarching theory-
construction program as exists in other disciplines
↓
Toothbrush problem: In psychology, theories are typically products of (small groups of)
individuals – Psychology lacks a coordinated program of theory construction (Mischel)
Lack of methodology in psychology:
1. Lack a collective, coordinated research program focussed on theory formation
2. Skills that are conducive to constructing theories are seldom taught in psychology
3. There is a strong focus on the hypothetico-deductive method (= the idea that science
progresses through repeated empirical tests of hypotheses entailed by theories)
,Theory construction methodology (TCM): A method for explanatory theory formation that is
designed to assist researchers in the development of theories – Based on Haig’s abductive
theory of method, in which scientific inquiry is a two-phase process in which empirical
phenomena are detected and then explained by theories that are built to understand them
- Theories have both predictive and explanatory virtues
The lack of explanatory theories in psychology hinders progress:
1. It creates the danger of inventing the wheel over and over again because we do not
have a good grasp on how different phenomena relate to each other
2. Without strong theories we cannot identify the most effective interventions for
changing a system in the desired way
3. Without theories we often do not know where to look when designing new studies
Theory construction methodology:
1. Step 1: Identify a set of phenomena that need explanation
2. Step 2: Come up with a proto-theory (= a set of principles that putatively explains the
phenomena)
- Analogical abduction: Drawing on existing stock of knowledge and framing
unfamiliar situations as if they were similar to familiar situations – Explanatory
inference
3. Step 3: Formalise the proto-theory and phenomena – Mathematical equation
4. Step 4: Evaluate how well the resulting formal theory actually explains the phenomena
5. Step 5: Overall evaluation of the theory
- Inference to the best explanation: Theories that are prized for their ability to
explain phenomena should be evaluated with respect to their explanatory
virtues
Distinction:
- Data: Relatively direct observations or
reports thereof – Pliable
- Phenomena: Stable and general features of
the world that scientists seek to explain – Empirical generalisations – Robust
- Theories: Help to explain the empirical phenomena that they are devised to explain
ARTICLE BY WAGENMAKERS, VAN DER MAAS & GRASMAN (2007) – AN EZ-
DIFFUSION MODEL FOR RESPONSE TIME AND ACCURACY
Cognitive psychometrics: The field of research that uses cognitive models for measurement
The speed-accuracy tradeoff: The complex relationship between an individual’s willingness to
respond slowly and make relatively fewer errors compared to their willingness to respond
quickly and make relatively more errors (E.g.: the Lexical Decision Task, in which the
participant’s task is to decide, as quickly and as accurately as possible, whether the word is a
real word in his language)
- Standard analysis: Just taking response time into account – Does not account for the
ubiquitous trade-off between reaction time and accuracy
, - Process model: Accounts for how people generate the observed data – Decomposition
of underlying processes – Help us to understand human cognition, decompose and
measure underlying processes and predict decisions
- Ratcliff’s Diffusion Model → EZ-Diffusion Model: Assumes that binary
decisions are based on a continuous process that fluctuates between two
possible outcomes – As soon as the process reaches a critical value, a decision
is made, and the corresponding response is executed – Different components of
the decision process are represented by different parameters of the model – A
continuous-time, continuous-state random-walk sequential sampling model
- Drift rate: The systematic influences that ‘drive’ the process
continuously in one direction (= v) –
Quantifies subject ability or task
difficulty
- Boundary separation: If speed is
Only the end result is
emphasised, the boundaries are close;
visible! The process is
if accuracy is emphasised, the
hypothesised
boundaries are wide (= a) –
Quantifies response caution
- Starting point: A priori
biases/expectations (= z)
- Non-decision time: The time that is needed to encode the stimulus and
to execute the response (= TER)
Much overlap = Task is difficult; lower drift rate
Little overlap = Task is easy; higher drift rate
Conclusion of the Ratcliff diffusion model: Older adults
are just as efficient in activating lexical content, but are
slower because they are more cautious (i.e. their boundary
separation is wider)
Why the diffusion model is not standardly applied as a
psychometric analysis tool:
1. It may take very many trials to obtain an accurate estimate of the error RT distribution
2. The parameter-fitting procedure is complex, and many experimental psychologists
will find the amount of effort prohibitive
LECTURE W1.1
Behavioural data science: A multidisciplinary field that aims to facilitate understanding,
prediction and change of human behaviour through the analysis of behaviourally defined
variables as they arise in large datasets, typically gathered using modern digital technology
and analysed with techniques for detecting patterns from high-dimensional data
- Multidisciplinary: Operates on the cross-roads of psychology, technology, statistics
and methodology
Skinner: ‘Human behaviour is possibly the most difficult subject ever submitted to statistical
analysis’
