UNIT 1 – What is empirical research? – Babbie Ch. 1 pp. 3-13
Decision making and empirical research
Preparing, making and evaluating decisions in politics, in business, or in any other social
activity is often seen as a sequence of activities, steps or phases. The distinction in phases is
done in various ways, but I think, without any argument to refute 5, 7, 8 or 9 phase
distinctions, it makes sense to distinguish between six of these phases. They are presented
in the following:
1. Identify, describe and analyze a problem or opportunity. Problem definition and –
analysis.
2. Develop options / design options. Design.
3. Compare options, using criteria. Multi-criteria analysis, ex ante evaluation.
4. Select a preferred alternative on the basis of this evaluation. Decision-making rules.
5. Act on the decision / implement the decision. Implementation.
6. Monitor and evaluate the consequences. Ex post evaluation.
How big is the problem?
Problem &
need analysis What are its causes?
Did the selected option have What are its consequences?
the expected outcome?
(outcome evaluation) Ex post choice Find & design
evaluation options Which options have been used
by others?
Did we do as planned? Implementation
Ex ante options
(process evaluation) evaluation Can we expect the option
to work?
Choice
Activities or ‘phases’ like problem analysis, ex ante evaluation, decision-making and
implementation are often done in a rather superficial way, without a lot of thinking or
‘research’. In many cases you do not reflect upon all the goals you may aim for. In most
cases you do not list down all possible options or you decide to not develop a new option.
You never evaluate all options using a large set of criteria. However, by reflecting on the six
phases it is easier to see what the limitations of your choice are. By careful analysis in all
phases, you can more easily defend the choices you have made. And it may be that because
of this, you also make better decisions. Thus, in some cases it makes sense to follow these
steps and make decisions ‘systematically’. You need to train yourself to be at least able to
systematically make decisions, and to be able to reflect on the limitations of a more
superficial choice.
In most phases of decision-making, empirical questions may arise. Empirical questions are
questions that can only be answered by observation.
- Empirical research: systematically answering empirical questions.
o Systematically: excluding the possibility that other answers are better than the
answer we give.
o Empirical: about things we can observe.
annemiekst 1
, Decisions-making and doing empirical research are closely connected activities, although
decision-making is often done without a lot of ‘research’ and a lot of ‘research’ is done
without the explicit intention to use it as part of decision-making. It is therefore not very
useful to oppose decision-making and doing research as alternative activities. It is not doing
research OR making decisions, for we often do research in the context of decision-making.
Opposing research and design makes no sense. And conceptualizing them in a sequential
order (first, we do research, then we design) is also confusing.
Research in the context of decision-making may be applied research, but that is not
necessarily the case. Applied research is ‘using existing knowledge (theories and methods) to
better understand a particular case or problem’. Not all research in the context of decision-
making is applied research, however, because sometimes research questions arise that are
new and require new theories and new ways of studying the topic. Decision-makers cannot
always rely exclusively on existing knowledge.
Theory: a systematic explanation for the observations that relate to a particular aspect of life
URE PROCEDURE
Procedure
Think about possible answers to our
Thinking questions Theory
Decide how we can test what we
Planning Research design
think
Observing
On the base of this planning, we
Data collection
observe things
Analyze the observations, that leads
Analyzing Data analysis
us to conclusions
Wheel of science Question
Not a logistical sequence of steps:
Answers/
anything goes… knowledge
Theory
A way to defend conclusions: logic
Induction Deduction
Data Research
analysis design
Confirmation bias: search for, analyze and recall Data
collection
information in a way that confirms preexisting beliefs,
THREE
while TYPES
giving OF CONFIRMATION
disproportionately BIAS
less consideration to alternative interpretations. It affects
what we think is “true”. Three ways of confirmation bias:
Emotional issues, convictions, sloppiness
Bias in the acquisition of … Bias in … Bias in remembering…
Information Reasoning Conclusions
annemiekst 2
, Why is confirmation bias so strong?
- Limitations in humans, relying on ‘heuristics’
- Wishful thinking
- Consistency (between initial evidence and new evidence)
Consequences of confirmation bias: mistakes in knowledge, bad decisions
Avoiding confirmation bias
- NOT intelligence (no relationship found)
- Systematic data gathering, being aware of what can go wrong
- Explicit and complete reasoning
Basic idea: clarify your pre-existing beliefs (theory), clarify the procedures you will use to test
these beliefs (methods), STICK TO THESE PROCEDURES
Agreement reality: those things we “know” as part and parcel of the culture we share with
those around us.
- Epistemology: the science of knowing; systems of knowledge
- Methodology: the science of finding out; procedures for scientific investigation
UNIT 2 – What are clear research questions? – Babbie Ch. 1 pp. 13-27, Ch. 4 pp. 89-92
An empirical research question is only clear if it refers to meaningful Units of Analysis and
Variables.
- Units of Analysis are the objects the research question is about.
- Variables are the possible characteristics (attributes) of these units.
o Attribute: a characteristic of a person or a thing
o Variable: a logical set of attributes
▪ Independent variable: a variable with values that are not problematical in
an analysis but are taken as simply given. An independent variable is
presumed to cause or determine a dependent variable.
▪ Dependent variable: a variable assumed to depend on or be caused by
another (independent variable).
▪ Note that any given variable might be treated as independent in one part
of an analysis and as dependent in another part of it.
Identifying units and variables
- If variable is known, ask “what or who is characterized by this variable?” -> unit
- If the unit is known, ask “what characteristic does the unit have?” -> variable
- If an attribute of a variable and the unit are known, but not the variable itself, ask “can
other units have other attributes?” -> variable
Unclear empirical research questions: in many vague empirical research questions the units
and variables cannot be meaningfully identified.
If you are very clear about what your units and variables are, you can clarify your questions.
annemiekst 3
Decision making and empirical research
Preparing, making and evaluating decisions in politics, in business, or in any other social
activity is often seen as a sequence of activities, steps or phases. The distinction in phases is
done in various ways, but I think, without any argument to refute 5, 7, 8 or 9 phase
distinctions, it makes sense to distinguish between six of these phases. They are presented
in the following:
1. Identify, describe and analyze a problem or opportunity. Problem definition and –
analysis.
2. Develop options / design options. Design.
3. Compare options, using criteria. Multi-criteria analysis, ex ante evaluation.
4. Select a preferred alternative on the basis of this evaluation. Decision-making rules.
5. Act on the decision / implement the decision. Implementation.
6. Monitor and evaluate the consequences. Ex post evaluation.
How big is the problem?
Problem &
need analysis What are its causes?
Did the selected option have What are its consequences?
the expected outcome?
(outcome evaluation) Ex post choice Find & design
evaluation options Which options have been used
by others?
Did we do as planned? Implementation
Ex ante options
(process evaluation) evaluation Can we expect the option
to work?
Choice
Activities or ‘phases’ like problem analysis, ex ante evaluation, decision-making and
implementation are often done in a rather superficial way, without a lot of thinking or
‘research’. In many cases you do not reflect upon all the goals you may aim for. In most
cases you do not list down all possible options or you decide to not develop a new option.
You never evaluate all options using a large set of criteria. However, by reflecting on the six
phases it is easier to see what the limitations of your choice are. By careful analysis in all
phases, you can more easily defend the choices you have made. And it may be that because
of this, you also make better decisions. Thus, in some cases it makes sense to follow these
steps and make decisions ‘systematically’. You need to train yourself to be at least able to
systematically make decisions, and to be able to reflect on the limitations of a more
superficial choice.
In most phases of decision-making, empirical questions may arise. Empirical questions are
questions that can only be answered by observation.
- Empirical research: systematically answering empirical questions.
o Systematically: excluding the possibility that other answers are better than the
answer we give.
o Empirical: about things we can observe.
annemiekst 1
, Decisions-making and doing empirical research are closely connected activities, although
decision-making is often done without a lot of ‘research’ and a lot of ‘research’ is done
without the explicit intention to use it as part of decision-making. It is therefore not very
useful to oppose decision-making and doing research as alternative activities. It is not doing
research OR making decisions, for we often do research in the context of decision-making.
Opposing research and design makes no sense. And conceptualizing them in a sequential
order (first, we do research, then we design) is also confusing.
Research in the context of decision-making may be applied research, but that is not
necessarily the case. Applied research is ‘using existing knowledge (theories and methods) to
better understand a particular case or problem’. Not all research in the context of decision-
making is applied research, however, because sometimes research questions arise that are
new and require new theories and new ways of studying the topic. Decision-makers cannot
always rely exclusively on existing knowledge.
Theory: a systematic explanation for the observations that relate to a particular aspect of life
URE PROCEDURE
Procedure
Think about possible answers to our
Thinking questions Theory
Decide how we can test what we
Planning Research design
think
Observing
On the base of this planning, we
Data collection
observe things
Analyze the observations, that leads
Analyzing Data analysis
us to conclusions
Wheel of science Question
Not a logistical sequence of steps:
Answers/
anything goes… knowledge
Theory
A way to defend conclusions: logic
Induction Deduction
Data Research
analysis design
Confirmation bias: search for, analyze and recall Data
collection
information in a way that confirms preexisting beliefs,
THREE
while TYPES
giving OF CONFIRMATION
disproportionately BIAS
less consideration to alternative interpretations. It affects
what we think is “true”. Three ways of confirmation bias:
Emotional issues, convictions, sloppiness
Bias in the acquisition of … Bias in … Bias in remembering…
Information Reasoning Conclusions
annemiekst 2
, Why is confirmation bias so strong?
- Limitations in humans, relying on ‘heuristics’
- Wishful thinking
- Consistency (between initial evidence and new evidence)
Consequences of confirmation bias: mistakes in knowledge, bad decisions
Avoiding confirmation bias
- NOT intelligence (no relationship found)
- Systematic data gathering, being aware of what can go wrong
- Explicit and complete reasoning
Basic idea: clarify your pre-existing beliefs (theory), clarify the procedures you will use to test
these beliefs (methods), STICK TO THESE PROCEDURES
Agreement reality: those things we “know” as part and parcel of the culture we share with
those around us.
- Epistemology: the science of knowing; systems of knowledge
- Methodology: the science of finding out; procedures for scientific investigation
UNIT 2 – What are clear research questions? – Babbie Ch. 1 pp. 13-27, Ch. 4 pp. 89-92
An empirical research question is only clear if it refers to meaningful Units of Analysis and
Variables.
- Units of Analysis are the objects the research question is about.
- Variables are the possible characteristics (attributes) of these units.
o Attribute: a characteristic of a person or a thing
o Variable: a logical set of attributes
▪ Independent variable: a variable with values that are not problematical in
an analysis but are taken as simply given. An independent variable is
presumed to cause or determine a dependent variable.
▪ Dependent variable: a variable assumed to depend on or be caused by
another (independent variable).
▪ Note that any given variable might be treated as independent in one part
of an analysis and as dependent in another part of it.
Identifying units and variables
- If variable is known, ask “what or who is characterized by this variable?” -> unit
- If the unit is known, ask “what characteristic does the unit have?” -> variable
- If an attribute of a variable and the unit are known, but not the variable itself, ask “can
other units have other attributes?” -> variable
Unclear empirical research questions: in many vague empirical research questions the units
and variables cannot be meaningfully identified.
If you are very clear about what your units and variables are, you can clarify your questions.
annemiekst 3
