Academic Project – Research Methods: The Essential Knowledge Base, Trochim et al.
Chapter 1 – Foundations of research methods
1.1 The research enterprise (KC 1.1, 1.7)
Research influences almost everything.
What is research?
→ a type of systematic investigation that is empirical in nature and is designed to contribute to public
knowledge.
It is empirical (collecting data) and should contribute to a broader base of knowledge than just the
researcher’s knowledge. it is systematic and should be rational and careful. The focus of this book is
on social research. When we conduct a research, we do a research project or study that addresses one
or more specific questions, collects specific data, involves conducting specific analysis, etc.
It is important to understand the broader effort that each research contributes to the research
enterprise: this is the macro-level effort to accumulate knowledge across multiple empirical systematic
public research projects. The most important aspects of this larger enterprise are translational
research, research syntheses and guidelines, evidence-based practice and an evolutionary perspective
on the research enterprise.
Business and management research is transdisciplinary; double hurdle (theoretical + practical);
science-practice gap/translational research; and evidence based management (KC 1.1).
Translational research
→ the systematic effort to move research from initial discovery to practice and ultimately to impacts
on our lives.
This means that research has to be useful or lead to something. Our research contributes to
instrumental knowledge that we hope can make our lives or our world better. That is, knowledge
gained from research may at some point be able to be put into practice.
The research enterprise is encompassing a research-practice continuum (the process of moving from
an initial research idea or discovery to practice, and the potential for the idea to influence our lives or
world) within which translation occurs:
- Basic research: generate discoveries and understand how discoveries work.
- Applied research: testing discovery under increasingly controlled conditions in real-world
contexts.
- Implementation & dissemination research: assess how well an innovation or discovery can be
distributed in and carried out in a broad range of contexts that extend beyond the original
controlled studies.
- Policy research: investigates existing policies or develops and tests new ones.
- Impact research: assesses the broader effect of a discovery or innovation on society.
Victor Roos
,Academic Project – Research Methods: The Essential Knowledge Base, Trochim et al.
Research syntheses and guidelines
→ a systematic study of multiple prior research projects that address the same research question or
topic and that summarizes the results in a manner that can be used by practitioners.
→ a systematic process that leads to a specific set of research-based recommendations for practice
that usually includes some estimates of how strong the evidence is for each recommendation.
There are two major types of research syntheses:
- Meta-analysis: uses statistical methods to combine the results of similar studies quantitatively
in order to allow general conclusions to be made.
- Systematic review: focuses on a specific question or issue and uses pre-planned methods to
identify select, assess, and summarize the findings of multiple studies. A systematic review
may include a meta-analysis.
Because a meta-analysis and/or quantitative synthesis can be too technical/written in a scientific style,
guidelines are developed. The research synthesis is in the middle of the research-practice continuum:
implementation & dissemination research.
Evidence-based practice (EBP)
→ a movement designed to encourage or require practitioners to employ practices that are based on
research evidence as reflected in research syntheses or practice guidelines.
Integration of research and practice is the goal of EBP. However, there are debates about what counts
as evidence: some say studies based on randomized experimental or strong quasi-experimental
designs, other say a broader range of studies. Some also argue that practice helps drive the
development of evidence instead of the other way around.
An evolutionary perspective on the research enterprise
Increasingly we view research as an evolutionary research. This view is based upon the idea of
evolutionary epistemology: the branch of philosophy that holds that ideas evolve through the process
of natural selection.
1.2 Conceptualizing research
Where research topics come from
Ideas for research projects don’t just happen, they come from somewhere, such as:
- Practical problems in the field.
- Literature in your specific field: reading literature and thinking of ways to extend or refine
previous research.
- Requests or proposals (RFP’s): a document issued by a government agency or other
organization that, typically, describes the problem that needs addressing, the context in which
it operates, the approach the agency would like you to take to investigate the problem, and
the amount the agency would be willing to pay for such research.
- Think up your own: influenced by your background, culture, education and experiences.
Victor Roos
,Academic Project – Research Methods: The Essential Knowledge Base, Trochim et al.
The literature review
→ a systematic compilation and written summary of all of the literature published in scientific journals
that is related to a research topic of interest. A literature review is typically included in the introduction
section of a research write-up.
It is designed to identify related research; it doesn’t have to be the same. Tips for literature review are:
- Concentrate your efforts on the research literature. Use research journals that have used a
blind or juried peer review system. These articles have a fairly high level of credibility. A peer
review is a system for reviewing potential research publications where authors submit
potential articles to a journal editor who solicits several reviewers who agree to give a critical
review of the paper. The paper is sent to these reviewers with no identification of the author
so that there will be no personal bias. Based on the reviewer’s recommendations, the editor
can accept the article, reject it, or recommend that the author revise and resubmit it.
- Do the review early. This helps you to determine what the necessary trade-offs are.
In literature review you should look for a study that is similar to yours. It helps you to ensure that you
include all of the major relevant constructs in your studies and it will help you find appropriate
measurement instruments.
Feasibility issues
You usually start thinking about feasibility at the beginning of your study. There are some
considerations. Many of those include trade-offs between rigor and practicality. These are:
- How long will the research take?
- Are there any ethical constraints?
- Can you acquire the cooperation needed?
- To what degree will costs be manageable?
1.3 The language of research (KC 1.3 – 1.5)
Basic research satisfies curiosity. Applied research finds answers to real-life problems (KC 1.3).
Research vocabulary
There are four terms that help describe some of the key aspects for social research:
- Theoretical: pertaining to theory. Social research is theoretical, meaning that much of it is
concerned with developing, exploring, or testing the theories or ideas that social researchers
have about how the world operates.
- Empirical: based on direct observations and measurements of reality.
Research is often a blend of theoretical and empirical.
- Probabilistic: based on probabilities. Even the most accurate of instruments have some
inaccuracy in them.
- Causal: pertaining to a cause-effect relationship, hypothesis, or relationship. Something is
causal if it leads to an outcome or makes an outcome happen.
Types of studies
There are three basic forms of studies (research by evidence, KC 1.3) (all cumulative):
- Descriptive studies: document what is going on or what exists. E.g. public opinion
- Relational studies: investigate the connection between two or more variables. E.g. the relation
between gender and voting preference.
- Causal studies/explanatory studies (KC 1.3): investigate a causal relationship between two
variables. E.g. voting campaign and voting preference.
Victor Roos
, Academic Project – Research Methods: The Essential Knowledge Base, Trochim et al.
Time in research
Time is very important in research. There are cross-sectional studies (studies that take place at a single
point in time) and longitudinal studies (studies that take place over time). Within longitudinal studies
there are two variations: repeated measures (2 – 20 waves of measurement over time) and time series
(many waves of measurement over time, usually ≥ 20).
Types of relationship
A relationship (def. page 39) refers to the correspondence between two variables.
Nature of a relationship
- Correlational relationship: two things perform in a synchronized manner.
- Causal relationship: the same as a correlational relationship + one variable cause the other to
occur.
Correlation does not automatically mean that there is a causal relationship. There could be a third
variable or missing variable problem (an unobserved variable that accounts for a correlation between
two variables). Be careful with interpreting correlations.
With a causal relationship (A causes B), you have to show that: valid, quantitative measures of A and
B are related to one another; A precedes B in time; A is related to B, even when new rule out all possible
causes of B; and understand theoretically why A and B ought to be related.
Pattern of a relationship
There are four types of relationships:
- No relationship. Value: 0.
- Positive relationship: high values for one
variable are associated with high values
on another variable and vice versa.
Value: 0 and +1.
- Negative relationship: high variables for
one variable are associated with low
values on another variable and vice
versa. Value: -1 and 0.
- Curvilinear relationship
Hypotheses
→ a specific statement of prediction
There are two types of hypotheses that can be one-tailed or two-tailed.
- Null hypothesis (H0): the hypothesis that describes the possible outcomes other than HA.
usually the H0 predicts there will be no effect of a program or treatment you are studying.
- Alternative hypothesis (HA): a specific statement of prediction that usually states what you
expect will happen in your study.
- One-tailed hypothesis: def. page 41.
- Two-tailed hypothesis: def. page 42.
The logic op hypothesis testing is based on:
- Two mutually exclusive hypothesis statements that, together, exhaust all possible outcomes,
need to be developed.
- The hypotheses must be tested so that one is necessarily accepted and the other rejected.
Victor Roos
Chapter 1 – Foundations of research methods
1.1 The research enterprise (KC 1.1, 1.7)
Research influences almost everything.
What is research?
→ a type of systematic investigation that is empirical in nature and is designed to contribute to public
knowledge.
It is empirical (collecting data) and should contribute to a broader base of knowledge than just the
researcher’s knowledge. it is systematic and should be rational and careful. The focus of this book is
on social research. When we conduct a research, we do a research project or study that addresses one
or more specific questions, collects specific data, involves conducting specific analysis, etc.
It is important to understand the broader effort that each research contributes to the research
enterprise: this is the macro-level effort to accumulate knowledge across multiple empirical systematic
public research projects. The most important aspects of this larger enterprise are translational
research, research syntheses and guidelines, evidence-based practice and an evolutionary perspective
on the research enterprise.
Business and management research is transdisciplinary; double hurdle (theoretical + practical);
science-practice gap/translational research; and evidence based management (KC 1.1).
Translational research
→ the systematic effort to move research from initial discovery to practice and ultimately to impacts
on our lives.
This means that research has to be useful or lead to something. Our research contributes to
instrumental knowledge that we hope can make our lives or our world better. That is, knowledge
gained from research may at some point be able to be put into practice.
The research enterprise is encompassing a research-practice continuum (the process of moving from
an initial research idea or discovery to practice, and the potential for the idea to influence our lives or
world) within which translation occurs:
- Basic research: generate discoveries and understand how discoveries work.
- Applied research: testing discovery under increasingly controlled conditions in real-world
contexts.
- Implementation & dissemination research: assess how well an innovation or discovery can be
distributed in and carried out in a broad range of contexts that extend beyond the original
controlled studies.
- Policy research: investigates existing policies or develops and tests new ones.
- Impact research: assesses the broader effect of a discovery or innovation on society.
Victor Roos
,Academic Project – Research Methods: The Essential Knowledge Base, Trochim et al.
Research syntheses and guidelines
→ a systematic study of multiple prior research projects that address the same research question or
topic and that summarizes the results in a manner that can be used by practitioners.
→ a systematic process that leads to a specific set of research-based recommendations for practice
that usually includes some estimates of how strong the evidence is for each recommendation.
There are two major types of research syntheses:
- Meta-analysis: uses statistical methods to combine the results of similar studies quantitatively
in order to allow general conclusions to be made.
- Systematic review: focuses on a specific question or issue and uses pre-planned methods to
identify select, assess, and summarize the findings of multiple studies. A systematic review
may include a meta-analysis.
Because a meta-analysis and/or quantitative synthesis can be too technical/written in a scientific style,
guidelines are developed. The research synthesis is in the middle of the research-practice continuum:
implementation & dissemination research.
Evidence-based practice (EBP)
→ a movement designed to encourage or require practitioners to employ practices that are based on
research evidence as reflected in research syntheses or practice guidelines.
Integration of research and practice is the goal of EBP. However, there are debates about what counts
as evidence: some say studies based on randomized experimental or strong quasi-experimental
designs, other say a broader range of studies. Some also argue that practice helps drive the
development of evidence instead of the other way around.
An evolutionary perspective on the research enterprise
Increasingly we view research as an evolutionary research. This view is based upon the idea of
evolutionary epistemology: the branch of philosophy that holds that ideas evolve through the process
of natural selection.
1.2 Conceptualizing research
Where research topics come from
Ideas for research projects don’t just happen, they come from somewhere, such as:
- Practical problems in the field.
- Literature in your specific field: reading literature and thinking of ways to extend or refine
previous research.
- Requests or proposals (RFP’s): a document issued by a government agency or other
organization that, typically, describes the problem that needs addressing, the context in which
it operates, the approach the agency would like you to take to investigate the problem, and
the amount the agency would be willing to pay for such research.
- Think up your own: influenced by your background, culture, education and experiences.
Victor Roos
,Academic Project – Research Methods: The Essential Knowledge Base, Trochim et al.
The literature review
→ a systematic compilation and written summary of all of the literature published in scientific journals
that is related to a research topic of interest. A literature review is typically included in the introduction
section of a research write-up.
It is designed to identify related research; it doesn’t have to be the same. Tips for literature review are:
- Concentrate your efforts on the research literature. Use research journals that have used a
blind or juried peer review system. These articles have a fairly high level of credibility. A peer
review is a system for reviewing potential research publications where authors submit
potential articles to a journal editor who solicits several reviewers who agree to give a critical
review of the paper. The paper is sent to these reviewers with no identification of the author
so that there will be no personal bias. Based on the reviewer’s recommendations, the editor
can accept the article, reject it, or recommend that the author revise and resubmit it.
- Do the review early. This helps you to determine what the necessary trade-offs are.
In literature review you should look for a study that is similar to yours. It helps you to ensure that you
include all of the major relevant constructs in your studies and it will help you find appropriate
measurement instruments.
Feasibility issues
You usually start thinking about feasibility at the beginning of your study. There are some
considerations. Many of those include trade-offs between rigor and practicality. These are:
- How long will the research take?
- Are there any ethical constraints?
- Can you acquire the cooperation needed?
- To what degree will costs be manageable?
1.3 The language of research (KC 1.3 – 1.5)
Basic research satisfies curiosity. Applied research finds answers to real-life problems (KC 1.3).
Research vocabulary
There are four terms that help describe some of the key aspects for social research:
- Theoretical: pertaining to theory. Social research is theoretical, meaning that much of it is
concerned with developing, exploring, or testing the theories or ideas that social researchers
have about how the world operates.
- Empirical: based on direct observations and measurements of reality.
Research is often a blend of theoretical and empirical.
- Probabilistic: based on probabilities. Even the most accurate of instruments have some
inaccuracy in them.
- Causal: pertaining to a cause-effect relationship, hypothesis, or relationship. Something is
causal if it leads to an outcome or makes an outcome happen.
Types of studies
There are three basic forms of studies (research by evidence, KC 1.3) (all cumulative):
- Descriptive studies: document what is going on or what exists. E.g. public opinion
- Relational studies: investigate the connection between two or more variables. E.g. the relation
between gender and voting preference.
- Causal studies/explanatory studies (KC 1.3): investigate a causal relationship between two
variables. E.g. voting campaign and voting preference.
Victor Roos
, Academic Project – Research Methods: The Essential Knowledge Base, Trochim et al.
Time in research
Time is very important in research. There are cross-sectional studies (studies that take place at a single
point in time) and longitudinal studies (studies that take place over time). Within longitudinal studies
there are two variations: repeated measures (2 – 20 waves of measurement over time) and time series
(many waves of measurement over time, usually ≥ 20).
Types of relationship
A relationship (def. page 39) refers to the correspondence between two variables.
Nature of a relationship
- Correlational relationship: two things perform in a synchronized manner.
- Causal relationship: the same as a correlational relationship + one variable cause the other to
occur.
Correlation does not automatically mean that there is a causal relationship. There could be a third
variable or missing variable problem (an unobserved variable that accounts for a correlation between
two variables). Be careful with interpreting correlations.
With a causal relationship (A causes B), you have to show that: valid, quantitative measures of A and
B are related to one another; A precedes B in time; A is related to B, even when new rule out all possible
causes of B; and understand theoretically why A and B ought to be related.
Pattern of a relationship
There are four types of relationships:
- No relationship. Value: 0.
- Positive relationship: high values for one
variable are associated with high values
on another variable and vice versa.
Value: 0 and +1.
- Negative relationship: high variables for
one variable are associated with low
values on another variable and vice
versa. Value: -1 and 0.
- Curvilinear relationship
Hypotheses
→ a specific statement of prediction
There are two types of hypotheses that can be one-tailed or two-tailed.
- Null hypothesis (H0): the hypothesis that describes the possible outcomes other than HA.
usually the H0 predicts there will be no effect of a program or treatment you are studying.
- Alternative hypothesis (HA): a specific statement of prediction that usually states what you
expect will happen in your study.
- One-tailed hypothesis: def. page 41.
- Two-tailed hypothesis: def. page 42.
The logic op hypothesis testing is based on:
- Two mutually exclusive hypothesis statements that, together, exhaust all possible outcomes,
need to be developed.
- The hypotheses must be tested so that one is necessarily accepted and the other rejected.
Victor Roos
