Responsible Research in Practice Summary

Responsible Research in Practice Lecture 1

Questions to ask for interview:
Did you formulate your hypothesis before or after looking into data?

Building on previous work, generating and specifying hypotheses:
1. Integrity of existing research
2. Reproducibility and replicability and robustness (3 R’s)
3. Specifying hypotheses

Integrity of existing research
 Values and behaviors of researchers
o Daily decisions of researchers and their practices
o These decisions and behaviors determine what ends up in our literature
 Integrity of the literature

Why bother with codes, education…?
 Codes will not apply to all of the complex daily decisions researchers make
o Education, continued calibration of moral compass, community.
 Professional standards (e.g. for research integrity) are updated over time and develop
with changes in the system.
 Mentorship and leadership  give everyone involved background and context
o E.g. transferring codes of conduct, setting good example, being transparent
about how researchers do things
 Reflection on our values will make it easier to react when future dilemmas arise e.g.
conflict of interest
 To start the conversation about how you personally, and as part of a team, relate to the
topics we discuss
o Make values explicit, we tend to overestimate how much other people share the
same values as us.

Responsible scholarship =
1. Integrity  daily decisions as a researcher/scholar,
2. Ethics  how you behave with participants,
3. Open science
-Conducting our work with integrity, and meeting the needs for better quality and efficiency (no
research waste!  publishing null findings too so people are aware of methods/effects that do
not work) in psychological science.


Why does responsible scholarship matter?
 Sfaeguards the quality and progress of science and its application
 Enables trust among members of the scientific community
 Safeguards the reputation of science

,  Fosters equal opportunities and outcomes (equity)
 Prevents research waste (taxpayers’ money)
 Build robust commutative psych science


Consider the goals of science vs. the pressures of the system
 Goal of science vs. goal of individual (if these overlap, you are intrinsically motivated to
perform good science)
 Consider competitiveness, hectic pace, “publish or perish” culture, external funding or
expulsion

Maybe responsible scholarship required the ability to be self-correcting. Science should be self-
correcting. E.g. when theories don’t hold up, we should try to update it, be transparent and
open to have peers to correct our work.




In order for change to occur, need to first provide the infrastructure to even make it possible to
spur and commit to the changes.

Mertonian Norms
 Communalism = common ownership of scientific discoveries – need for public sharing
 Universalism = science regardless of who (race, nationality, gender etc) and everyone’s
claims assessed objectively
 Disinterestedness= scholars work for the benefit of a common scientific goal, rather
than personal gain
 Organized skepticism = scholarly work should be open to critical and organized scrutiny
of scientific community.  self-correcting behavior
Principles from code of conduct for research integrity (RI)

,  Honesty
o Did I report the research process accurately?
 Scrupulousness
o Did I take the best possible care in designing, undertaking and reporting my
research?
 Transparency
o Is it clear to others how data were obtained and results were achieved?
 Independence
o Was my research guided by non-scientific/non-scholarly considerations?
 Responsibility
o Is my research relevant? Did I take into account the interests of my subjects?

Distortions to integrity of psychology as a field
 Scientific misconduct
 Questionable research practices
 Poor research practices (Competence)
 Honest errors (fallibility)
These daily gray areas are problematic but they are also the area where we can spur the most
change and improvement.

Methods + practices = research integrity of literature


Replicability = testing the reliability of a prior finding with different data

Robustness = refers to testing the reliability of a prior finding using the same data and a
different analysis strategy (different routes). If no effects are found using different strategies, it
is likely a very fragile effect.

, Reproducibility = testing the reliability of a prior finding using the same data and the same
analysis strategy – basically rerunning the study. Can I even run the script?




Replication
“Many labs” projects
 Different types of replications e.g. direct replication, direct replication with an
extension, conceptual replication e.g. once in children and then perform same study in
adults in order to see whether we can generalize findings to adults. Each type of
replication has a different function.
 Iterative replication: start with direct replication  direct replication with extension 
conceptual replication (Repeat)

Large-scale replication
 Replication projects reveal we have a problem
 Large-scale collaborations and independent replications can work towards solving the
problem
 We need to invest more in figuring out what replication means and how to do it well
 More replications are needed, but how do we select what is worthy of replication? We
cannot repeat everything!

Existing literature has historically had small sample sizes. This suggests we cannot be sure about
what effect is being studied and if it really holds. Therefore, it would be ideal to replicate these
studies.