Researchers consider validity and reliability with each new study they design. This is because validity and reliability are not fixed but rather reflect a particular study’s unique variables, research design, instruments, and participants.
In the context of research design, two types of validity, which speak to the quality of different features of the research process, are considered: internal validity and external validity. Assuming that the findings of a research study are internally valid—i.e., the researcher has used controls to determine that the outcome is indeed due to manipulation of the independent variable or the treatment—external validity refers to the extent to which the findings can be generalized from the sample to the population or to other settings and groups. Reliability refers to the replicability of the findings.
For this Discussion, you will consider threats to internal and external validity in quantitative research and the strategies used to mitigate these threats. You will also consider the ethical implications of designing quantitative research.
Post an explanation of a threat to internal validity and a threat to external validity in quantitative research. Next, explain a strategy to mitigate each of these threats. Then, identify a potential ethical issue in quantitative research and explain how it might influence design decisions. Finally, explain what it means for a research topic to be amenable to scientific study using a quantitative approach.
Be sure to support your Main Issue Post and Response Post with reference to the week’s Learning Resources and other scholarly evidence in APA Style.
Babbie, E. (2017) Basics of social research (7th ed.). Boston, MA: Cengage Learning.
Chapter 3, “The Ethics and Politics of Social Research”
Burkholder, G. J., Cox, K. A., Crawford, L. M., & Hitchcock, J. H. (Eds.). (2020). Research designs and methods: An applied guide for the scholar-practitioner. Thousand Oaks, CA: Sage.
Chapter 12, “Quality Considerations”
Chapter 13, “Ethical Considerations”
Research Theory, Design, and Methods Walden University
© 2016 Laureate Education, Inc. Page 1 of 2
Threats to Internal Validity
(Shadish, Cook, & Campbell, 2002)
1. Ambiguous temporal precedence. Based on the design, unable to determine
with certainty which variable occurred first or which variable caused the other.
Thus, unable to conclude with certainty cause-effect relationship. Correlation
of two variables does not prove causation.
2. Selection. The procedures for selecting participants (e.g., self-selection or
researcher sampling and assignment procedures) result in systematic
differences across conditions (e.g., experimental-control). Thus, unable to
conclude with certainty that the “intervention” caused the effect; could be due
to way in which participants are selected.
3. History. Other events occur during the course of treatment that can interfere
with treatment effects and could account for outcomes. Thus, unable to
conclude with certainty that the “intervention” caused the effect; could be due
to some other event to which the participants were exposed.
4. Maturation. Natural changes that participants experience (e.g., grow older,
get tired) during the course of the intervention could account for the
outcomes. Thus, unable to conclude with certainty that the “intervention”
caused the effect; could be due to the natural change/maturation of the
participants.
5. Regression artifacts. Participants who are at extreme ends of the measure
(score higher or lower than average) are likely to “regress” toward the mean
(scores get lower or higher, respectively) on other measures or retest on
same measure. Thus, regression can be confused with treatment effect.
6. Attrition (mortality). Refers to dropout or failure to complete the
treatment/study activities. If differential dropout across groups (e.g.,
experimental-control) occurs, could confound the results. Thus, effects may
be due to dropout rather than treatment.
7. Testing. Experience with test/measure influences scores on retest. For
example, familiarity with testing procedures, practice effects, or reactivity can
influence subsequent performance on the same test.
8. Instrumentation. The measure changes over time (e.g., from pretest to
posttest), thus making it difficult to determine if effects or outcomes are due to
instrument vs. treatment. For example, observers change definitions of
behaviors they are tracking, or the researcher alters administration of test
items from pretest to posttest.
9. Additive and interactive effects of threats to validity. Single threats interact,
such that the occurrence of multiple threats has an additive effect. For
example, selection can interact with history, maturation, or instrumentation.
Research Theory, Design, and Methods Walden University
© 2016 Laureate Education, Inc. Page 2 of 2
Reference
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-
experimental designs for generalized causal inference. Boston, MA:
Houghton-Mifflin.
Threats to Internal Validity
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton-Mifflin.
Litmus Test for a Doctoral-Level Research Problem
Background on these “litmus test” questions
· The distinguishing characteristic of doctoral-level research (as opposed to masters level) is that it must make an original contribution to the field. However, students may struggle to identify what will authentically contribute to their field or discipline.
· The most critical step in making such a contribution is to first identify a research problem with the 4 doctoral hallmarks below. Identifying a doctoral-level research problem is “necessary, but not sufficient,” to produce doctoral-level capstone.
REQUIRED DOCTORAL HALLMARKS OF THE RESEARCH PROBLEM
In Walden’s scholar-practitioner model, a research problem shows promise of contributing meaningfully to the field ONLY if the answer to ALL of the following questions is “yes.”
Yes
No
1. JUSTIFIED?
Is there evidence that this problem is significant to the professional field?
There must be relevant statistics (expressing an unjust inequality, financial impact, lost efficiency, etc.), documentable discrepancies (e.g., two models that are difficult to reconcile), and/or other scholarly facts that point to the significance and urgency of the problem. The problem must be an authentic “puzzle” that needs solving, not merely a topic that the researcher finds interesting.
2. GROUNDED IN THE LITERATURE?
Can the problem be framed in a way that will enable the researcher to either build upon or counter the previously published findings on the topic?
For most fields, this involves articulating the problem within the context of a theoretical or conceptual framework. Although there are multiple ways to ground a study in the scientific literature, the essential requirement is that the problem is framed in such a way that the new findings will have implications for the previous findings.
3. ORIGINAL?
For research doctorates (Ph.D.):
Does the problem reflect a meaningful gap in the research literature?
For the professional doctorates (Ed.D. and D.B.A.):
Does the problem describe a meaningful gap in practice?
4. AMENABLE TO SCIENTIFIC STUDY?
Can a scholarly, systematic method of inquiry be applied to address the problem?
The framing of the problem should not reveal bias or present a foregone conclusion. Even if the researcher has a strong opinion on the expected findings, scholarly objectivity must be maximized by framing the problem in the context of a systematic inquiry that permits multiple possible conclusions.
