This paper analyzes two distinct educational research designs — a quasi-experimental study and an ex post facto study — by examining their sampling techniques, levels of measurement, statistical methods, and the inferences drawn from their results. The quasi-experimental study investigates the effect of reading strategy instruction on middle school students' science literacy, while the ex post facto study explores professional development and its relationship to school climate and educational outcomes in high-poverty middle grades schools. The paper also evaluates the reliability, validity, and generalizability of each design, highlighting the trade-offs inherent in non-randomized educational research.
The intent of the experimental method — otherwise known as quantitative research or a laboratory study — is to formulate a hypothesis, collect data, and test that hypothesis according to scientific principles that obstruct bias as carefully as possible, and then to analyze the data using statistical measures. The experimental method uses random sampling as a core component of its design.
Practical considerations usually limit the degree of control researchers have when structuring experiments. For instance, it is not always possible to randomize; researchers often must work with the sample of participants already available. This is when quasi-experimentation is employed. This particular study reviewed below is a quasi-experiment.
There were two levels or conditions in this study: (1) an inquiry-based science curriculum, and (2) an inquiry-based science curriculum combined with reading strategy instruction.
Three types of relevant psychological instruments were used to measure science literacy. An ANCOVA (analysis of covariance) was then conducted, using the pretest score as the covariate and the posttest score as the dependent variable.
Even a small amount of reading on science-related topics could positively affect the science literacy of middle school students, and possibly of individuals more generally.
Reliability refers to the consistency or stability of the experimental effect — it must prove capable of replication in future studies, which is closely related to generalizability. Validity refers to the soundness of the study: whether the experiment actually explains what it claims to explain.
The generalizability and reliability of this study are somewhat problematic. Although data were drawn from 10 classes, all classes came from a single school. Characteristics particular to that school — its student body, geographical location, and local context — may prevent the results from being applied to other schools or localities.
Regarding validity, exhaustive methods were taken to prevent bias and to remain as objective as possible. Nonetheless, at least one confounding variable intruded: reading educators and two sixth-grade science teachers were included in the study, which may have introduced uncontrolled influences on outcomes.
"Defining features of retrospective ex post facto design"
"Professional development effects on middle grades outcomes"
"Key distinctions between quasi-experimental and ex post facto"
Always verify citation format against your institution’s current style guide requirements.