True Experimental Designs: The Gold Standard in Educational Research

When an educational researcher wants to confidently say that a teaching method caused better student outcomes – not just that the two things happened at the same time – they need a research design powerful enough to rule out all other possible explanations. That is precisely what true experimental designs do. Recognised as the most rigorous form of quantitative research, these designs give researchers the tools to manipulate one variable, control for others, and draw credible cause-and-effect conclusions. This post unpacks the defining features of true experimental designs and walks through their three main types used in educational research.

What makes a design truly “experimental”?
Why internal validity matters
The three main types of true experimental designs
1. Pre-test post-test control group design
2. Post-test only control group design
3. Solomon four groups design
Strengths and limitations of true experimental designs
Choosing the right design

What makes a design truly “experimental”?

Not every study that tests an intervention qualifies as a true experiment. According to the foundational work of Campbell and Stanley (1966), experimental research designs fall into three categories – pre-experimental, quasi-experimental, and true experimental – with the key differentiator being the use of randomisation. A true experimental design has four non-negotiable elements: manipulation of the independent variable, use of a control group, random assignment, and control of extraneous variables.

Random assignment is the cornerstone. It means every participant has an equal chance of being placed in either the experimental or control group – not based on ability, preference, or any other factor. This single step is what separates a true experiment from a quasi-experiment, and it is what allows researchers to say, with confidence, that differences in outcomes were caused by the intervention and not by pre-existing group differences.

Why internal validity matters

The entire purpose of true experimental design is to protect internal validity – the degree to which a study can establish a genuine cause-and-effect relationship. Random assignment and control groups reduce bias, and the deliberate manipulation of the independent variable allows for precise measurement of its impact on the dependent variable. When internal validity is high, a researcher can be genuinely confident that tutoring caused the improvement in test scores, not the fact that more motivated students happened to seek tutoring.

Beyond internal validity, researchers also consider external validity – whether findings can be generalised beyond the study setting. As we will see, some true experimental designs handle this better than others.

The three main types of true experimental designs

There are three designs that meet all the criteria for a true experiment. Each one builds on the last, addressing the limitations left open by its predecessor.

1. Pre-test post-test control group design

This is the most commonly used true experimental design in educational research. Participants are randomly assigned to either a treatment group or a control group. Both groups are measured on the dependent variable before the intervention (the pre-test), the experimental group receives the treatment, and then both groups are measured again (the post-test). The effect of the intervention is calculated as the difference in pre-test to post-test change between the two groups.

Consider a straightforward example: a school wants to test whether a structured phonics programme improves reading comprehension in Grade 3 students. Students are randomly assigned to the phonics group or the regular curriculum group. Both groups take a reading assessment at the start of the term (pre-test) and at the end (post-test). Any gain in the phonics group that is not seen in the control group can be attributed to the programme.

This design controls for many key threats to internal validity, including maturation (students would be growing older regardless), history (external events affect both groups equally), and selection bias (eliminated by random assignment). The pre-test also helps control for attrition bias – the unequal loss of participants between groups over time – by establishing baseline comparability.

Its main limitation is a phenomenon called the testing effect. Taking a pre-test can itself influence how participants respond to the intervention. Students who answer a reading comprehension pre-test may become more attuned to the kinds of questions being asked, and that awareness – not the phonics programme – could partly explain their improved post-test scores. This is a threat to both internal and external validity, and it is the problem the next two designs set out to solve.

2. Post-test only control group design

This design takes a simpler approach: it eliminates the pre-test entirely. Participants are randomly assigned to the experimental or control group, the treatment is administered, and outcomes are measured only once – after the intervention. The difference in post-test scores between the two groups is taken as the effect of the treatment.

At first glance, removing the pre-test seems like a weakness. If there is no baseline, how can researchers be sure the groups were equivalent to begin with? The answer lies in random assignment. When researchers randomise learners into intervention and control groups, they are statistically controlling for all other potential influences – demographic characteristics, prior knowledge, and motivation – because these factors are distributed roughly equally across both groups by chance. With a sufficiently large sample, random assignment makes a pre-test unnecessary to establish initial equivalence.

The practical advantages are significant. This design is relatively inexpensive and eliminates the threats associated with pre-testing. It is especially useful when the act of measuring participants before the intervention would itself alter their behaviour – for example, a survey about attitudes toward a new classroom technology might make students more alert to the technology when it is introduced, contaminating the results. By skipping the pre-test, this design avoids that problem entirely.

Its main drawback is the absence of baseline data. Without a pre-test, there is no way to check whether random assignment successfully produced equivalent groups, which can be a concern with smaller sample sizes. In large-scale studies, this is rarely a problem; in smaller classroom-based research, it warrants more caution.

3. Solomon four groups design

The Solomon four groups design is the most comprehensive – and most demanding – of the three. Developed by psychologist Richard L. Solomon in 1949, it was specifically created to address a limitation that the previous two designs cannot fully resolve: the interaction between the pre-test and the treatment itself.

The design works by combining the two designs above into a single study with four randomly assigned groups:

Group 1: Pre-test → Treatment → Post-test
Group 2: Pre-test → No treatment → Post-test
Group 3: No pre-test → Treatment → Post-test
Group 4: No pre-test → No treatment → Post-test

Pretest sensitisation occurs when a pre-test increases or decreases how responsive participants are to the study’s intervention. If students in the pre-tested groups perform differently from those in the non-pre-tested groups – even when both received the same treatment – it means the pre-test itself was influencing outcomes. The Solomon design makes this interaction visible and measurable.

By comparing Group 1’s post-test with Group 3’s post-test, the researcher can determine how much the pre-test affected the treatment group. By comparing Group 2’s post-test with Group 4’s post-test, they can assess whether the pre-test influenced the control group independently. This design employs a combination of pretest-posttest design and posttest-only design to combat threats to both internal and external validity that simpler designs leave unresolved.

To illustrate: suppose a university is evaluating a new academic writing software. Students who took a pre-test on writing skills might already be primed to notice and engage with the software differently than students who had no pre-test. The Solomon design would allow the research team to statistically separate the software’s genuine effect from any pre-test-induced alertness.

The statistical analysis typically uses a two-factor analysis of variance (ANOVA), with treatment and presence of pre-test as the two factors and post-test scores as the dependent variable. A significant interaction between treatment and pre-test indicates pretest sensitisation, which would compromise the external validity of a simpler two-group design.

The Solomon design is, by most accounts, one of the benchmarks for sociological and educational research, capable of addressing most internal and external validity threats simultaneously. Its major drawback, however, is practical: it requires roughly twice as many participants, considerably more resources, and far more complex statistical handling than a standard two-group design. As a result, most researchers opt for simpler designs while acknowledging their limitations, reserving the Solomon design for high-stakes or well-funded studies where the risk of pretest sensitisation is judged to be significant.

Strengths and limitations of true experimental designs

Across all three designs, true experiments share a core set of strengths. They offer the highest level of internal validity available in social science research. Only a true experimental design can establish a genuine cause-effect relationship within a group – no other quantitative method comes close. Their rigorous structure also supports replicability, meaning other researchers can run the same study in different settings and compare results.

At the same time, true experiments have real-world constraints. Random assignment is not always ethically or logistically possible in educational settings – you cannot always randomly assign students to classrooms or schools without disrupting existing structures. External validity can be limited if the controlled study environment does not reflect the complexity of real classrooms. And these designs, particularly the Solomon four groups model, can be costly in time, participants, and funding. Researchers must weigh these trade-offs carefully when choosing a design.

Choosing the right design

The choice between the three true experimental designs comes down to the specific concerns of the study. If establishing a clear baseline is important and the risk of testing effects is low, the pre-test post-test control group design is the most practical and widely used option. If the pre-test itself might contaminate the results and sample sizes are large enough for random assignment to do its work, the post-test only design is cleaner and more efficient. When the research question demands the highest possible confidence in both internal and external validity – and resources allow – the Solomon four groups design provides the most comprehensive answer.

What all three have in common is their commitment to a fundamental principle: that credible evidence about what works in education requires deliberate control, careful structure, and genuine randomisation. Without these, a study can describe what happened, but it cannot say why.

What do you think? In your experience or field of study, which of the three true experimental designs seems most feasible to implement in a real school setting – and what would be the biggest practical barrier? If a researcher suspects that administering a pre-test might itself change how students engage with a new teaching method, which design would you recommend they use, and why?

How useful was this post?

Click on a star to rate it!

Average rating 5 / 5. Vote count: 1

No votes so far! Be the first to rate this post.

References

PDF 📄

Table of Contents

What makes a design truly “experimental”? 🔗

Why internal validity matters 🔗

The three main types of true experimental designs 🔗

1. Pre-test post-test control group design 🔗

2. Post-test only control group design 🔗

3. Solomon four groups design 🔗

Strengths and limitations of true experimental designs 🔗

Choosing the right design 🔗